Aviation accident text data extraction method based on cyclic BERT and corner labeling

Abstract

The present invention belongs to the field of natural language processing technology, and specifically relates to an aviation accident text data extraction method based on cyclic BERT and corner labeling. The method comprises the following steps: S1, using a double-loop BERT model to identify trigger words of aviation accident report text data; S2, using a classifier model to jointly calculate argument vectors of the aviation accident report text data; S3, calculating the total confidence of the text data argument vector based on the vectors of the text data trigger words and the argument labels; S4, using a corner labeling matrix to decode event triples of the aviation accident report text data to obtain a total loss value of the text data; S5, judging and extracting the aviation accident report text data based on the total loss value. The present invention uses a double-loop training model to identify contextual information and argument positions, and then jointly calculates argument confidence to reduce redundant errors and cascade errors, increase the accuracy of triple prediction, and use a corner labeling method to reduce model complexity and increase the comprehensiveness of the extraction results.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to an aviation accident text data extraction method based on cyclic BERT and corner marking. Background Art

[0002] During aviation accident investigation and analysis, aviation accident report texts describe the accident process, causes, and related events. This wealth of information is crucial for accident investigation and safety improvements. However, these texts are typically stored in a discrete, unstructured format, exhibiting complex structures and a large amount of information. This makes it difficult to establish strong correlations, limiting data analysis and making it difficult to gain a deeper understanding of the root causes and potential risks. Therefore, there is an urgent need to utilize effective information extraction techniques to accurately and efficiently extract key event information from unstructured aviation accident texts.

[0003] Deep learning-based knowledge extraction technology utilizes deep learning models to identify entities, relations, attributes, and events in text, storing them in the form of (subject, relation, object) triples (SROs). This structured representation of textual information can be used for tasks such as building knowledge graphs, semantic understanding and reasoning, and data mining and analysis. Traditional information extraction typically employs a pipeline approach, dividing it into two independent subtasks: named entity recognition (NER) followed by relation extraction (RE). While this approach offers modeling flexibility, it suffers from issues such as error accumulation, lack of information exchange between subtasks, information redundancy, and difficulty extracting long dependencies. With the advancement of deep learning technology, scholars have proposed joint extraction techniques based on deep learning, aiming to simultaneously extract entities and relations using a single model to address the challenges of pipeline approaches. For example, CasRel (joint decoding-based entity relation extraction model) is an end-to-end binary annotation framework that addresses the problem of discretizing entity pairs by mapping a function from subject to object. TPLinker (a single-step entity-relation joint extraction model based on entity linking) is a single-stage relation extraction model that solves the joint extraction of entity relations through token pair linking. PRGC (joint relation triple extraction based on latent relations and global correspondence) uses relation-specific sequence annotation to handle overlapping entities and reduces redundancy by predicting latent relations. OneRel (a single-module single-step entity-relation joint extraction model) uses corner labeling to achieve a single-module single-step extraction of entities and relations.

[0004] While these models have demonstrated promising performance on general-purpose datasets, extracting knowledge from aviation accident text remains challenging. This is primarily due to the large amount of underlying logic and entity pair overlap in aviation accident reports, leading to a lack of targeted knowledge extraction models. Therefore, in order to effectively extract structured data from aviation accident report text, it is necessary to develop knowledge extraction models suitable for this domain. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides an aviation accident text data extraction method based on cyclic BERT and corner labeling. Through the double-loop pre-training model, it can accurately identify contextual information and argument positions, and then reduce redundant errors and cascade errors by jointly calculating argument confidence, thereby increasing the accuracy of triple prediction. The corner labeling method is used to reduce model complexity and increase the comprehensiveness and accuracy of the extraction results.

[0006] To achieve the above objectives, the present invention discloses the following technical solutions:

[0007] A method for extracting aviation accident text data based on cyclic BERT and corner tagging, comprising:

[0008] S1: Using a dual-loop BERT model to identify trigger words in aviation accident report text data;

[0009] Obtain text data from aviation accident reports, embed feature vectors of the text data, encode the embedded vectors of the text data into label sequences, and embed and annotate the trigger word positions of the text data. Perform secondary encoding processing on the feature vectors of the position embedding using the BERT model, identify the argument vectors, and complete the text data trigger word training. Specifically,

[0010]

[0011] in, is the vector of the trigger word in the text data; softmax is the function mapping relationship; W trigger The weight of the context information vector of the trigger word in the text data; h i is the vector of the context information of the i-th trigger word; b trigger is the vector deviation of the trigger word in the text data; i is the number of the aviation accident report text data;

[0012] S2: Use the classifier model to jointly calculate the argument vector of the text data of aviation accident reports;

[0013] Use double-loop BERT to jointly represent the feature vector, calculate the probability of the text data argument vector through the classifier model, and combine the text data with the argument head word probability and the probability of the tail word of the joint argument Combining them, the predicted text data meta-label is:

[0014]

[0015] in, is the i-th argument vector of the predicted text data; is the probability of the head word of the i-th argument; is the probability of the tail word of the i-th argument;

[0016] S3: Calculate the total confidence CN of the text data argument vector based on the vector of the text data trigger word in step S1 and the text data argument label in step S2:

[0017]

[0018] Among them, CN is the total confidence of the text data argument vector; Π is the product operation; Confidence is the confidence calculation function mapping;

[0019] S4: Decoding the aviation accident event triples of the aviation accident report text data using the corner labeling matrix to obtain the total loss value of the text data;

[0020] S41: The argument vector with the highest total confidence CN of the text data argument vector in step S3 is stored in the corner labeling matrix R for decoding feature labeling; the aviation accident event triplet feature value of the text data is calculated using the corner labeling matrix assignment function as follows:

[0021]

[0022] in, is the eigenvalue of the aviation accident event triple of the text data; R is the corner label matrix of the aviation accident event triple; is the vector of the subject word in the aviation accident event triple; O is the vector of the object word in the aviation accident event triple; is the cyclic calculation of subject words and object words; φ is the ReLU activation function mapping; W is the weight parameter for classifier model training; is the concatenation operation of the subject word vector and the object word vector; b is the bias parameter for classifier model training;

[0023] S42: Predict the loss value L based on the trigger word trigger , argument prediction loss value L arg And the corner mark prediction loss value L rel The weighted summation is used to obtain the total loss value L of the text data. total for:

[0024] L total =αLtrigger +βL arg +γL rel ;

[0025] Among them, L total is the total loss value of text data; L trigger is the trigger word prediction loss value; L arg Predict the loss value for the argument; L rel is the corner mark prediction loss value; α is the trigger word prediction parameter; β is the argument prediction parameter; γ is the corner mark prediction parameter;

[0026] S5: For the aviation accident event triplet feature value of the text data in step S41, set the error output threshold; when the total loss value L of the text data output in step S42 is total When the error is greater than or equal to the error output threshold, the weight matrix parameters and bias item parameters of the classifier model in step S2 are updated using the back propagation algorithm, and the process returns to step S2 to continue. When the total loss value L of the text data output in step S42 is less than or equal to the error output threshold, the process returns to step S2 to continue the process. total When the error is lower than the error output threshold, the aviation accident event triplet in step S41 is output as the extracted aviation accident report text data.

[0027] Preferably, in step S1, text data feature vector embedding is performed, the embedding vector of the text data is encoded into a label sequence, and the trigger word position embedding annotation of the text data is performed, specifically as follows:

[0028] S11: Embed the text data of the aviation accident report into feature vectors; embed the text data into entity word embedding, segment embedding, and position embedding to form the embedding vector of the text data:

[0029] x i =e i +p i +s i ;

[0030] Among them, x i is the embedding vector of text data; e i is the entity word embedding vector of text data; p i is the segment embedding vector of text data; i Embed vectors for the positions of text data;

[0031] S12: Encode the embedding vector of the text data in step S11 into a label sequence to implement training of entity words, trigger words, and context information in the text data of the aviation accident report:

[0032] h i =BERT(x i );

[0033] Among them, h i is the vector of the contextual information of the i-th trigger word; BERT is the multi-layer Transformer hidden function mapping;

[0034] S13: The trigger word position embedding annotation of the text data of the aviation accident report; when BERT is used for processing for the first time, the position embedding mark in the text data is 0. After processing through multiple layers of hidden units, the trigger word and the trigger word position are identified; when BERT is used for processing for the second time, the trigger word position embedding mark in the text data is 1.

[0035] Preferably, in step S2, a double-loop BERT is used to jointly represent the feature vectors, and the probability of the text data argument vector is calculated by the classifier model, specifically:

[0036] S21: Use the double-loop BERT model to jointly represent the feature vectors, and obtain the joint feature vector of the text data argument vector as follows:

[0037] J i ={B1(w1+e1),…,B i (w i +e i ),…B m (w m +e m )};

[0038] Among them, J i is the i-th joint feature vector of the text data argument vector; B i is the i-th embedding vector of the context information of the text data; w i is the i-th word vector of the text data; m is the total number of joint eigenvectors of the text data argument vectors;

[0039] S22: The probability of determining the text data argument vector through the classifier model is:

[0040]

[0041] in, Combined argument head word probability for text data; The probability of the tail word of the joint argument for text data; is the weight of the head word of the text data argument; is the weight of the tail word of the text data argument; J s is the joint feature vector of the head word of the text data argument; J e is the joint feature vector of the tail words of the text data argument; B is the embedding vector of the context information of the text data; w is the word vector of the text data; e is the embedding vector of the entity word of the text data.

[0042] Preferably, the decoding feature flags in step S41 include: a first decoding feature flag HB-TB, a second decoding feature flag HB-TE, a third decoding feature flag HE-TE, a fourth decoding feature flag Minus, and a fifth decoding feature flag Null;

[0043] The first decoding feature mark HB-TB is decoded from the character corresponding to the start position of the subject word to the character corresponding to the start position of the object word;

[0044] The second decoding feature mark HB-TE is decoded from the character corresponding to the start position of the subject word to the character corresponding to the end position of the object word;

[0045] The third decoding feature mark HE-TE is decoded from the character corresponding to the end position of the subject word to the character corresponding to the end position of the object word;

[0046] The fourth decoding feature mark Minus is to decode in sequence according to the subword position order;

[0047] The fifth decoding feature flag Null is a word that does not need to be decoded and output.

[0048] Preferably, the trigger word prediction loss value L in step S42 is trigger Specifically, the trigger word loss value is calculated by the cross entropy loss function:

[0049]

[0050] Among them, L trigger is the prediction loss value of the trigger word; N is the number of trigger word samples; C is the number of trigger word categories; is the real label of the trigger word; is the predicted probability of the trigger word; c is the category number of the trigger word.

[0051] Preferably, the argument prediction loss value L in step S42 is arg , specifically:

[0052] The argument prediction loss model is used to measure the aviation accident event argument. The equations for the predicted loss value at the beginning of the argument and the predicted loss value at the end of the argument are as follows:

[0053]

[0054] in, Predict the loss value for the starting position of the argument; Predict the loss value for the end position of the argument; is the real label of the starting position of the argument; Predict the probability of the starting position of the argument; is the actual label of the end position of the argument; Predict the probability of the end position of the argument;

[0055] Combining the predicted loss value of the starting position and the predicted loss value of the ending position of each argument, the total predicted loss value of the aviation accident event argument is:

[0056]

[0057] Among them, A is the set of all arguments; L arg Predict the loss value for the argument.

[0058] Preferably, the corner mark prediction loss value L in step S42 is rel Specifically, the corner label matrix prediction loss is used to measure the difference between the corner label matrix extracted by the double-loop BERT model and the actual result. The specific formula is:

[0059]

[0060] Among them, L rel is the corner mark prediction loss value; K is the number of relationship categories; is the true label of the k-th category relationship between the i-th entity and the j-th entity; is the predicted probability of the corner label matrix; k is the relationship category number.

[0061] Preferably, the error output threshold is set in step S5, specifically:

[0062] S511: Calculate the error term of the output layer of the classifier model:

[0063]

[0064] Among them, δ(L total ) is the error term of the output layer of the classifier model; is the total loss function L total The weighted input z(L total )’s partial derivatives; is the predicted probability of the classifier model; y is the true label of the classifier model;

[0065] S512: Calculate the error term of the hidden layer of the classifier model:

[0066]

[0067] Wherein, δ(l) is the error term of the classifier model at layer l; W(l+1) is the weight of the classifier model at layer l+1; δ(l+1) is the error value of the classifier model at layer l+1; is the element-wise multiplication calculation; g′(z(l)) is the derivative of the activation function;

[0068] S513: Construct the error output threshold Δ as:

[0069]

[0070] Among them, Δ is the error output threshold, specifically the error ratio of the loss function difference between the current layer and the output layer to the output layer loss function value. When Δ ≥ 50%, the current loss function error does not meet the requirements, and gradient calculation is required to update the classifier model parameters.

[0071] Preferably, in step S5, the back propagation algorithm is used to update the weight matrix parameters and bias item parameters of the classifier model in step S2, specifically:

[0072] S521: Calculate the total loss function L total Gradient:

[0073]

[0074]

[0075] in, is the loss function L total The partial derivative of the weight matrix W(l) of the lth layer; W(l) is the weight matrix of the lth layer of the classifier model; a(l-1) is the activation value of the (l-1)th layer of the classifier model, which represents the output of the neurons in this layer; is the loss function L total The partial derivative of the bias term b(l) of the lth layer; b(l) is the bias of the lth layer of the classifier model;

[0076] S522: Update the classifier model parameters using the gradient descent optimization algorithm:

[0077]

[0078] Where W(l) is the weight matrix of the lth layer of the classifier model; b(l) is the bias of the lth layer of the classifier model; α is the set learning rate, which is used to control the step size of parameter update.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] (1) Based on the traditional event extraction method, the present invention proposes a corner-labeled joint event extraction method based on a double-loop BERT to extract key information from aviation accident report texts and solve the problems of cascade errors and redundant errors existing in step-by-step event extraction. The method is used to extract key information such as accident phenomena, accident causes, accident results, and main objects from aviation accident report texts.

[0081] (2) The present invention uses a double-loop BERT to identify and classify trigger words and locate arguments, combines context information and feature vectors to jointly calculate argument confidence, and adopts a corner marking method to jointly decode event information, thereby enhancing information relevance and improving the accuracy of text information extraction.

[0082] (3) The present invention improves the corner marking strategy in the traditional model. On the basis of being able to solve the problem of entity pair overlap, it solves the problem of implicit logical relationships in the same argument and the problem of entity pair overlap, thereby improving the processing capability of aviation accident information. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is a control block diagram of the aviation accident text data extraction method based on cyclic BERT and corner marking of the present invention;

[0084] Figure 2 This is a schematic diagram of the dual-loop BERT model architecture of the present invention;

[0085] Figure 3 Schematic diagram of the embedding layer representation of the dual-loop BERT model of the present invention;

[0086] Figure 4 This is an example diagram of the corner marking method of the present invention;

[0087] Figure 5 This is an example diagram of the optimized corner marking method of the present invention. DETAILED DESCRIPTION

[0088] The exemplary embodiments, features, and aspects of the present invention will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0089] The embodiment of the present invention provides a method for extracting aviation accident text data based on cyclic BERT and corner marking, such as Figure 1As shown, a double-loop BERT model is used to identify trigger words in aviation accident report text data; a classifier model is used to jointly calculate the argument vector of the aviation accident report text data; the total confidence of the text data argument vector is calculated based on the vector of the trigger word and the argument label of the text data; the event triples of the aviation accident report text data are decoded using a corner labeling matrix to obtain the total loss value of the text data; and the aviation accident report text data is extracted based on the total loss value. The method includes:

[0090] Step S1: Use a double-loop BERT model to identify trigger words in aviation accident report text data. Obtain the text data of the aviation accident report, embed the text data feature vector, encode the text data embedding vector into a label sequence, and embed the trigger word position in the text data. Specifically,

[0091] Step S11: Embed the text data of the aviation accident report with a feature vector; embed the text data through entity word embedding, segment embedding, and position embedding to form the embedding vector of the text data:

[0092] x i =e i +p i +s i ;

[0093] Among them, x i is the embedding vector of text data; e i is the entity word embedding vector of text data; p i is the segment embedding vector of text data; i Embedding vectors for the positions of text data.

[0094] Step S12: Encode the embedding vector of the text data in step S11 into a label sequence to implement training of entity words, trigger words, and contextual information in the text data of the aviation accident report:

[0095] h i =BERT(x i );

[0096] Among them, h i is the vector of the contextual information of the i-th trigger word; BERT is the multi-layer Transformer hidden function mapping.

[0097] Step S13: The trigger word position in the text data of the aviation accident report is embedded and marked; when BERT is used for processing for the first time, the position embedding mark in the text data is 0. After processing through multiple layers of hidden units, the trigger word and the trigger word position are identified; when BERT is used for processing for the second time, the trigger word position in the text data is embedded and marked as 1.

[0098] Step S14: The position embedded feature vector is subjected to secondary encoding processing by the BERT model to identify the argument vector and complete the text data trigger word training, specifically:

[0099]

[0100] in, is the vector of the trigger word in the text data; softmax is the function mapping relationship; W trigger The weight of the context information vector of the trigger word in the text data; h i is the vector of the context information of the i-th trigger word; b trigger is the vector deviation of the trigger word in the text data; i is the number of the aviation accident report text data.

[0101] like Figure 2 The figure shows a schematic diagram of the architecture of the corner-labeled joint event extraction model based on double-loop BERT of the present invention; the overall architecture of the model is explained. The model is divided into three modules: encoding layer, prediction layer and decoding layer. The encoding layer adopts BERT pre-training model encoding to convert the input sentence into a feature vector, and encodes the feature vector into a label sequence through multiple layers of hidden units. The prediction layer is responsible for the prediction of trigger words and arguments, and uses the BERT pre-training model to identify entities and trigger words. Through the double-loop BERT method, the trigger words are used to locate arguments and learn contextual information. Through the activation function in the classifier, the entity, trigger words and semantic information are combined to jointly calculate the confidence and mark the argument position with the highest confidence, and store it in the corner labeling matrix in the decoding layer. The decoding layer uses the corner labeling method to combine and score all possible argument pairs through the corner labeling matrix of the predefined relationship, and outputs the combination with the highest score as structured triple data.

[0102] like Figure 3 The figure shows a schematic representation of the embedding layer of the dual-loop BERT model of the present invention. It illustrates how sentences fed into the BERT model are encoded into feature vectors using methods such as word embedding, segment embedding, and position embedding. Word embedding assigns a feature vector to each subword and symbol in the sentence, adding a "[CLS]" symbol at the beginning of the input sentence for classification and a "[SEP]" symbol at the end of each segment to segment the sentence. Segment embedding classifies and labels subwords within the same segment, using the same symbol. Position embedding marks the position of each subword and symbol, with the initial position embedding value being 0.

[0103] Step S2: Use the classifier model to jointly calculate the argument vector of the text data of the aviation accident report.

[0104] Step S21: Use the double-loop BERT to jointly represent the feature vectors, and calculate the text data argument vector through the classifier model:

[0105] J i ={B1(w1+e1),…,B i (w i +e i ),…B m (w m +e m )};

[0106] Among them, J i is the i-th joint feature vector of the text data argument vector; B i is the i-th embedding vector of the context information of the text data; w i is the i-th word vector of the text data; m is the total number of joint feature vectors of the text data argument vectors.

[0107] Step S22: Determine the probability of the text data argument vector through the classifier model:

[0108]

[0109] in, Combined argument head word probability for text data; The probability of the tail word of the joint argument for text data; is the weight of the head word of the text data argument; is the weight of the tail word of the text data argument; J s is the joint feature vector of the head word of the text data argument; J e is the joint feature vector of the tail words of the text data argument; B is the embedding vector of the context information of the text data; w is the word vector of the text data; e is the embedding vector of the entity word of the text data.

[0110] Step S23: Combine the text data with the argument head word probability and the probability of the tail word of the joint argument Combining them, the predicted text data meta-label is:

[0111]

[0112] in, is the i-th argument vector of the predicted text data; is the probability of the head word of the i-th argument; is the probability of the tail word of the i-th argument.

[0113] Step S3: Calculate the total confidence CN of the text data argument vector based on the text data trigger word vector in step S1 and the text data argument label in step S2:

[0114]

[0115] Among them, CN is the total confidence of the text data argument vector; Π is the product operation; Confidence is the confidence calculation function mapping.

[0116] Step S4: Use the corner marker matrix to decode the aviation accident event triples of the aviation accident report text data to obtain the total loss value of the text data.

[0117] Step S41: The argument vector with the highest total confidence CN of the text data argument vector in step S3 is stored in the corner marker matrix R for decoding feature marking, specifically including: the first decoding feature marker HB-TB is decoded from the word corresponding to the starting position of the subject word to the word corresponding to the starting position of the object word; the second decoding feature marker HB-TE is decoded from the word corresponding to the starting position of the subject word to the word corresponding to the ending position of the object word; the third decoding feature marker HE-TE is decoded from the word corresponding to the ending position of the subject word to the word corresponding to the ending position of the object word; the fourth decoding feature marker Minus is decoded in sequence according to the sub-word position order; the fifth decoding feature marker Null is a word that does not need to be decoded and output.

[0118] The corner label matrix assignment function is used to calculate the aviation accident event triple eigenvalues ​​of the text data:

[0119]

[0120] in, is the eigenvalue of the aviation accident event triple of the text data; R is the corner label matrix of the aviation accident event triple; is the vector of the subject word in the aviation accident event triple; O is the vector of the object word in the aviation accident event triple; is the cyclic calculation of subject words and object words; φ is the ReLU activation function mapping; W is the weight parameter for classifier model training; is the concatenation operation of the subject word vector and the object word vector; b is the bias parameter for classifier model training.

[0121] like Figure 4The following is an example diagram of the corner marking method of the present invention; taking "foreign object causes blade damage" as an example, in the matrix where the predefined relationship is "causes", for the "HB-TB" marking position, the word "foreign object" corresponding to the row is the start of the subject word, and the word "blade" corresponding to the column is the start of the object word. By traversing row by row, the triple (foreign object, causes, blade) is parsed; for the "HB-TE" marking position, the word "foreign object" corresponding to the row is the start of the subject word, and the word "damage" corresponding to the column is the end of the object word. By traversing row by row, the triple (foreign object, causes, blade damage) is parsed; for the "HE-TE" marking position, the word "object" corresponding to the row is the end of the subject word, and the word "damage" corresponding to the column is the end of the object word. By traversing row by row, the triple (foreign object, causes, blade damage) is parsed.

[0122] If there is an implicit logical relationship in the same argument, for example, in the event of "blade fatigue damage", there is a logical relationship of (blade damage, cause, blade fatigue). Then when decoding the subject "blade damage", the word "fatigue" needs to be skipped. Therefore, based on the original four marking type tags ("HB-TB", "HB-TE", "HE-TE" and "-"), the "Null" marking is added, indicating that the word at this position is not recognized and decoding continues by skipping this word. This marking usually appears in the middle part, as Figure 3 shown. The "Null" marking is different from the "-" marking. The former is defined as a key marking during decoding. If this marking is detected, the information at this position will be skipped and decoding will continue backward, while the latter is not recognized during decoding.

[0123] such as Figure 5The following is an example diagram of the optimized corner marking method of the present invention, which illustrates the optimized corner marking method example adopted when processing sentences with implicit logical relationships in the same argument. Taking the event of "the blade is damaged due to fatigue" as an example, in this event, there is a logical relationship of (blade damage, cause, blade fatigue). When decoding the main body word "blade damage", several words such as "due to", "fatigue", "produce", and "generate" need to be skipped. When decoding the object word "blade fatigue", the word "due to" needs to be skipped. Therefore, during the line-by-line traversal, for the "Null" marker after the identified "HB-TB" marker, skip the word "due to" corresponding to this marker column; for the "Null" marker after the identified "HB-TE" marker, skip the words "due to", "fatigue", "produce", and "generate" corresponding to each marker row. Based on this, in the matrix with the predefined relationship of "cause", for the "HB-TB" marker position, the word "blade" in the row is the start of the main body word, and the word "blade" in the column is the start of the object word. Through line-by-line traversal, a triple (blade, cause, blade) is parsed; for the "HB-TE" marker position, the word "more" in the row is the start of the main body word, and the word "damage" in the column is the end of the object word. Through line-by-line traversal, a triple (more, cause, blade damage) is parsed; for the "HE-TE" marker position, the word "foreign object" in the row is the end of the main body word, and the word "damage" in the column is the end of the object word. Through line-by-line traversal, a triple (foreign object, cause, blade damage) is parsed.

[0124] Step S42: Calculate the trigger word loss value L through the cross-entropy loss function trigger It is:

[0125]

[0126] Among them, L trigger is the predicted loss value of the trigger word; N is the number of trigger word samples; C is the number of trigger word categories; is the true label of the trigger word; is the predicted probability of the trigger word; c is the trigger word category number.

[0127] The equations for measuring the argument of the aviation accident event, the predicted loss value of the argument start position, and the predicted loss value of the argument end position using the argument prediction loss model are:

[0128]

[0129] Among them, is the predicted loss value of the argument start position; is the predicted loss value of the argument end position; is the true label of the argument start position; is the predicted probability of the argument start position; is the actual label of the end position of the argument; Predict the probability of where the argument ends.

[0130] Combining the predicted loss value of the starting position and the predicted loss value of the ending position of each argument, the total predicted loss value of the aviation accident event argument is:

[0131]

[0132] Among them, A is the set of all arguments; L arg Predict the loss value for the argument.

[0133] The corner marker matrix prediction loss is used to measure the difference between the corner marker matrix extracted by the double-loop BERT model and the actual result. The corner marker prediction loss value L rel The specific formula is:

[0134]

[0135] Among them, L rel is the corner mark prediction loss value; K is the number of relationship categories; is the true label of the k-th category relationship between the i-th entity and the j-th entity; is the predicted probability of the corner label matrix; k is the relationship category number.

[0136] Predict the loss value L based on the trigger word trigger , argument prediction loss value L arg And the corner mark prediction loss value L rel The weighted summation is used to obtain the total loss value L of the text data. total for:

[0137] L total =αL trigger +βL arg +γL rel ;

[0138] Among them, L total is the total loss value of text data; L trigger is the trigger word prediction loss value; L arg Predict the loss value for the argument; L rel is the corner tag prediction loss value; α is the trigger word prediction parameter; β is the argument prediction parameter; γ is the corner tag prediction parameter.

[0139] S5: For the aviation accident event triplet feature value of the text data in step S41, set the error output threshold; calculate the classifier model output layer error term as:

[0140]

[0141] Among them, δ(L total ) is the error term of the output layer of the classifier model; is the total loss function L total The weighted input z(L total )’s partial derivatives; is the predicted probability of the classifier model; y is the true label of the classifier model.

[0142] The error term of the hidden layer of the classifier model is calculated as:

[0143]

[0144] Wherein, δ(l) is the error term of the classifier model at layer l; W(l+1) is the weight of the classifier model at layer l+1; δ(l+1) is the error value of the classifier model at layer l+1; is the element-wise multiplication calculation; g′(z(l)) is the derivative of the activation function.

[0145] The construction error output threshold is:

[0146]

[0147] Among them, Δ is the error ratio between the loss function difference of the current layer and the output layer and the loss function value of the output layer. When Δ≥50%, the current loss function error does not meet the requirements, and gradient calculation is required to update the classifier model parameters.

[0148] When the total loss value L of the text data output in step S42 is total When the error is greater than or equal to the error output threshold, the back propagation algorithm is used to update the weight matrix parameters and bias parameters of the classifier model in step S2, and the process returns to step S2 to continue. The total loss function L is calculated. total Gradient:

[0149]

[0150] in, is the loss function L total The partial derivative of the weight matrix W(l) of the lth layer; W(l) is the weight matrix of the lth layer of the classifier model; a(l-1) is the activation value of the (l-1)th layer of the classifier model, which represents the output of the neurons in this layer; is the loss function L total The partial derivative of the bias term b(l) of the lth layer; b(l) is the bias of the lth layer of the classifier model.

[0151] The classifier model parameters are updated using the gradient descent optimization algorithm as follows:

[0152]

[0153] Where W(l) is the weight matrix of the lth layer of the classifier model; b(l) is the bias of the lth layer of the classifier model; α is the set learning rate, which is used to control the step size of parameter update.

[0154] When the total loss value L of the text data output in step S42 is total When the error is lower than the output threshold, the aviation accident event triplet in step S41 is output as the extracted aviation accident report text data.

[0155] The aviation accident report text data obtained by the embodiment of the present invention is:

[0156] "text":"A fatigue crack in the No. 1 main landing gear axle was not detected. The fatigue crack was caused by a corroded hole in the brake system connection flange.","entities":["No. 1 main landing gear","axle","fatigue crack","brake system","flange","corroded hole"], "triggers":["undetected","connection","corrosion","caused"], "arguments":["A fatigue crack in the No. 1 main landing gear axle was not detected.","The fatigue crack was caused by a corroded hole in the brake system connection flange"]. "triples"[["axle","exists","fatigue crack"],["flange","exists","corroded hole"],["fatigue crack","located","axle"],["corroded hole","located","flange"],["axle","located","No.1 main landing gear"],["hole","occurs","corrosion"],["axle","occurs","fatigue crack not found"],["fatigue crack","cause of occurrence","a corroded hole in the brake system connection flange"].

[0157] The embodiments of the present invention have the following beneficial effects: In order to extract key accident information from aviation accident report texts and solve the problems of cascading errors and redundant errors existing in step-by-step fault event extraction, the embodiments of the present invention propose a corner-labeled joint event extraction method based on a double-loop BERT, which is used to extract key information such as accident phenomena, accident causes, accident results, and main objects from aviation accident report texts. The double-loop BERT model is used to identify and classify trigger words and locate arguments, and the argument confidence is jointly calculated by combining contextual information and feature vectors. The corner-labeled method is used to jointly decode event information, enhance information relevance, and thereby improve the accuracy of text information extraction. On the basis of solving the problem of entity pair overlap, it can solve the problem of implicit logical relationships in the same argument and the problem of entity pair overlap, thereby improving the processing capability of aviation accident information. The analysis of actual cases can prove that the present invention can successfully obtain accident information.

[0158] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for extracting aviation accident text data based on cyclic BERT and corner tagging, characterized by , which includes: S1: Using a dual-loop BERT model to identify trigger words in aviation accident report text data; Obtain text data from aviation accident reports, embed feature vectors of the text data, encode the embedded vectors of the text data into label sequences, and embed and annotate the trigger word positions of the text data. Perform secondary encoding processing on the feature vectors of the position embedding using the BERT model, identify the argument vectors, and complete the text data trigger word training. Specifically, in, is the vector of the trigger word in the text data; softmax is the function mapping relationship; W trigger The weight of the context information vector of the trigger word in the text data; h i is the vector of the context information of the i-th trigger word; b trigger is the vector deviation of the trigger word in the text data; i is the number of the aviation accident report text data; S2: Use the classifier model to jointly calculate the argument vector of the text data of aviation accident reports; Use double-loop BERT to jointly represent the feature vector, calculate the probability of the text data argument vector through the classifier model, and combine the text data with the argument head word probability and the probability of the tail word of the joint argument Combining them, the predicted text data meta-label is: in, is the i-th argument vector of the predicted text data; is the probability of the head word of the i-th argument; is the probability of the tail word of the i-th argument; S3: Calculate the total confidence CN of the text data argument vector based on the vector of the text data trigger word in step S1 and the text data argument label in step S2: Among them, CN is the total confidence of the text data argument vector; Π is the product operation; Confidence is the confidence calculation function mapping; S4: Decoding the aviation accident event triples of the aviation accident report text data using the corner labeling matrix to obtain the total loss value of the text data; S41: The argument vector with the highest total confidence CN of the text data argument vector in step S3 is stored in the corner labeling matrix R for decoding feature labeling; the aviation accident event triplet feature value of the text data is calculated using the corner labeling matrix assignment function as follows: in, is the eigenvalue of the aviation accident event triple of the text data; R is the corner label matrix of the aviation accident event triple; is the vector of the subject word in the aviation accident event triple; O is the vector of the object word in the aviation accident event triple; is the cyclic calculation of subject words and object words; φ is the ReLU activation function mapping; W is the weight parameter for classifier model training; is the concatenation operation of the subject word vector and the object word vector; b is the bias parameter for classifier model training; S42: Predict the loss value L based on the trigger word trigger , argument prediction loss value L arg And the corner mark prediction loss value L rel The weighted summation is used to obtain the total loss value L of the text data. total for: L total =αL trigger +βL arg +γL rel Among them, L total is the total loss value of text data; L trigger is the trigger word prediction loss value; L arg Predict the loss value for the argument; L rel is the corner mark prediction loss value; α is the trigger word prediction parameter; β is the argument prediction parameter; γ is the corner mark prediction parameter; S5: For the aviation accident event triplet feature value of the text data in step S41, set the error output threshold; when the total loss value L of the text data output in step S42 is total When the error is greater than or equal to the error output threshold, the weight matrix parameters and bias item parameters of the classifier model in step S2 are updated using the back propagation algorithm, and the process returns to step S2 to continue. When the total loss value L of the text data output in step S42 is less than or equal to the error output threshold, the process returns to step S2 to continue the process. total When the error is lower than the error output threshold, the aviation accident event triplet in step S41 is output as the extracted aviation accident report text data.

2. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: In step S1, text data feature vectors are embedded, the embedded vectors of the text data are encoded into a label sequence, and the trigger word positions of the text data are embedded and labeled, specifically: S11: Embed the text data of the aviation accident report with a feature vector; embed the text data through entity word embedding, segment embedding, and position embedding to form the embedding vector of the text data: x i =e i +p i +s i Among them, x i is the embedding vector of text data; e i is the entity word embedding vector of text data; p i is the segment embedding vector of text data; i Embed vectors for the positions of text data; S12: Encode the embedding vector of the text data in step S11 into a label sequence to implement training of entity words, trigger words and context information in the text data of aviation accident reports: h i =BERT(x i ) Among them, h i is the vector of the contextual information of the i-th trigger word; BERT is the multi-layer Transformer hidden function mapping; S13: The trigger word position embedding annotation of the text data of the aviation accident report; when BERT is used for processing for the first time, the position embedding mark in the text data is 0. After processing through multiple layers of hidden units, the trigger word and the trigger word position are identified; when BERT is used for processing for the second time, the trigger word position embedding mark in the text data is 1.

3. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: In step S2, the double-loop BERT is used to jointly represent the feature vectors, and the probability of the text data argument vector is calculated through the classifier model, specifically: S21: Use the double-loop BERT model to jointly represent the feature vectors, and obtain the joint feature vector of the text data argument vector as follows: J i ={B1(w1+e1),…,B i (w i +e i ),...B m (w m +e m )} Among them, J i is the i-th joint feature vector of the text data argument vector; B i is the i-th embedding vector of the context information of the text data; w i is the i-th word vector of the text data; m is the total number of joint eigenvectors of the text data argument vectors; S22: The probability of determining the text data argument vector through the classifier model is: in, Combined argument head word probability for text data; The probability of the tail word of the joint argument for text data; is the weight of the head word of the text data argument; is the weight of the tail word of the text data argument; J s is the joint feature vector of the head word of the text data argument; J e is the joint feature vector of the tail words of the text data argument; B is the embedding vector of the context information of the text data; w is the word vector of the text data; e is the embedding vector of the entity word of the text data.

4. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: The decoding signatures in step S41 include: a first decoding signature HB-TB, a second decoding signature HB-TE, a third decoding signature HE-TE, a fourth decoding signature Minus, and a fifth decoding signature Null; The first decoding feature mark HB-TB is decoded from the character corresponding to the start position of the subject word to the character corresponding to the start position of the object word; The second decoding feature mark HB-TE is decoded from the character corresponding to the start position of the subject word to the character corresponding to the end position of the object word; The third decoding feature mark HE-TE is decoded from the character corresponding to the end position of the subject word to the character corresponding to the end position of the object word; The fourth decoding feature mark Minus is to decode in sequence according to the subword position order; The fifth decoding feature flag Null is a word that does not need to be decoded and output.

5. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: The trigger word prediction loss value L in step S42 trigger Specifically, the trigger word loss value is calculated by the cross entropy loss function: Among them, L trigger is the prediction loss value of the trigger word; N is the number of trigger word samples; C is the number of trigger word categories; is the real label of the trigger word; is the predicted probability of the trigger word; c is the category number of the trigger word.

6. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: The argument prediction loss value L in step S42 arg , specifically: The argument prediction loss model is used to measure the aviation accident event argument. The equations for the predicted loss value at the beginning of the argument and the predicted loss value at the end of the argument are as follows: in, Predict the loss value for the starting position of the argument; Predict the loss value for the end position of the argument; is the real label of the starting position of the argument; Predict the probability of the starting position of the argument; is the actual label of the end position of the argument; Predict the probability of the end position of the argument; Combining the predicted loss value of the starting position and the predicted loss value of the ending position of each argument, the total predicted loss value of the aviation accident event argument is: Among them, A is the set of all arguments; L arg Predict the loss value for the argument.

7. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: The corner mark prediction loss value L in step S42 rel Specifically, the corner label matrix prediction loss is used to measure the difference between the corner label matrix extracted by the double-loop BERT model and the actual result. The specific formula is: Among them, L rel is the corner mark prediction loss value; K is the number of relationship categories; is the true label of the k-th category relationship between the i-th entity and the j-th entity; is the predicted probability of the corner label matrix; k is the relationship category number.

8. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: In step S5, the error output threshold is set, specifically: S511: Calculate the error term of the output layer of the classifier model: Among them, δ(L total ) is the error term of the output layer of the classifier model; is the total loss function L total The weighted input z(L total )’s partial derivatives; is the predicted probability of the classifier model; y is the true label of the classifier model; S512: Calculate the error term of the hidden layer of the classifier model: Wherein, δ(l) is the error term of the classifier model at layer l; W(l+1) is the weight of the classifier model at layer l+1; δ(l+1) is the error value of the classifier model at layer l+1; is the element-wise multiplication calculation; g′(z(l)) is the derivative of the activation function; S513: Construct the error output threshold Δ as: Among them, Δ is the error output threshold, specifically the error ratio of the loss function difference between the current layer and the output layer to the output layer loss function value. When Δ ≥ 50%, the current loss function error does not meet the requirements, and gradient calculation is required to update the classifier model parameters.

9. The aviation accident text data extraction method based on cyclic BERT and corner marking according to claim 1 is characterized by: In step S5, the back propagation algorithm is used to update the weight matrix parameters and bias parameters of the classifier model in step S2, specifically: S521: Calculate the total loss function L total Gradient: in, is the loss function L total The partial derivative of the weight matrix W(l) of the lth layer; W(l) is the weight matrix of the lth layer of the classifier model; a(l-1) is the activation value of the (l-1)th layer of the classifier model, which represents the output of the neurons in this layer; is the loss function L total The partial derivative of the bias term b(l) of the lth layer; b(l) is the bias of the lth layer of the classifier model; S522: Update the classifier model parameters using the gradient descent optimization algorithm: Where W(l) is the weight matrix of the lth layer of the classifier model; b(l) is the bias of the lth layer of the classifier model; α is the set learning rate, which is used to control the step size of parameter update.

Images