Semi-supervised relation extraction method for aircraft fault data
By introducing a semi-supervised relation extraction method into the field of aircraft fault diagnosis, and utilizing pseudo-label generation and adaptive threshold selection, the problems of inconsistent data labeling quality and imbalanced distribution are solved, the performance of the relation extraction model is improved, and the engineering application of knowledge graphs is promoted.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-23
- Publication Date
- 2026-04-14
AI Technical Summary
In the field of aircraft fault diagnosis, existing technologies lack publicly available datasets, and the distribution of unstructured maintenance data is scattered and heterogeneous from multiple sources, resulting in poor performance of relation extraction models and poor consistency in data annotation quality, which affects the quality of knowledge graph construction.
A semi-supervised relation extraction method is adopted. By introducing a pseudo-label generation module into the labeled data, the pseudo-label generation is optimized using meta-learning theory. Combined with an adaptive threshold selection method, the quality and selection accuracy of pseudo-labels are improved, and a relation extraction model for aircraft unstructured fault data is constructed.
It improves the performance of relation extraction models in the field of aircraft fault diagnosis, enhances the consistency and distribution balance of data annotation quality, and strengthens the engineering application capabilities of knowledge graphs.
Smart Images
Figure CN121858744A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of knowledge management and fault diagnosis technology, specifically relating to a semi-supervised relation extraction method for aircraft fault data. Background Technology
[0002] With the development of artificial intelligence, knowledge graph technology has been widely applied in fault diagnosis, bringing new opportunities and challenges to the fault diagnosis of high-end equipment with high reliability and complex system structures, such as aircraft and missiles. Relation extraction, as a crucial step in fault knowledge graph modeling, aims to extract relationships between entities from unstructured text data, directly impacting the quality of knowledge graph construction. Currently, most relation extraction tasks require high-quality labeled data under supervised conditions, but this method demands a high quantity and quality of data. For relation extraction in aircraft fault diagnosis, there is a lack of publicly available datasets; furthermore, unstructured maintenance data is scattered, heterogeneous from multiple sources, and lacks standardization. Moreover, relation labeling during data preprocessing is labor-intensive and easily introduces subjective human judgment, leading to inconsistent labeling quality and reducing the effectiveness of the relation extraction model, severely impacting the engineering application of fault knowledge graphs in aircraft fault diagnosis. Summary of the Invention
[0003] To address the aforementioned problems and the issues of poor data labeling consistency and imbalanced distribution in existing aircraft unstructured fault data relation extraction methods, this invention proposes a semi-supervised relation extraction method for aircraft fault data, specifically including the following steps:
[0004] Step 1: Relationship Classification;
[0005] Specifically as follows:
[0006] Step 1.1: For the input text H = [h1, h2, ..., h...] L ], h i This represents the i-th character in the text, where i ∈ [1, L], and L represents the total number of characters in the text; a leading and trailing symbol is inserted only before and after each identified entity position [E]. i start ],[E i end ], to mark the start and end positions of entities in the text, and no symbols are inserted before or after the position of unrecognized entities, as shown in formula (1);
[0007] H = [h1,...,[E] i start ],h i ,[E i end],...,[E j start ],h j ,[E j end ],...h L (1)
[0008] The above formula means that only the i-th character and the i-th character are recognized entities, and a leading symbol and a trailing symbol are inserted before and after them respectively;
[0009] After updating the word vectors of the text, all leading symbols [E] are processed. i start ], [E j start ] Perform vector concatenation to obtain vector h as the vector representation of entity pairs in the text, which is used to characterize the relationship information between entities in the text, as shown in formula (2);
[0010]
[0011] in, d represents the dimension of the word vector; The leading symbol [E] i start ], [E j start The entity vector information; during training, the suffix is used to tell the module the position of the last character of the entity;
[0012] Step 1.2: Use a two-layer fully connected layer as a relation classifier, as shown in formula (3);
[0013]
[0014] Among them, I n For the predicted relation type, C ζ (·) represents a fully connected neural network, which processes text information... Convert to It consists of a text encoder and a relation classifier. The result of Formula 1 is given by n, which represents the number of pseudo-labels generated in the labeled data. E1 and E2 represent the manually labeled entity information, respectively. Let N real labels and M pseudo-labels be represented by the label set S. all ={g1,g2,...,g N p1, p2, ..., p M}, where g1, g2, ..., g N p1, p2, ..., p M Let N represent the real labels and M pseudo labels, respectively. To improve the relation extraction capability of the relation classification module parameter ζ, the following loss function is adopted. As shown in formula (4);
[0015]
[0016] Among them, I n These are the labels predicted from the labeled data, where n is the number of pseudo-labels generated in the labeled data, and I... m Here, represents the predicted labels from unlabeled data, m is the number of pseudo-labels generated in the unlabeled data, loss(·,·) is the cross-entropy loss function, and one_hot(·) indicates that the relation labels are represented using one-hot encoding. To facilitate adjusting the model's dependence on pseudo-labels, the optimization of classifier C... ζ When setting parameters, set parameter 0 ≤ q m ≤1 is used as the confidence coefficient of the pseudo-label loss term, as shown in formula (5). The confidence coefficient is the maximum output probability of the pseudo-label generation network in recognizing pseudo-labels.
[0017]
[0018] in, For unlabeled text information, This is a fully connected neural network, represented as a pseudo-label generation network;
[0019] Step 2: Generating pseudo-tags;
[0020] Specifically as follows:
[0021] Step 2.1: To reduce the impact of network structure differences on the collaborative training process, the pseudo-label generation module and the relation classification module adopt the same network structure; the optimization function of the pseudo-label generation module is shown in formula (6);
[0022]
[0023] Wherein, the pseudo-label generation module parameter ζ′ represents the validation phase parameter of the relation classification module after one gradient update using the loss in formula (4), C ζ′ This indicates the network parameters generated by the pseudo-labels;
[0024] Step 2.2: Optimization of parameter ζ′ of the pseudo-label generation module, as shown in formula (7);
[0025]
[0026] Where α is the meta-learning weight, and the validation loss of the relation classification module parameter ζ′ is used to update the pseudo-label generation module parameter η. The meta-learning parameters of the relation classification network are represented; the optimization function of the pseudo-label generation module is shown in Equation (8);
[0027]
[0028] Among them, g m For unlabeled data, H(·,·) represents the cross-entropy loss function. This represents the label predicted by the relation classification model. The pseudo-labels generated by the pseudo-label generation network are represented by one_hot(·), where one_hot(·) indicates relation label encoding.
[0029] Step 3: Adaptive pseudo-label selection;
[0030] Specifically as follows:
[0031] Step 3.1: Global threshold calculation;
[0032] The global threshold τ within each training time step of the model is estimated using the exponential moving average (EMA). t And initialize it to 1 / C, where C represents the number of relation types, as shown in formula (9);
[0033]
[0034] Where λ∈(0,1) is the momentum decay of the EMA, U represents the total number of data points, and μ represents the proportion of unlabeled data points to the total number of data points. Let τ represent the value of τ at the previous moment, and t represent the number of iterations;
[0035] Step 3.2: Local threshold calculation;
[0036] To determine the threshold corresponding to each relation type, the expected value of the predicted value of the pseudo-label generation module for relation type c is used as the specific threshold for that relation, as shown in formula (10).
[0037]
[0038] Where, p t (c)=[p t (1),p t (2),...,p t [C] is a local threshold encompassing all relation types, p t (1),p t (2),...,p t (C) represents the local thresholds for C different relation types. q represents the average threshold of the relationship type at time t-1. m (c) represents the threshold for the c-th relation type, q m (c) represents the local threshold at time t;
[0039] Step 3.3: Combine the global threshold and the local threshold to obtain the adaptive global threshold. As shown in formula (11);
[0040]
[0041] Where, τ t This represents the global threshold for the t-th iteration, and MaxNorm(·) is the maximum normalization function. It is a function for finding the maximum value; the semi-supervised training objective for the t-th iteration. As shown in formula (12);
[0042]
[0043] in, Represented as a relational classification network The output, Q m The representation is the relation label encoding one_hot(g) m ).
[0044] In one embodiment of the present invention, the method further includes step 4: constructing an aircraft unstructured relation dataset for a semi-supervised relation extraction model for aircraft fault data;
[0045] Specifically as follows:
[0046] Step 4.1: Divide the sample set into a training set and a test set. The training set is used to train the model's parameters, and the test set is used to evaluate the model's performance on unknown data.
[0047] Step 4.2: Train the semi-supervised relation extraction model based on adaptive meta-learning using the training set. Set a maximum number of training iterations during the training process. After each training iteration, the training set and test set data are redistributed according to the original ratio. Stop training when the maximum number of training iterations is reached.
[0048] Step 4.3: Test the trained relation extraction model using the test set to obtain a relation extraction model for aircraft unstructured fault data.
[0049] This invention first establishes two identical networks to classify relationships in labeled data and generate pseudo-labels for unlabeled data, respectively. Next, it introduces meta-learning theory, using the validation loss of the relationship classification model to meta-update the training parameters of the pseudo-label generation model. Then, in the pseudo-label selection process, an adaptive thresholding method is proposed to improve the sensitivity of the pseudo-label selection process to few sample relationship types. This method enables semi-supervised relationship extraction for unstructured aircraft fault data. Attached Figure Description
[0050] Figure 1 This is a diagram of a semi-supervised relation extraction model for aircraft fault data in this invention. Detailed Implementation
[0051] The invention will be further explained below with reference to the accompanying drawings and specific implementation examples.
[0052] This invention provides a semi-supervised relation extraction method for aircraft fault data, mainly including relation classification, pseudo-label generation, and adaptive threshold selection, such as... Figure 1 As shown in the diagram. The relationship classification module is connected to the pseudo-label generation module, the pseudo-label generation module is connected to the adaptive pseudo-label selection module, and the adaptive pseudo-label selection module is connected to the relationship classification module; specifically as follows:
[0053] Step 1: Relationship Classification;
[0054] This method extracts semantic information representing relationships between entities from text, thereby achieving entity relationship extraction. First, a text encoder is used to mine the contextual semantic information of the fault text; then, a relationship classifier is used to extract relationships from unstructured fault data; the details are as follows:
[0055] Step 1.1: For the input text H = [h1, h2, ..., h...] L ], h i This represents the i-th character in the text, where i ∈ [1, L], and L represents the total number of characters in the text. This invention inserts a leading and trailing symbol [E] before and after each identified entity position, respectively. i start ],[E i end ], to mark the start and end positions of entities in the text (entity recognition is not the subject of this invention, so manual recognition is adopted, and the recognition method is well known to those skilled in the art), no symbols are inserted before or after the position of unrecognized entities, as shown in formula (1).
[0056] H = [h1,...,[E] i start ],h i ,[E i end ],...,[E j start ],h j ,[E j end ],...h L (1)
[0057] The above formula means that only the i-th character and the i-th character are recognized entities, and a leading symbol and a trailing symbol are inserted before and after them, respectively.
[0058] After updating the word vectors of the text, all leading symbols [E] are processed. i start ], [E j start ] Perform vector concatenation to obtain vector h as the vector representation of entity pairs in the text (entity pair information is manually labeled, and is called entity pair information because it is obtained by concatenating two entity information), which is used to represent the relationship information between entities in the text, as shown in formula (2).
[0059]
[0060] in, d represents the dimension of the word vector. Wherein, The leading symbol [E] i start ], [E j start The entity vector information. During training, the suffix is used to tell the module the position of the last character of the entity, a function well known to those skilled in the art.
[0061] Step 1.2: Use a two-layer fully connected layer as a relation classifier. Its core principle is shown in formula (3).
[0062]
[0063] Among them, I n For the predicted relation type, C ζ (·) represents a fully connected neural network, which processes text information... Convert to It consists of a text encoder and a relation classifier (the composition method is well known to those skilled in the art), wherein, The symbol represents E1 (i.e., this symbol represents E1), where n is the number of pseudo-labels generated in the labeled data, and E1 and E2 represent the manually labeled entity information, respectively. Let N real labels and M pseudo-labels be represented by the label set S. all ={g1,g2,...,g N p1, p2, ..., p M}, where g1, g2, ..., g N p1, p2, ..., p M Let N represent the real labels and M the pseudo labels, respectively. To improve the relation extraction capability of the relation classification module parameter ζ, this invention adopts the following loss function. As shown in formula (4).
[0064]
[0065] Among them, I n These are the labels predicted from the labeled data, where n is the number of pseudo-labels generated in the labeled data, and I... m is the predicted label from unlabeled data, m is the number of pseudo-labels generated in the unlabeled data, loss(·,·) is the cross-entropy loss function (this function is well known to those skilled in the art), and one_hot(·) indicates that the relation label is represented using one-hot encoding. To facilitate adjusting the model's dependence on pseudo-labels, this invention optimizes the classifier C... ζ When setting parameters, set parameter 0 ≤ q m ≤1 is used as the confidence coefficient of the pseudo-label loss term, as shown in formula (5). The confidence coefficient is the maximum output probability of the pseudo-label generation network in recognizing pseudo-labels.
[0066]
[0067] in, For unlabeled text information, This is a fully connected neural network, represented as a pseudo-label generation network.
[0068] Step 2: Generating pseudo-tags;
[0069] A pseudo-label generation module is employed to generate pseudo-labels for unlabeled relation data, thereby effectively expanding the training samples of relation data and improving the classification performance of the relation classification module. Simultaneously, to improve the quality of the pseudo-labels generated by the pseudo-label generation module, this invention introduces a meta-learning mechanism, using the validation loss of the relation classification module parameters as the meta-learning objective to update the parameters of the pseudo-label generation module; specifically as follows:
[0070] Step 2.1: To reduce the impact of network structure differences on the collaborative training process, the pseudo-label generation module and the relation classification module adopt the same network structure. The optimization function of the pseudo-label generation module is shown in Equation (6).
[0071]
[0072] Wherein, the pseudo-label generation module parameter ζ′ represents the validation phase parameter of the relation classification module after one gradient update using the loss in formula (4), C ζ′ This indicates the network parameters generated by the pseudo-labels.
[0073] Step 2.2: Optimize the parameter ζ′ of the pseudo-label generation module, as shown in formula (7).
[0074]
[0075] Where α is the meta-learning weight, and the validation loss of the relation classification module parameter ζ′ is used to update the pseudo-label generation module parameter η. The meta-learning parameters of the relation classification network are represented. Finally, the optimization function of the pseudo-label generation module is shown in Equation (8).
[0076]
[0077] Among them, g m For unlabeled data, H(·,·) represents the cross-entropy loss function. This represents the label predicted by the relation classification model. The pseudo-labels generated by the pseudo-label generation network are represented by one_hot(·), which indicates the relation label encoding.
[0078] Step 3: Adaptive pseudo-label selection;
[0079] This step aims to select higher-quality pseudo-labels from the pseudo-label data generated by the pseudo-label generation module, thereby improving the performance of relation extraction; specifically as follows:
[0080] Step 3.1: Global threshold calculation.
[0081] To improve the computational efficiency of the global threshold, the exponential moving average (EMA) method is used to estimate the global threshold τ of the model at each training time step (Wen Wen, Hu Zhixin, Hao Zhifeng. A deep exponential moving average learning method for sequence recommendation [J]. Computer Science and Exploration, 2025, 19(03):774-786.). t It is initialized to 1 / C, where C represents the number of relation types, as shown in formula (9).
[0082]
[0083] Where λ∈(0,1) is the momentum decay of the EMA, U represents the total number of data points, and μ represents the proportion of unlabeled data points to the total number of data points. Let τ represent the value of τ at the previous moment, and t represent the number of iterations.
[0084] Step 3.2: Local threshold calculation;
[0085] The local thresholding method aims to adapt to the imbalance of relation types in semi-supervised relation extraction. By estimating the threshold for each relation type separately, the global threshold can be adjusted accordingly. To determine the threshold for each relation type, this invention uses the expected value of the predicted value of the pseudo-label generation module for relation type c as the specific threshold for that relation, as shown in formula (10).
[0086]
[0087] Where, p t (c)=[p t(1),p t (2),...,p t [C] is a local threshold encompassing all relation types, p t (1),p t (2),...,p t (C) represents the local thresholds for C different relation types. q represents the average threshold of the relationship type at time t-1. m (c) represents the threshold for the c-th relation type, q m (c) represents the local threshold at time t.
[0088] Step 3.3: Combine the global threshold and the local threshold to obtain the final adaptive global threshold. As shown in formula (11).
[0089]
[0090] Where, τ t Let represent the global threshold for the t-th iteration, and MaxNorm(·) be the maximum normalization function (which is well known to those skilled in the art). It is a function for finding the maximum value. Finally, the semi-supervised training objective for the t-th iteration. As shown in formula (12).
[0091]
[0092] in, Represented as a relational classification network The output, Q m The representation is the relation label encoding one_hot(g) m ).
[0093] Step 4: Construct an unstructured relation dataset for aircraft and use it to evaluate a semi-supervised relation extraction model for aircraft fault data; details are as follows:
[0094] Step 4.1: Divide the sample set into a training set and a test set, where the training set is used to train the model's parameters and the test set is used to evaluate the model's performance on unknown data;
[0095] Step 4.2: Train the semi-supervised relation extraction model for aircraft fault data using the training set. Set a maximum number of training iterations during the training process, and redistribute the training and test set data according to the original ratio after each training iteration. Stop training when the maximum number of training iterations is reached.
[0096] Step 4.3: Test the trained semi-supervised relation extraction model for aircraft fault data using the test set to obtain the relation extraction model for unstructured aircraft fault data.
Claims
1. A semi-supervised relation extraction method for aircraft fault data, characterized in that, Specifically, the following steps are included: Step 1: Relationship Classification; Specifically as follows: Step 1.1: For the input text H = [h1, h2, ..., h...] L ], h i This represents the i-th character in the text, where i ∈ [1, L], and L represents the total number of characters in the text; a leading and trailing symbol is inserted only before and after each identified entity position [E]. istart ],[E iend ], to mark the start and end positions of entities in the text, and no symbols are inserted before or after the position of unrecognized entities, as shown in formula (1); H=[h1,...,[E istart ],h i ,[E iend ],...,[E jstart ],h j ,[E jend ],...h L ] (1) The above formula means that only the i-th character and the i-th character are recognized entities, and a leading symbol and a trailing symbol are inserted before and after them respectively; After updating the word vectors of the text, all leading symbols [E] are processed. istart ], [E jstart ] Perform vector concatenation to obtain vector h as the vector representation of entity pairs in the text, which is used to characterize the relationship information between entities in the text, as shown in formula (2); in, d represents the dimension of the word vector; The leading symbol [E] istart ], [E jstart The entity vector information; during training, the suffix is used to tell the module the position of the last character of the entity; Step 1.2: Use a two-layer fully connected layer as a relation classifier, as shown in formula (3); Among them, I n For the predicted relation type, C ζ (·) represents a fully connected neural network, which processes text information... Convert to It consists of a text encoder and a relation classifier. The result of Formula 1 is given by n, which represents the number of pseudo-labels generated in the labeled data. E1 and E2 represent the manually labeled entity information, respectively. Let N real labels and M pseudo-labels be represented by the label set S. all ={g1,g2,...,g N p1, p2, ..., p M }, where g1, g2, ..., g N p1, p2, ..., p M Let N represent the real labels and M pseudo labels, respectively. To improve the relation extraction capability of the relation classification module parameter ζ, the following loss function is adopted. As shown in formula (4); Among them, I n These are the labels predicted from the labeled data, where n is the number of pseudo-labels generated in the labeled data, and I... m Here, represents the predicted labels from unlabeled data, m is the number of pseudo-labels generated in the unlabeled data, loss(·,·) is the cross-entropy loss function, and one_hot(·) indicates that the relation labels are represented using one-hot encoding. To facilitate adjusting the model's dependence on pseudo-labels, the optimization of classifier C... ζ When setting parameters, set parameter 0 ≤ q m ≤1 is used as the confidence coefficient of the pseudo-label loss term, as shown in formula (5). The confidence coefficient is the maximum output probability of the pseudo-label generation network in recognizing pseudo-labels. in, For unlabeled text information, This is a fully connected neural network, represented as a pseudo-label generation network; Step 2: Generating pseudo-tags; Specifically as follows: Step 2.1: To reduce the impact of network structure differences on the collaborative training process, the pseudo-label generation module and the relation classification module adopt the same network structure; the optimization function of the pseudo-label generation module is shown in formula (6); Wherein, the pseudo-label generation module parameter ζ′ represents the validation phase parameter of the relation classification module after one gradient update using the loss in formula (4), C ζ′ This indicates the network parameters generated by the pseudo-labels; Step 2.2: Optimization of parameter ζ′ of the pseudo-label generation module, as shown in formula (7); Where α is the meta-learning weight, and the validation loss of the relation classification module parameter ζ′ is used to update the pseudo-label generation module parameter η. The meta-learning parameters of the relation classification network are represented; the optimization function of the pseudo-label generation module is shown in Equation (8); Among them, g m For unlabeled data, H(·,·) represents the cross-entropy loss function. This represents the label predicted by the relation classification model. The pseudo-labels generated by the pseudo-label generation network are represented by one_hot(·), where one_hot(·) indicates relation label encoding. Step 3: Adaptive pseudo-label selection; Specifically as follows: Step 3.1: Global threshold calculation; The global threshold τ within each training time step of the model is estimated using the exponential moving average (EMA). t And initialize it to 1 / C, where C represents the number of relation types, as shown in formula (9); Where λ∈(0,1) is the momentum decay of the EMA, U represents the total number of data points, and μ represents the proportion of unlabeled data points to the total number of data points. Let τ represent the value of τ at the previous moment, and t represent the number of iterations; Step 3.2: Local threshold calculation; To determine the threshold corresponding to each relation type, the expected value of the predicted value of the pseudo-label generation module for relation type c is used as the specific threshold for that relation, as shown in formula (10). Where, p t (c)=[p t (1),p t (2),...,p t [C] is a local threshold encompassing all relation types, p t (1),p t (2),...,p t (C) represents the local thresholds for C different relation types. q represents the average threshold of the relationship type at time t-1. m (c) represents the threshold for the c-th relation type, q m (c) represents the local threshold at time t; Step 3.3: Combine the global threshold and the local threshold to obtain the adaptive global threshold. As shown in formula (11); Where, τ t This represents the global threshold for the t-th iteration, and MaxNorm(·) is the maximum normalization function. It is a function for finding the maximum value; the semi-supervised training objective for the t-th iteration. As shown in formula (12); in, Represented as a relational classification network The output, Q m The representation is the relation label encoding one_hot(g) m ).
2. A semi-supervised relation extraction method for aircraft fault data, characterized in that, The method also includes step 4: constructing an aircraft unstructured relation dataset for a semi-supervised relation extraction model for aircraft fault data; Specifically as follows: Step 4.1: Divide the sample set into a training set and a test set. The training set is used to train the model's parameters, and the test set is used to evaluate the model's performance on unknown data. Step 4.2: Train the semi-supervised relation extraction model based on adaptive meta-learning using the training set. Set a maximum number of training iterations during the training process. After each training iteration, the training set and test set data are redistributed according to the original ratio. Stop training when the maximum number of training iterations is reached. Step 4.3: Test the trained relation extraction model using the test set to obtain a relation extraction model for aircraft unstructured fault data.