A method for automatically constructing a high-end equipment manufacturing fault knowledge graph

Abstract

The application discloses a kind of methods for automatically constructing high-end equipment manufacturing industry fault knowledge graph, its characteristics are using the method of subject and object contrast learning, the interaction between the context features of subject and the context features of object is considered in the subject and object contrast learning in relation extraction, so that the context features between relevant subject and object are more similar, while the context features between irrelevant entities and objects are more different, to improve the performance of the model. Compared with the prior art, the application has the advantages of simple method, effectively improving the effect of the model, especially in the scene with more relationship types, it is effective and can improve the accuracy of subject and object extraction and relationship type judgment, and has good application prospect.

Description

Technical Field

[0001] This invention relates to the field of intelligent learning technology, specifically a method for automatically constructing a fault knowledge graph for the high-end equipment manufacturing industry. Background Technology

[0002] In the field of high-end equipment, the writing and application of fault case texts are of significant practical importance. These texts are typically written by business experts or professional maintenance personnel to record equipment anomalies and troubleshooting processes, including fault symptoms, causes, solutions, and troubleshooting procedures. However, the application and dissemination of this fault case knowledge are limited by the degree of data structuring.

[0003] To improve the utilization rate of fault case knowledge, identifying core entities in the data, such as component units, performance characteristics, fault states, and fault detection tools, and their relationships, is particularly important. Component units, as the basic building blocks of equipment, reflect the equipment's operating status through their performance characteristics; fault states indicate abnormal conditions that occur during equipment operation; and fault detection tools are the equipment or methods used to diagnose faults. Identifying these core entities and their relationships helps us better understand fault cases and improve the efficiency and accuracy of fault diagnosis. If we perform relationship extraction on the fault case text, it is easy to discover that there are many-to-many relationships between the subjects and objects of the fault; that is, one subject may correspond to multiple objects, and one object may correspond to multiple subjects.

[0004] Existing fault case knowledge relies on extracting the subject and object of relationships and determining the relationship type based on context. This involves judging each character in the text according to the context, allowing the model to learn fault case knowledge. However, this method only judges each character and lacks direct information about entities and relationships. Therefore, it is difficult to accurately determine the boundaries of relational entities and the type of relationship, resulting in low efficiency and accuracy in fault diagnosis. The application and promotion of this fault case knowledge are limited by the degree of data structuring. Summary of the Invention

[0005] The purpose of this invention is to provide a method for automatically constructing a fault knowledge graph for the high-end equipment manufacturing industry, addressing the shortcomings of existing technologies. It employs a subject-object comparative learning approach, considering the interaction between the contextual features of subjects and objects during relation extraction. This makes the contextual features between relevant subjects and objects more similar, while simultaneously increasing the differences in contextual features between unrelated entities and objects, thereby improving model performance. The method is simple and significantly improves model effectiveness, especially in scenarios with many relation types. It can improve the accuracy of subject-object extraction and relation type judgment, further enhancing the efficiency and accuracy of fault diagnosis, and has promising application prospects.

[0006] The specific technical solution to achieve the purpose of this invention is: a method for automatically constructing a fault knowledge graph for high-end equipment manufacturing, characterized by employing a subject-object comparative learning method. In relation extraction, the subject-object comparative learning considers the interaction between the contextual features of the subject and the contextual features of the object, making the contextual features between related subjects and objects more similar, while simultaneously increasing the differences in contextual features between unrelated entities and objects, thereby improving model performance. Specifically, this method includes: obtaining fault case samples to be extracted; inputting the samples into a fault case knowledge relation extraction model; obtaining the predicted relation type and the subject and object of the relation; training the fault case samples; and automatically constructing a fault knowledge graph for high-end equipment manufacturing. The training process for the fault case samples includes:

[0007] Step 1: Process the collected fault case samples and label the data according to the boundary position of the subject and object of the relationship and the type of relationship.

[0008] Step 2: Map the samples to the corresponding indices in the vocabulary and feed them into a self-attention model pre-trained on a large corpus to obtain the text representation.

[0009] Step 3: Add positional information to the text representation to emphasize the positional relationship between characters, decode it to obtain the required start and end pointers, and add auxiliary tasks here.

[0010] Step 5: Iterate the three types of start pointers and end pointers to train the fault case knowledge relationship extraction model multiple times, obtain the optimal model and save the model parameters to realize the automatic construction of fault knowledge graph of high-end equipment manufacturing industry.

[0011] The process of adding auxiliary tasks in step 3 includes:

[0012] 3-1: Obtain the subject and object representations of all samples in each batch during the training process based on their location.

[0013] 3-2: Calculate the loss value by performing similarity calculations on all entity representation vectors and object representation vectors.

[0014] 3-3: Perform backpropagation based on the loss value, update the model parameters, and train the model.

[0015] The step of calculating the loss value based on the similarity between all prompt representation vectors and the combined representation vector specifically includes:

[0016] During training, the subject and object representations of all relations in all samples of a batch are concatenated along the first dimension to obtain two new tensors. These tensors are then averaged along the second dimension to obtain two new matrices. In this process, it is necessary to ensure a one-to-one correspondence between subjects and objects of the same relation in the first dimension. When calculating similarity, it is necessary to ensure that the similarity between corresponding subjects and objects is as high as possible, and the similarity between non-corresponding subjects and objects is as low as possible. The rule is shown in equation (a) below:

[0017]

[0018] Where S represents the principal matrix, S = [s1, s2, ..., s... n ]s i Let i represent the subject of the i-th relation. This indicates the calculation of similarity, where O represents the object matrix, O = [o1, o2, ..., o2]. n o i Let j represent the object of the j-th relation.

[0019] The tags are constructed such that the relationship between corresponding relationships is 1. The tag construction rules are shown in Table 1 below:

[0020] Table 1 Label Construction Rules

[0021]

[0022]

[0023] In calculating cross-entropy loss, it is not necessary to deliberately assign labels to subjects and objects that do not correspond to each other; when calculating the loss, these labels will be made as large as possible.

[0024] Furthermore, the process of calculating similarity is as follows: the subject representation vector and the object representation vector are respectively S = [s1, s2, ..., s...]. n ], O = [o1, o2, ..., o n The calculation is shown in equation (b) below:

[0025]

[0026] Where subject represents the subject vector, object represents the object vector, ||subject|| represents the magnitude of the subject vector, and ||object|| represents the magnitude of the object vector.

[0027] The further, final losses specifically comprise four parts:

[0028] The loss is compared to the learning loss, entity extraction loss, the starting position loss of the subject and object in the sequence, and the ending position loss of the subject and object in the sequence.

[0029] The three start and end pointers obtained are used to train the model. For the relation extraction task, three global pointers are needed. One is used to extract entities, which is a subsequence of the input. The global pointer will output a G... C×L×L The matrix is ​​given, where C is the number of classes, representing the number of relation types, and L is the sequence length, representing the maximum length of the input sequence. The goal is to find elements greater than a threshold, i.e., g. cij >threshold. Where c∈C represents a relation type, i∈L represents the starting position of the entity in the sequence, and j∈L represents the ending position of the entity in the sequence. A global pointer is used to extract the head positions of the relevant subjects and objects. The output shape of the global pointer is the same as before, while each element g... cij They have different meanings. c∈C represents a relation type, i∈L represents the header of the subject entity, and j∈L represents the header of the object entity. Each element g in the last global pointer output cij This indicates the ending positions of the subject and object entities in the input sequence that have a relation. Combining the outputs of the three global pointers yields the relation triple (s, r, o).

[0030] Compared with existing technologies, this invention utilizes subject-object contrastive learning in relation extraction, considering the interaction between the contextual features of the subject and the contextual features of the object. This makes the contextual features between relevant subjects and objects more similar, while simultaneously increasing the differences in contextual features between unrelated entities and objects. This consistently leads to performance improvements. Based on a simple yet powerful baseline (an English language model based on BERT-base-cased, followed by an MLP layer, and finally a CRF decoder), extensive experiments were conducted on two widely used English relation extraction datasets to validate the effectiveness of the proposed subject-object contrastive learning. Experimental results show that this invention does indeed improve model performance, especially in scenarios with many relation types, significantly improving the accuracy of subject and object extraction and relation type judgment, demonstrating promising application prospects. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention;

[0032] Figure 2 A schematic diagram showing the one-to-one correspondence between the subject representation and the object representation in the first dimension;

[0033] Figure 3 A diagram illustrating the similarity calculation between each subject and all objects;

[0034] Figure 4 This is a table comparing the present invention with the prior art.

[0035] The present invention will be further described in detail through the following specific implementation examples.

[0036] Example 1

[0037] See Figure 1 The following steps will automatically construct a fault knowledge graph for the high-end equipment manufacturing industry:

[0038] 1. Data labeling

[0039] The collected high-end equipment failure case texts are labeled according to the boundary positions of the subject and object of the relationship and the type of relationship.

[0040] 2. Data processing

[0041] Data segmentation is performed to ensure that a relation triple is not split into two text segments, while also ensuring that contextual information is not lost. The purpose of segmentation is to shorten the length of the input text and reduce costs.

[0042] 3. Process the data into indexes corresponding to the words in the vocabulary list.

[0043] See Figure 2 Based on location information, subject and object representations are obtained, ensuring a one-to-one correspondence between subjects and objects with the same relationship in the first dimension. The data is processed into indices corresponding to words in the vocabulary, and the label corresponding to each sample is quantified, that is, the start and end positions of the relation subject, the start and end positions of the relation object, and the relation type are recorded respectively, so as to be input into the model for calculation.

[0044] 4. Building the model

[0045] Three decoders are concatenated after any Transformer encoder model. The decoders can be any entity extraction decoder, conditional random field, Span, global pointer, etc.

[0046] 5. Calculate the loss

[0047] See Figure 3When calculating similarity, each subject is compared with all objects, ensuring that the similarity between subjects and objects with the same relationship is as high as possible, while the similarity between subjects and objects with different relationships is as low as possible. When calculating the loss, all subjects and objects with relationships in each training batch are extracted based on their location information. Due to different contexts, even if the same entity appears in different samples, its representation will be different. All subjects and objects are concatenated in the first dimension, and the average is calculated in the second dimension, resulting in two matrices of the same shape, with subjects and objects with the same relationship corresponding one-to-one in the first dimension. The similarity of each subject with all objects is calculated, and when constructing similarity labels, subjects and objects with the same relationship are guaranteed to have a similarity of 1. This allows subjects and objects with the same relationship to become increasingly similar during model training, while the differences between subjects and objects with different relationships become increasingly larger.

[0048] 6. Optimize the model

[0049] Backpropagation is performed based on the loss to optimize the model.

[0050] 7. Iterative Training

[0051] The model is trained through multiple iterations to obtain the optimal model.

[0052] 8. Prediction Results

[0053] After obtaining the optimal model, predictions are made on unlabeled, unstructured equipment failure case texts. The prediction results are then processed to obtain structured knowledge data of the failure cases.

[0054] See Figure 4 This invention is compared with previous methods, where Baseline is the basic scheme of comparison, and Baseline+SOC is the invention itself. All others refer to previous schemes. It can be seen that the final score after adding this invention is higher than without it. The highest score is achieved on the MYT dataset, and the score on the WebNLG dataset is also very high. The reason for the lower score is due to the very poor Baseline of only 88.61. Even on such a poor Baseline, adding this scheme can improve the score to 90.44. The effect is very significant.

[0055] The above embodiments should be understood as being used only to illustrate the present invention and not to limit the scope of protection of the present invention. After reading the contents of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for automatically constructing a fault knowledge graph for high-end equipment manufacturing, characterized in that, The method includes: obtaining fault case samples to be extracted, inputting the samples into a fault case knowledge relation extraction model, obtaining the predicted relation types and the subjects and objects of the relations, training the fault case samples, and automatically constructing a fault knowledge graph for the high-end equipment manufacturing industry. The training of the fault case samples includes the following steps: Step 1: Case Sample Processing The collected fault case samples are labeled according to the boundary positions of the relational subjects and objects and the type of relation; Step 2: Text Representation The samples are mapped to the corresponding indices in the vocabulary and fed into a self-attention model pre-trained on a large corpus to obtain text representations. Step 3: Obtaining the start and end pointers Positional information is added to the text representation, and then it is decoded to obtain the start and end pointers of three global pointers, and auxiliary tasks are added here; Step 4: Train the model The three types of start pointers and end pointers are used to iteratively train the fault case knowledge relationship extraction model, obtain the optimal model and save the model parameters, so as to realize the automatic construction of fault knowledge graph of high-end equipment manufacturing industry. Step 3 yields the start and end pointers of three global pointers. One global pointer is used to extract entities, which are subsequences of the input. The global pointer will output a G... C×L×L A matrix, where C is the number of classes, representing the number of relation types; L is the sequence length, representing the maximum length of the input sequence; find the elements greater than a threshold, i.e., g. cij > threshold, where c C represents a relation type; i L represents the starting position of the entity in the sequence; j L represents the end position of the entity in the sequence; another global pointer is used to extract the head positions of the relevant subjects and objects. The output shape of the global pointer is the same as that of the global pointer used to extract entities, while each element g cij They have different meanings, c C represents a relation type; i L represents the header of the main entity; j L represents the header of the object entity; each element g in the output of the last global pointer... cij This indicates the end position of the subject and object entities in the input sequence that have a relation. After combining the outputs of the three global pointers, the relation triple (s, r, o) can be obtained.

2. The method for automatically constructing a fault knowledge graph for high-end equipment manufacturing according to claim 1, characterized in that, Adding auxiliary tasks in step 3 specifically includes: 3-1: Obtain the subject and object representations of all samples in each batch during the training process based on their location; 3-2: Calculate the similarity between all entity representation vectors and object representation vectors to obtain the cross-entropy loss value; 3-3: Perform backpropagation based on the cross-entropy loss value, update the model parameters, and train the model.

3. The method for automatically constructing a fault knowledge graph for high-end equipment manufacturing according to claim 2, characterized in that, The similarity calculation between all entity representation vectors and object representation vectors yields the cross-entropy loss value, specifically including: 1): Concatenate the subject and object representations of all relations in all samples of a batch during the training process according to the first dimension to obtain two new tensors. Then, average them according to the second dimension to obtain two new matrices. In this process, it is necessary to ensure that the subjects and objects of the same relation correspond one-to-one in the first dimension. 2) When calculating similarity, it is necessary to ensure that the similarity between corresponding subjects and objects is as high as possible, and the similarity between non-corresponding subjects and objects is as low as possible. The rule is shown in equation (a) below: （a）； Where S represents the principal matrix, S=[s1,s2,......,s n ]s i Represents the subject of the i-th relation; ⨂ denotes the similarity calculation; O represents the object matrix, O=[o1,o2,......,o... n o i Let j represent the object of the j-th relation; 3) Construct tags such that the relationship between corresponding relationships is 1. The tag construction rules are shown in Table 1 below: Table 1 Tag Construction Rules The similarity calculation process is as follows: the subject representation vector and the object representation vector are respectively S=[s1,s2,......,s... n ],O=[o1,o2,......,o n The calculation is shown in equation (b) below: （b）； Where subject represents the subject vector; object represents the object vector; ||subject|| represents the magnitude of the subject vector; ||object|| represents the magnitude of the object vector; The cross-entropy loss value includes four parts: contrastive learning loss, entity extraction loss, the starting position loss of the subject and object in the sequence, and the ending position loss of the subject and object in the sequence.

Images