Fault diagnosis model construction method based on knowledge graph subgraph embedding

Abstract

The invention provides a fault diagnosis model construction method based on knowledge graph sub-graph embedding, and relates to the technical field of fault diagnosis, and the method comprises the steps: collecting an unstructured text, cleaning and correcting the unstructured text to obtain an initial corpus, constructing a fault knowledge graph ontology and a data layer, generating a sub-graph by taking a fault object as a center, converting the sub-graph into a training set, and carrying out the fault diagnosis. And constructing a fault dependent encoder, synchronously training the encoder and the large language model to obtain a diagnosis model, and reasoning to generate a reply. By means of knowledge graph sub-graph embedding, fault dependency encoder, interleaving training and the like, structured storage and semantic comprehension capabilities are integrated, accurate and efficient fault diagnosis of a complex system is realized, and the problems of knowledge fragmentation and model adaptation are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fault diagnosis, in particular to a fault diagnosis model construction method based on knowledge graph subgraph embedding. BACKGROUND

[0002] With the rapid development of industrial equipment towards complexity and intelligence, the types of faults are increasingly diverse and the correlation is complex, and the unstructured text data (such as fault cases and maintenance manuals) relied on by traditional fault diagnosis is growing explosively. The key information such as fault objects, causes and phenomena contained in these data is scattered and not uniformly expressed, which is difficult to be directly used for efficient diagnosis. At the same time, although large language models have strong semantic understanding ability, they lack structured knowledge support, and knowledge graphs need to be adapted to models to play their value, so the demand for technology integration is urgent.

[0003] Traditional fault diagnosis methods have obvious limitations: rule-based methods rely on manual rule setting and are difficult to cope with complex equipment multi-fault scenarios, and have poor scalability; pure machine learning methods rely on manual feature engineering and are insufficient in mining entity correlation implied in unstructured text; large language models are easily affected by knowledge fragmentation when applied alone, and the generated answers may deviate from the actual fault logic; the structured information of knowledge graphs is difficult to directly input into the model, and the adaptability of the two is poor, which makes it difficult to balance the accuracy and efficiency of diagnosis.

[0004] Therefore, it is necessary to design a fault diagnosis model construction method based on knowledge graph subgraph embedding to solve the problems of low utilization rate of unstructured fault data, insufficient adaptation of structured knowledge and language models, dependence on manual fault diagnosis and poor accuracy, and difficulty in coping with complex equipment multi-fault scenarios in the prior art. SUMMARY

[0005] In view of this, the present application provides a fault diagnosis model construction method based on knowledge graph subgraph embedding, aiming at solving the problems of low utilization rate of unstructured fault data, insufficient adaptation of structured knowledge and language models, and difficulty in balancing the accuracy and efficiency of fault diagnosis in the prior art.

[0006] In one aspect, the present application provides a fault diagnosis model construction method based on knowledge graph subgraph embedding, comprising: collecting unstructured text data, and performing cleaning and error correction processing on the unstructured text data to obtain an initial corpus; determining entity types, defining the relationship between entities, completing the construction of the fault knowledge graph ontology, performing named entity recognition based on a preset method, extracting entities corresponding to the entity types from the initial corpus using a large language model, merging entities with consistent semantics in the extracted entities to obtain semi-structured data, extracting the relationship between entities in the semi-structured data to obtain triples, and storing the triples to obtain a fault knowledge graph data layer; extracting fault objects from the fault knowledge graph data layer, generating a subgraph with the fault object as the center node, converting the subgraph into a training data set composed of natural language descriptions, constructing a fault dependency encoder, and converting the subgraph using the fault dependency encoder to obtain a graph embedding token; synchronously training the fault dependency encoder and the large language model using the training data set to obtain a fault diagnosis model, extracting fault phenomena from user input fault questions, matching fault phenomenon entities from the fault knowledge graph data layer using the fault phenomena, generating a subgraph with the fault phenomenon entity as the center, inputting the subgraph into the fault dependency encoder to obtain the graph embedding token, inputting the graph embedding token into the fault diagnosis model to obtain a fault diagnosis answer, evaluating the fault diagnosis answer and summarizing to generate a fault diagnosis reply.

[0007] Further, when cleaning and correcting the unstructured text data, it includes: removing duplicate text, meaningless characters, and text content unrelated to fault diagnosis from the unstructured text data; identifying misspelled words, syntax errors, and ambiguous content in the unstructured text data, and correcting the identified error content with reference to common terms and standard expressions in the fault diagnosis field; The initial corpus includes fault case summaries, fault phenomena, fault analysis, and treatment information.

[0008] Further, the entity types include fault objects, fault causes, fault phenomena, and maintenance measures. The relationship between entities includes the occurrence relationship between fault objects and fault phenomena, the caused relationship between fault phenomena and fault causes, the phenomenon-maintenance measure relationship between fault phenomena and maintenance measures, and the cause-maintenance measure relationship between fault causes and maintenance measures. When performing named entity recognition based on a preset method and extracting entities corresponding to the entity types from the initial corpus using a large language model, it includes: The preset method is a json-like format prompt; The json-like format prompt includes four fields: instruction, schema, example, and input. providing a preset number of examples for the large language model, and using the large language model to extract fault objects, fault causes, fault phenomena, and maintenance measures corresponding to the entity type from unstructured text of the initial corpus; When the entities with consistent semantics in the extracted entities are merged to obtain semi-structured data, the method comprises: calculating the edit distance between different entities, screening entities with an edit distance less than a preset distance threshold, comparing the semantic similarity of the screened entities, retaining entities with a semantic similarity greater than a preset similarity threshold, and finally manually screening and confirming entities with consistent semantics and merging to obtain semi-structured data.

[0009] Further, extracting the relationships between the entities in the semi-structured data to obtain triples, and storing the triples to obtain a fault knowledge graph data layer, comprises: According to the relationships between the entities defined in the constructed fault knowledge graph ontology, the corresponding relationships between fault objects and fault phenomena, fault phenomena and fault causes, fault phenomena and maintenance measures, and fault causes and maintenance measures are identified from the semi-structured data, and the triples are formed in the format of head entity-relation-tail entity. Each triple contains a head entity, a relationship between entities, and a tail entity. The head entity and the tail entity are both entities in the semi-structured data, and the relationship is the relationship between entities defined when the fault knowledge graph ontology is constructed. Selecting a graph database as a storage carrier, importing the generated triples into the graph database one by one, establishing an association index of entities and relationships in the graph database, and the fault knowledge graph data layer contains all the triples and entity association information.

[0010] Further, extracting fault objects from the fault knowledge graph data layer and generating subgraphs with the fault objects as center nodes, and converting the subgraphs into a training data set composed of natural language descriptions, comprises: Traverse all entities in the fault knowledge graph data layer, and filter out entities of the entity type fault object to obtain a list of all fault objects; Traverse each fault object node in the list of fault objects, and perform a search on each fault object node to obtain all adjacent nodes within a preset subgraph depth jump range of the fault object node, the relationships between entities of the fault object node and adjacent nodes, and the relationships between entities of adjacent nodes; When the number of adjacent nodes retrieved is greater than the preset number of subgraph nodes, multiple splitting retrievals are performed, the adjacent nodes are grouped and combined, each combination contains the fault object node and part of the adjacent nodes, and multiple subgraphs are generated, each of which contains the preset number of subgraph nodes, and all the subgraphs completely cover all the adjacent nodes; When the number of adjacent nodes retrieved is less than or equal to the preset number of subgraph nodes, a combination retrieval is performed, the fault object node and all adjacent nodes are combined, and a subgraph with a node number less than or equal to the preset number of subgraph nodes is generated; The natural language description of each subgraph includes fault background task description, fault diagnosis related question, subgraph node information, subgraph node connection relationship, and preset answer; The natural language descriptions of all the subgraphs are sorted and summarized to form the training data set.

[0011] Further, a fault dependency encoder is constructed, and the fault dependency encoder is used to convert the subgraph to obtain graph embedding tokens, including: The fault dependency encoder is composed of a BERT component, a multi-graph neural component, and a domain projection component; The BERT component is used to process the text information of each node in the subgraph, and different lengths of node text are converted into node features with consistent lengths; The relationship between nodes in the subgraph is represented by an adjacency matrix, and the element value in the adjacency matrix is determined according to whether there is an association relationship between nodes, if there is an association relationship, the element value is 1, if there is no association relationship, the element value is 0; All the node features are integrated to form a feature matrix, the number of rows of the feature matrix is the number of nodes in the subgraph, and the number of columns is the dimension of the node feature; The feature matrix and the adjacency matrix are input into the multi-graph neural component, the multi-graph neural component learns node information and node relationship through multiple rounds, and outputs the overall feature of the subgraph; The overall feature is input into the domain projection component, and the domain projection component converts the overall feature into graph embedding tokens aligned with the text features of the large language model.

[0012] Further, the training data set is used to synchronously train the fault dependency encoder and the large language model to obtain a fault diagnosis model, including: The parameters of the BERT component in the fault-dependent encoder are fixed and not adjusted; the base parameters of the large language model are fixed and not adjusted; trainable Lora units are added at the Q and K matrices of the multi-head self-attention of each layer of the large language model; and the multi-graph layer neural component, the domain projection component, and the Lora unit of the fault-dependent encoder are taken as trainable parameter objects; The trainable parameter objects are synchronously trained in an interleaved training manner. In a single interleaved training process, the fault-dependent encoder is trained using the training data set for a predetermined number of times, and the large language model is trained using the training data set for a predetermined number of times. The interleaved training process is repeated for a predetermined number of rounds. In each round of interleaved training, samples are selected from the training data set, and the fault background task description, the fault diagnosis related question, and the corresponding graph embedding token in the samples are input into the fault-dependent encoder and the large language model being trained to obtain model output results. The model output results are compared with the preset answers in the samples, and model errors are calculated by a cross-entropy loss function. The model errors are fed back to the trainable parameter objects by a back propagation algorithm, and the weight parameters and bias parameters of the multi-graph layer neural component, the projection matrix parameters of the domain projection component, and the low-rank matrix parameters of the Lora unit are adjusted based on the model errors. When the model error is greater than a preset error threshold, the number of rounds of interleaved training is increased, and the parameter adjustment step is repeated until the model error is less than or equal to the preset error threshold. When the model error is less than or equal to the preset error threshold, the interleaved training is stopped, and the fault diagnosis model is obtained.

[0013] Further, when extracting fault phenomena from the fault question input by the user and matching fault phenomenon entities from the fault knowledge graph data layer using the fault phenomena, the following steps are included: The fault-related question input by the user is received, and the fault-related question is input into the large language model. The large language model completes named entity recognition based on In-Context Learning, extracts text content related to fault phenomena from the fault-related question as fault phenomenon description. The fault phenomenon description is compared with the fault phenomenon entities in the fault knowledge graph data layer using a BM25 method and a semantic similarity hybrid method. The matching degree of the fault phenomenon description and each fault phenomenon entity is calculated. The matching degree is a weighted average of the similarity calculated by the BM25 method and the semantic similarity. The fault phenomenon entity with the highest matching degree is selected as the target fault phenomenon entity.

[0014] Further, the subgraph is generated centered on the fault phenomenon entity, the subgraph is input into the fault dependency encoder to obtain the graph embedding token, and then the graph embedding token is input into the fault diagnosis model to obtain the fault diagnosis answer, comprising: Based on the preset subgraph depth and the preset subgraph node number, the subgraph is collected from the fault knowledge graph data layer with the target fault phenomenon entity as the center node, and the collection process is consistent with the way of extracting a fault object from the fault knowledge graph data layer and generating a subgraph with the fault object as the center node. The collected subgraph is input into the trained fault dependency encoder, and the BERT component, multi-graph layer neural component and domain projection component are processed to output the corresponding graph embedding token. The graph embedding token, user input fault problem and fault background task description are input into the trained fault diagnosis model together, and the fault diagnosis model analyzes and processes the input information to output multiple fault diagnosis answers.

[0015] Further, when evaluating the fault diagnosis answers and generating a fault diagnosis reply, comprising: Input all the fault diagnosis answers into a large language model, the large language model evaluates the usefulness of each answer to answer the user's target question, and scores all the fault diagnosis answers according to the usefulness; Arrange the answers in order from high to low according to the score to form an ordered answer list; Add the fault diagnosis answers in the ordered answer list greater than the high score threshold to the new context window in order according to the score from high to low; When the content in the added context window reaches the preset token limit, stop adding; When the content in the added context window does not reach the preset token limit, continue adding until the content in the context window reaches the preset token limit; Based on the fault diagnosis answers greater than the high score threshold in the context window, integrate key information to generate a comprehensive fault diagnosis reply, and the fault diagnosis reply contains fault cause analysis and maintenance measure suggestion.

[0016] Compared with the prior art, the beneficial effect of the application is that the fault diagnosis model construction method based on knowledge graph subgraph embedding of the application realizes the structured storage and associated mining of fault information by means of a knowledge graph, solves the problem of data fragmentation, relies on the strong semantic understanding ability of a large language model, improves the diagnostic logic accuracy through subgraph embedding and model fusion, reduces the dependence on artificial feature engineering, and efficiently outputs a reply containing fault cause analysis and maintenance measures, thereby significantly improving the accuracy and efficiency of fault diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustration only and are not intended to limit the application thereto. Moreover, like reference numerals in the drawings designate similar parts throughout the several views. In the drawings: Figure 1 A flow chart of the fault diagnosis model construction method based on knowledge graph subgraph embedding provided for the embodiments of the application is shown in the figure; Figure 2 A schematic diagram of the construction of the fault field knowledge graph in the examples of the application is shown in the figure; Figure 3 A graph of the large model instruction fine-tuning method enhanced by the graph convolution network in the examples of the application is shown in the figure; Figure 4 A framework diagram for realizing fault diagnosis based on multi-round dialogue in the examples of the application is shown in the figure. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood and so that the scope of the present application can be conveyed to those skilled in the art. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0019] Referring to Figures 1-2 In some embodiments of the present application, a fault diagnosis model construction method based on knowledge graph subgraph embedding includes: Collecting unstructured text data and performing cleaning and error correction processing on the unstructured text data to obtain an initial corpus; determining entity types, defining relationships between entities, completing the construction of the fault knowledge graph ontology, performing named entity recognition based on a predetermined method, extracting entities corresponding to the entity types from the initial corpus using a large language model, merging entities with consistent semantics among the extracted entities to obtain semi-structured data, extracting relationships between entities in the semi-structured data to obtain triples, and storing the triples to obtain a fault knowledge graph data layer; extracting fault objects from the fault knowledge graph data layer, generating a subgraph with the fault objects as center nodes, converting the subgraph into a training data set composed of natural language descriptions, constructing a fault dependency encoder, and converting the subgraph using the fault dependency encoder to obtain a graph embedding token; synchronously training the fault dependency encoder and the large language model using the training data set to obtain a fault diagnosis model, extracting fault phenomena from user input fault questions, matching fault phenomenon entities from the fault knowledge graph data layer using the fault phenomena, generating a subgraph with the fault phenomenon entities as the center, inputting the subgraph into the fault dependency encoder to obtain the graph embedding token, inputting the graph embedding token into the fault diagnosis model to obtain a fault diagnosis answer, evaluating the fault diagnosis answer, and summarizing to generate a fault diagnosis reply.

[0020] Specifically, unstructured text data is the basic data source for building fault diagnosis models, mainly including two categories of fault case analysis data and maintenance manual, and both types of data have the characteristics of fixed format, scattered information, but contain core fault diagnosis related information. Among them, fault case analysis data mainly comes from industrial field operation records, post-fault review reports, and fault troubleshooting logs of technical personnel, etc. The content covers scene information (such as equipment operating conditions, environmental temperature, load conditions, etc.), specific fault phenomenon description (such as equipment abnormal sound parts, error code at shutdown, parameter fluctuation range, etc.), fault troubleshooting process record (such as sequentially detected components, used detection tools, excluded non-fault reasons, etc.), final positioning of fault root cause (such as component wear, poor circuit contact, program logic error, etc.), and effect feedback after fault handling (such as equipment operating parameters after maintenance, whether the fault recurs, etc.), and some cases also contain the historical frequency of similar faults and difference comparison analysis. Maintenance manual is mainly the official technical document provided by the equipment manufacturer, the unified maintenance specification manual of the industry or the internal maintenance experience document of the enterprise, which involves the structure of each component system (mechanical, electrical, control system, etc.), the function parameters and normal running threshold of each component, the classification list of common faults, the standard maintenance process (disassembly steps, component replacement specification, installation and commissioning points, etc.) for different faults, the safety operation criteria in maintenance process, the performance verification method and qualified judgment standard after maintenance, and some manuals also mark the replacement cycle of vulnerable parts, the correlation between fault and component wear degree, and other key information.Furthermore, this type of unstructured text data often contains repetitive content (such as similar descriptions of multiple similar faults), meaningless information (such as garbled text and irrelevant notes), and expression flaws (such as typos, inconsistent terminology, and vague descriptions). Therefore, it needs to be transformed into a standardized initial corpus through subsequent cleaning and error correction to provide reliable data support for subsequent steps such as entity extraction and knowledge graph construction. The cleaning process for the unstructured text data specifically involves removing repetitive text with completely identical content or overlapping core information, garbled text and special symbols other than punctuation marks, and other meaningless characters that do not involve... Textual content unrelated to fault diagnosis, including fault descriptions and maintenance operations, undergoes error correction. Specifically, this involves identifying typos, grammatical errors, and vague descriptions that do not clearly define the fault location or its manifestations. These errors are then corrected by referring to common terminology and standard expressions in the fault diagnosis field. The fault knowledge graph ontology serves as the core framework and standard for constructing a fault domain knowledge graph. It first clarifies four entity types: fault object (referring to the equipment or component that has failed), fault cause (referring to the various factors that lead to the fault), and fault phenomenon (referring to the specific characteristics exhibited by the fault). The system first defines maintenance measures (operational steps used to resolve faults). Based on the logical flow of fault diagnosis, it defines four types of relationships between entities: the "occurrence" relationship between the fault object and the fault phenomenon (characterizing the occurrence of a specific fault phenomenon in the fault object), the "caused" relationship between the fault phenomenon and the fault cause (characterizing the fault phenomenon caused by a specific fault cause), the "phenomenon-maintenance measure" relationship between the fault phenomenon and maintenance measures (characterizing maintenance measures taken for a specific fault phenomenon), and the "cause-maintenance measure" relationship between the fault cause and maintenance measures (characterizing maintenance measures taken for a specific fault cause). This provides a clear basis for subsequent entity extraction from the initial corpus and relationship extraction from semi-structured data to generate triples, ensuring the structure and standardization of the knowledge graph. The fault knowledge graph ontology consists of entity types and relationships between entities. From the construction process, it is necessary to first determine the four entity types: fault object, fault cause, fault phenomenon, and maintenance measures, and then define the entity relationships such as the occurrence relationship between the fault object and the fault phenomenon, and the caused relationship between the fault phenomenon and the fault cause. By clarifying these two core components, the construction of the fault knowledge graph ontology is completed.

[0021] Specifically, "performing named entity recognition" refers to the operation of using a large language model to find out and extract specific content corresponding to the four entity types of fault object, fault reason, fault phenomenon and maintenance measure from the unstructured text of the initial corpus after providing a preset number of examples to the large language model according to the fault object, fault reason, fault phenomenon, maintenance measure and the four types of entity types specified in the constructed fault knowledge graph ontology. The extracted entities will be used in the subsequent step of merging semantically consistent entities to obtain semi-structured data; the json format prompt is a prompt text used for named entity recognition based on a preset method, which contains four fields of instruction, schema, example and input, wherein instruction is the task description text of the entity extraction task to be completed, schema is the list of entity labels to be extracted that specifies the entity types to be extracted, example is a dictionary list embedded with example text (input) and extracted entities (output) in example text, and input is the unstructured text in the initial corpus for entity recognition.

[0022] Specifically, extracting entities corresponding to the entity types from the initial corpus refers to, on the basis of having determined the four types of entities of fault object, fault reason, fault phenomenon and maintenance measure and completed the construction of the fault knowledge graph ontology, using a large language model to find out and extract specific content matching the four types of preset entity types, such as specific equipment or components belonging to "fault object" and specific factors belonging to "fault reason", from the initial corpus (containing fault case analysis, maintenance manual, etc.) that has been cleaned and corrected, rather than extracting irrelevant content.

[0023] Specifically, the "semantically consistent entity" refers to an entity extracted from the initial corpus, which has different text expressions but the same core meaning, such as "machine tool spindle jam" and "spindle rotation jam", which need to be identified and merged through edit distance calculation, semantic similarity comparison and manual screening; "semi-structured data" is the data obtained by extracting entities corresponding to preset entity types from the initial corpus using a large language model and merging semantically consistent entities, which contains fault objects, fault causes, fault phenomena, maintenance measures and their entity type labels, and is between unstructured text and fully structured data, without explicit association between entities; "triplet" is a structured data unit formed by identifying the relationship between entities in the semi-structured data according to the defined relationship between entities in the constructed fault knowledge graph ontology, in the format of (head entity, relationship, tail entity), for example (CNC machine tool spindle, occurs, rotation jam), the head entity and the tail entity are both entities in the semi-structured data, and the relationship is a preset association type between entities; "fault knowledge graph data layer" is the basic data part of the knowledge graph formed by storing all triplets into a graph database, which contains all triplets and the association index of entities and relationships, and can support fast query of entities and relationships, providing data support for subsequent subgraph generation, fault phenomenon entity matching and other operations.

[0024] Specifically, the subgraph is a local graph structure with a fault object or a fault phenomenon entity as the center node, extracted from the fault knowledge graph according to the preset depth and node number, containing peripheral associated nodes and relationships, used to focus on the core information related to a specific fault; the natural language description is to convert the subgraph into a text content containing fault background task description, fault diagnosis related questions, subgraph node information and connection relationship, and preset answers, so as to form a training data set, so that the model can understand the semantic information of the subgraph; the fault dependency encoder is composed of BERT component, multi-graph layer neural component and domain projection component, which converts the subgraph node text into uniform dimension node features through the BERT component, learns the overall features of the subgraph through the multi-graph layer neural component combined with the adjacency matrix, and then converts it into a representation aligned with the text features of the large language model through the domain projection component; the graph embedding token is a vector representation output by the fault dependency encoder, which can represent the global information of the subgraph and is compatible with the features of the large language model, used to input the fault diagnosis model to assist in generating accurate fault diagnosis answers.

[0025] Specifically, the preset mode refers to a json format prompt designed to realize named entity recognition and guide the large language model to extract and determine the entity corresponding to the entity type, which includes instruction, schema, example, and input fields. The instruction is a task description text, which clearly informs the large language model of the task of named entity recognition and extraction of entities corresponding to four types of entity types: fault object, fault reason, fault phenomenon, and maintenance measure. The schema is a list of entity labels to be extracted, which clearly lists the four types of entity types to be extracted and defines the extraction range. The example is a dictionary list containing "input" and "output". The "input" is a fault-related example text, and the "output" is the extracted entity corresponding to the four types of entity types from the example text, providing operation examples for the large language model. The input is the unstructured text in the initial corpus for entity recognition, which is the object of the large language model's entity extraction operation. The json format prompt provides task instructions, extraction standards, operation examples, and processing objects for the large language model, thereby standardizing the named entity recognition process and entity extraction results.

[0026] It can be understood that by cleaning and correcting the unstructured text, constructing the fault knowledge graph and generating subgraphs, combining the fault dependency encoder to realize subgraph embedding and large language model text feature alignment, and using interleaved training to optimize model adaptability, the structured storage and association mining of fault information is realized by relying on the knowledge graph to solve the problem of data fragmentation, the diagnostic logic accuracy is improved by relying on the strong semantic understanding ability of the large language model through subgraph embedding and model fusion, and the dependence on manual feature engineering is reduced, the reply containing fault reason analysis and maintenance measures is efficiently output, and the accuracy and efficiency of fault diagnosis are significantly improved.

[0027] In some embodiments of the present application, when the unstructured text data is cleaned and corrected, it includes: Removing duplicate text, meaningless characters, and text content unrelated to fault diagnosis from the unstructured text data; Identifying misspelled words, syntax errors, and ambiguous content in the unstructured text data, and correcting the identified error content by referring to general terms and standard expressions in the fault diagnosis field; The initial corpus includes fault case summaries, fault phenomena, fault analysis, and processing information.

[0028] Specifically, the collected unstructured text data includes fault case analysis data and maintenance manuals; the repetitive text is a text segment with completely consistent content or overlapping core information, the meaningless character is a random code or special symbol other than a punctuation mark, the text content irrelevant to fault diagnosis is a text not involving fault description or maintenance operation, and the expression ambiguous content is a text segment not explicitly indicating a fault site or fault performance.

[0029] It can be understood that, by removing redundant invalid information and correcting incorrect content, the accuracy and effectiveness of the initial corpus are ensured. High-quality data foundation is provided for subsequent entity extraction and relationship construction, data noise interference in subsequent steps is reduced, and the reliability of knowledge graph construction is improved.

[0030] Referring to Figure 2 In some embodiments of the present application, the entity types include: a fault object, a fault cause, a fault phenomenon, and a maintenance measure. The inter-entity relationship includes: an occurrence relationship between the fault object and the fault phenomenon, a caused relationship between the fault phenomenon and the fault cause, a phenomenon-maintenance measure relationship between the fault phenomenon and the maintenance measure, and a cause-maintenance measure relationship between the fault cause and the maintenance measure. Based on the preset manner, the named entity recognition is performed, and the large language model is used to extract entities corresponding to the entity types from the initial corpus, including: The preset manner is a json-like format prompt. The json-like format prompt includes four fields of instruction, schema, example, and input. A preset number of examples are provided for the large language model, and the large language model is used to extract fault objects, fault causes, fault phenomena, and maintenance measures corresponding to the entity types from the unstructured text of the initial corpus. When the entities with consistent semantics in the extracted entities are merged to obtain semi-structured data, including: The edit distances between different entities are calculated, entities with an edit distance less than a preset distance threshold are screened, the semantic similarity of the screened entities is compared, entities with a semantic similarity greater than a preset similarity threshold are retained, and finally the entities with consistent semantics are confirmed by manual screening and merged to obtain semi-structured data.

[0031] Specifically, the fault object is a device or component that has failed, the fault cause is a factor that leads to the failure, the fault phenomenon is a characteristic exhibited by the failure, and the repair measure is an operation step to address the failure; the occurrence relationship indicates that the fault object exhibits the fault phenomenon, the caused relationship indicates that the fault phenomenon is caused by the fault cause, the phenomenon-repair measure relationship indicates a repair measure taken for the fault phenomenon, and the cause-repair measure relationship indicates a repair measure taken for the fault cause; the instruction is a task description text used to describe the named entity recognition and entity extraction tasks corresponding to the entity types that need to be completed, the schema is a label list corresponding to the entity types to be extracted, and is used to specify that the entity types to be extracted are the four types of fault object, fault cause, fault phenomenon, and repair measure, and the example is a dictionary list containing input and output, the input is unstructured text in the initial corpus used for entity recognition, and the output is the extracted entity corresponding to the entity type in the example text.

[0032] Specifically, the "preset quantity" is a quantitative standard set in advance for a specific link in the process of building a fault diagnosis model based on knowledge graph subgraph embedding. Specifically, it can refer to the number of examples provided to a large language model for a named entity recognition task, the target number of entities filtered by edit distance or semantic similarity in the entity merging process, etc. Its role is to provide clear execution basis for entity extraction, entity merging, and other operations, ensuring that these links proceed according to the expected standard, and ensuring the quality and efficiency of subsequent steps such as semi-structured data generation and knowledge graph construction. The "preset quantity" is obtained through the following specific operations: first, determine the specific application scenario it corresponds to (such as the number of named entity recognition examples, the number of entity filtering in entity merging, etc.), then analyze the task difficulty in this scenario (such as the complexity of entity types in named entity recognition), the historical performance of similar data (such as the number of effective examples in the past, the reasonable size of entity filtering), and then test the impact of different quantity values on the subsequent operation effect (such as entity extraction accuracy, entity merging quality), and finally select a value that balances operation effect and resource consumption as the preset quantity.

[0033] Specifically, the preset distance threshold is a critical value for judging the degree of text difference between different entities, corresponding to the edit distance, which is an index for measuring the number of insertion, deletion, replacement and other operations between two strings. When the edit distance between entities is less than the threshold, it indicates that the text difference between the two entities is small, and it has the value of further semantic comparison. The preset similarity threshold is a critical value for judging the closeness of semantic association between entities, corresponding to the semantic similarity, which reflects the consistency of entity meaning. When the semantic similarity between entities filtered by edit distance is greater than the threshold, it indicates that the semantic association between the two entities is close, and it can be determined as semantically consistent entities. The preset distance threshold and the preset similarity threshold are obtained by the following specific operations: first, prepare a sample set containing known semantically consistent and semantically inconsistent entities. This sample set covers common fault objects, fault causes, fault phenomena, maintenance measures and other entity types in the field of fault diagnosis. Then, select multiple different distance values as candidate distance thresholds and multiple different similarity values as candidate similarity thresholds. Then, apply each candidate distance threshold and candidate similarity threshold to the entity merging operation of the sample set in turn, and count the accuracy of entity merging under each operation. Finally, according to the statistical results, select the candidate distance value that can make the accuracy of entity merging reach the expected standard as the preset distance threshold, and select the candidate similarity value that can make the accuracy of entity merging reach the expected standard as the preset similarity threshold.

[0034] Specifically, a preset number of model input and output examples are given, such as one example , where is a predefined function (text format) about prompt, which is used to convert one task-related example into a natural language prompt. Given the task definition , the example , and a new input , the goal is to generate the output by LLM, that is: The prompt format designed here adopts a json-like string structure, which consists of the following four fields: 1) “instruction”: that is, the task description, which specifies the role of the model and the task to be completed in natural language; 2) “schema”: this is a list of labels that need to be extracted, which clearly indicates the key fields of the entity to be extracted, reflecting the user's needs, and is dynamically variable; 3) “example”: this is a list of embedded dictionaries, which contains “input” and “output” to give examples of completing the NER task in a specified scenario; 4) "input": refers to the source text for named entity recognition.

[0035] Using this method to extract entities from unstructured text data, the corresponding fault object, fault phenomenon, fault cause and maintenance measures in each fault case are extracted. The extracted entities can be merged by editing distance, semantic similarity and manual screening, etc. to complete knowledge fusion, and finally arranged in the form of semi-structured data.

[0036] It can be understood that, through the preset mode of the json format prompt, the pertinence and accuracy of the entity extraction of the large language model are improved; the entity merging logic solves the problem of semantic repetition and ensures the consistency of the semi-structured data, laying a standard foundation for the generation of triples.

[0037] Referring to Figure 2 As shown in the drawings, in some embodiments of the present application, the relationship between the entities in the semi-structured data is extracted to obtain triples, and the triples are stored to obtain a fault knowledge graph data layer, comprising: According to the inter-entity relationship defined in the constructed fault knowledge graph ontology, the corresponding relationship between the fault object and the fault phenomenon, the fault phenomenon and the fault cause, the fault phenomenon and the maintenance measure, and the fault cause and the maintenance measure is identified from the semi-structured data, and the triples are formed in the format of head entity-relation-tail entity. Each triple contains a head entity, an inter-entity relationship and a tail entity. The head entity and the tail entity are both entities in the semi-structured data, and the relationship is the inter-entity relationship defined when the fault knowledge graph ontology is constructed. Select a graph database as a storage carrier, and import the generated triples into the graph database one by one. An association index of entities and relationships is established in the graph database. The fault knowledge graph data layer contains all the triples and entity association information.

[0038] Specifically, the graph database is a carrier for storing the fault knowledge graph data layer, which can realize the data persistence of triples. The association index of entities and relationships is an index structure established in the graph database, which is used to quickly query the corresponding relationship and associated entities of an entity. The entity association information is the core component of the fault knowledge graph data layer, which contains the corresponding association content between the head entity, the relationship and the tail entity in all triples. First, the inter-entity relationship is extracted from the semi-structured data to generate triples in the form of (head entity, relationship, tail entity). Then, a graph database is selected as a storage carrier, and the generated triples are imported into the graph database one by one. During or after the import process, an association index of entities and relationships is established in the graph database. The entity association information is naturally formed during the storage process of the triples and contained in the fault knowledge graph data layer.

[0039] It can be understood that the relationship structure is realized by the head entity-relation-tail entity format, and the data storage and retrieval efficiency is improved by means of the graph database and the association index. The degree of structuring and the convenience of access of the fault knowledge graph data layer are ensured, and efficient data support is provided for subsequent subgraph generation.

[0040] In some embodiments of the present application, the fault objects are extracted from the fault knowledge graph data layer, and the subgraphs are generated with the fault objects as the center nodes. When the subgraphs are converted into a training data set composed of natural language descriptions, the method comprises the following steps: All entities in the fault knowledge graph data layer are traversed, and entities of the entity type of fault objects are screened out to obtain a list of all fault objects; Each fault object node in the list of fault objects is traversed, and for each fault object node, all adjacent nodes within a predetermined subgraph depth range of the fault object node, the inter-entity relationship between the fault object node and the adjacent nodes, and the inter-entity relationship between the adjacent nodes are retrieved and obtained; When the number of adjacent nodes retrieved and obtained is greater than the predetermined number of subgraph nodes, multiple split retrievals are performed, the adjacent nodes are grouped and combined, each combination contains the fault object node and part of the adjacent nodes, and multiple subgraphs are generated, each of which contains a predetermined number of subgraph nodes, and all the subgraphs completely cover all the adjacent nodes; When the number of adjacent nodes retrieved and obtained is less than or equal to the predetermined number of subgraph nodes, a combination retrieval is performed, the fault object node and all adjacent nodes are combined, and a subgraph with a node number less than or equal to the predetermined number of subgraph nodes is generated; The natural language description of each subgraph includes fault background task description, fault diagnosis related question, subgraph node information, subgraph node connection relationship, and predetermined answer; The natural language descriptions of all the subgraphs are sorted and summarized to form the training data set.

[0041] Specifically, the preset subgraph depth refers to the hierarchical boundary of retrieving adjacent nodes outward from the fault object node, which is used to define the knowledge range covered by the subgraph and determines the farthest association distance of the adjacent nodes that can be included in the subgraph and the center node; the preset subgraph node number refers to the upper limit of the total number of nodes allowed to be contained in each generated subgraph, which is used to control the size of the subgraph and ensure that the subgraph can adapt to the processing needs of the subsequent model; the preset subgraph depth and the preset subgraph node number are obtained through the following specific operations: first, the size characteristics such as the total amount of entities and the average association density between entities in the fault knowledge graph data layer are counted, and then the processing efficiency of the fault dependency encoder for different node number inputs and the text feature adaptation ability of the large language model are tested, combined with the integrity of the natural language description converted after the generation of the subgraph, the final values are determined after multiple groups of parameter debugging and comparison.

[0042] Specifically, the "preset subgraph depth hop" refers to the association level of retrieving adjacent nodes outward from the fault object node before generating the subgraph centered on the fault object node, and each hop represents a layer of direct association relationship, for example, when the preset subgraph depth hop is S, all adjacent nodes within the association range of the fault object node are retrieved, including the directly associated nodes (1 hop), the indirectly associated nodes through 1 node (2 hops), and S layers, and the entity relationship between nodes is also retrieved simultaneously, so as to limit the association range and information boundary of the subgraph and adapt to the subsequent subgraph node number specification and model processing needs.

[0043] Specifically, the fault background task description is a task description text related to fault diagnosis, the fault diagnosis related question is generated based on the entity and relationship information in the subgraph, the subgraph node information describes the content of each node in the subgraph, the subgraph node connection relationship describes the association relationship between the nodes in the subgraph, and the preset answer is determined based on the subgraph information and the fault diagnosis logic.

[0044] It can be understood that the preset node number requirement is adapted by the dynamic grouping combination strategy to ensure that the subgraph can completely cover the associated information and meet the model processing needs; the natural language description format of the subgraph is unified to provide standardized input for the construction of the training data set and improve the effectiveness of the model training.

[0045] Referring to Figure 3 In some embodiments of the present application, a fault dependency encoder is constructed, and the graph embedding token is obtained by converting the subgraph using the fault dependency encoder, which includes: The fault dependency encoder is composed of a BERT component, a multi-graph layer neural component, and a domain projection component; The BERT component is used to process the text information of each node in the subgraph, and different lengths of node text are converted into node features with consistent lengths; The relationship between nodes in the subgraph is represented by an adjacency matrix, and the element value in the adjacency matrix is determined according to whether there is an association relationship between the nodes. If there is an association relationship, the element value is 1, and if there is no association relationship, the element value is 0. All the node features are integrated to form a feature matrix, the number of rows of the feature matrix is the number of nodes in the subgraph, and the number of columns is the dimension of the node feature; The feature matrix and the adjacency matrix are input into the multi-graph layer neural component, which learns node information and relationship between nodes through multiple rounds, and outputs the overall feature of the subgraph; The overall feature is input into the domain projection component, which converts the overall feature into a graph embedding token aligned with the text feature of the large language model.

[0046] Specifically, the node feature is used to represent the semantic information of the node text, and the overall feature is used to represent the global information of the subgraph. The BERT component processes the text information of each node in the subgraph, which includes: the BERT component first performs word segmentation processing on the text information of each node in the subgraph, and then adds special symbols for identifying sequence start and separation to the word pieces. Subsequently, the word pieces are converted into initial vectors through the embedding layer of the BERT component, and the relationship between the word pieces is learned through multiple layers of multi-head self-attention mechanism and calculated through feedforward neural network. Finally, the vector representing the semantic information of the node text is output, and the output vectors corresponding to node texts of different lengths are adjusted to have consistent lengths. All the node features are integrated to form a feature matrix, which includes: first, the BERT component processes the text information of each node in the subgraph, and converts node texts of different lengths into node features of consistent lengths (i.e., vectors of fixed dimensions), and then takes each node in the subgraph as a unit, and arranges the node feature vectors corresponding to each node as a row of the feature matrix in the order of the arrangement of the nodes in the subgraph. Finally, the feature matrix is formed, wherein the number of rows of the feature matrix is the number of nodes in the subgraph, and the number of columns is the dimension of the node feature. The dimension of the node feature refers to the length of the node feature vector generated by the BERT component for each node in the subgraph. The dimension is determined by the output dimension of the BERT component used, and is a fixed value set when converting node texts of different lengths into vectors of consistent lengths. Its function is to realize the standardized representation of node text information, and provide a unified dimension input basis for subsequent learning of node information and relationship by inputting the feature matrix and the adjacency matrix into the multi-graph layer neural component.

[0047] Specifically, the fault dependency encoder based on graph convolution network is constructed, and the information in the knowledge graph is converted into graph embedding token, which is the basis for subsequent fine-tuning and reasoning based on the knowledge graph. The encoder consists of BERT, g layer GNN and a domain projector.

[0048] First, considering the inconsistency between the entity text descriptions in the knowledge graph, the BERT model is introduced to uniformly represent the text node_text of different lengths to generate node_embedding of consistent length as the node feature.

[0049] The knowledge graph is essentially a complex network, in which the nodes not only represent independent entities, but also the relationships between nodes carry important information. Then, considering the association between fault information, the relationship between nodes is represented by an adjacency matrix. And all node_embeddings of the subgraph and the adjacency matrix obtained from the association are input into the GNN. In this way, GNN can comprehensively learn the node information and the relationship between nodes, and finally obtain a comprehensive graph representation, i.e. graph_embedding.

[0050] Specifically, for the kth subgraph of the ith central node, a subgraph containing N nodes, the BERT is used to encode and represent each node information to obtain , which ensures that text of different lengths can have consistent embedding. Then, the embedding of multiple nodes is connected to obtain a feature matrix .

[0051] wherein, is the text length of node in the kth subgraph of the ith central node, is the hidden layer dimension of BERT, .

[0052] In multiple levels of graph neural network, the representation of each node is updated by using the features of neighbor nodes, so as to realize the transmission and aggregation of information. The initial node feature can be represented as , then the output of the layer of the fault dependency encoder can be represented as: wherein, , is the adjacency matrix, is the identity matrix. is the degree matrix of . , , the learnable parameter .

[0053] After graph convolution in the failure-dependent encoder, the g layer, the output is obtained . The output is then mapped by a domain projector to graph tokens and aligned with the language tokens of the LLM.

[0054] where the learnable parameters , are the final output.

[0055] Before each inference or training, the number of <graphtoken>All are constructed according to this method.

[0056] It can be understood that the node feature dimension is unified through the BERT component, the subgraph structure relationship is captured through the multi-layer neural component, and the feature alignment with the large language model is realized through the domain projection component. The adaptation problem of graph structure data and the language model is solved, and technical support is provided for the effective use of subgraph information.

[0057] Referring to Figure 3 As shown in the method for training the fault diagnosis model according to some embodiments of the present application, when the fault dependency encoder and the large language model are synchronously trained using the training data set, the method comprises the following steps: The parameters of the BERT component in the fault dependency encoder are fixed and not adjusted, the basic parameters of the large language model are fixed and not adjusted, a trainable Lora unit is added at the Q and K matrices of the multi-head self-attention of each layer of the large language model, and the multi-layer neural component, the domain projection component, and the Lora unit of the fault dependency encoder are taken as trainable parameter objects; The trainable parameter objects are synchronously trained using an interleaved training method. In a single interleaved training process, the fault dependency encoder is trained a preset number of times using the training data set, the large language model is trained a preset number of times using the training data set, and the interleaved training process is repeated a preset number of times. In each round of interleaved training, samples are selected from the training data set, the fault background task description, the fault diagnosis related question, and the corresponding graph embedding token in the samples are input into the fault dependency encoder and the large language model being trained, and the model output result is obtained. The model output result is compared with the preset answer in the sample, the model error is calculated through a cross-entropy loss function, the model error is fed back to the trainable parameter objects through a back propagation algorithm, and the weight parameters and bias parameters of the multi-layer neural component, the projection matrix parameters of the domain projection component, and the low-rank matrix parameters of the Lora unit are adjusted based on the model error. When the model error is greater than a preset error threshold, the number of rounds of interleaved training is increased, the parameter adjustment step is repeated, and the model error is less than or equal to the preset error threshold. When the model error is less than or equal to the preset error threshold, the interleaved training is stopped, and the fault diagnosis model is obtained.

[0058] Specifically, multi-head self-attention is a core component in the Transformer architecture, which refers to dividing the feature vectors of the input text into multiple independent subspaces, calculating the attention weights between queries (Q), keys (K), and values (V) in each subspace to capture local and global correlations, and then integrating the results of each subspace to enable the model to learn complex dependencies in the text from different dimensions in parallel; Q and K matrices are the core components of the multi-head self-attention mechanism in the Transformer architecture on which large language models are based, where Q is the Query matrix used to convert each position feature of the input text into a vector representing the query requirements of that position, and K is the Key matrix used to convert each position feature of the input text into a vector representing the information attributes of that position. The similarity between the two vectors (such as dot product operation) is calculated to obtain the attention weight, which measures the importance of different position information in the input text to the current position, thereby enabling the model to focus on key information. In claim 7, a trainable Lora unit is added to the Q and K matrices, which can fine-tune the parameters related to the attention mechanism without adjusting the base parameters of the large language model.

[0059] Specifically, the Lora unit is a trainable component added to the Q and K matrices of each layer of multi-head self-attention in the large language model during synchronous training of the fault dependency encoder and the large language model. It adopts a low-rank matrix decomposition structure and adjusts its low-rank matrix parameters through training to adapt to the fusion requirements of the graph embedding token in the fault diagnosis field and the text features of the large language model, while fixing the base parameters of the large language model to avoid significant adjustments to the model. The Lora unit, together with the multi-graph layer neural component and the domain projection component in the fault dependency encoder, serves as a trainable parameter object, and the error calculated by the cross-entropy loss function and the backpropagation algorithm are used for parameter optimization to support the development of the interleaved training process and obtain the fault diagnosis model.

[0060] Specifically, the "preset number of times" refers to the predetermined number of times of training the fault-dependent encoder and the large language model in a single process in interleaved training, the "preset number of rounds" refers to the total number of times of repeating the complete interleaved training process, the "model error" is the difference value between the model output result calculated by the cross-entropy loss function and the preset answer in the training sample, and the "preset error threshold" is a preset error threshold value for judging whether the model training has reached convergence and can stop training. The "preset number of times", "preset number of rounds" and "preset error threshold" are obtained through multiple pre-experiments: first, design multiple groups of different training times, rounds and error threshold candidate values, respectively, for model training, then evaluate the performance indicators such as fault diagnosis accuracy, recall rate and training efficiency of the model under different candidate values, and select the values that can make the model performance optimal and the training cost reasonable; the "model error" is calculated directly by the cross-entropy loss function after comparing the output result of the model with the preset answer in the sample.

[0061] Specifically, when adjusting the parameters based on the model error, the model error calculated by the cross-entropy loss function is first transmitted from the model output end to the components where the trainable parameters are located layer by layer through the backpropagation algorithm. For multi-layer neural components, the gradients of the weight parameters and bias parameters are calculated according to the error, and the parameters are updated using the gradient optimization method. For the field projection component, the gradient of the projection matrix is obtained through error feedback and the matrix elements are adjusted. For the low-rank matrix of the Lora unit, the gradient of the low-rank matrix is also calculated through backpropagation, and then the related matrix parameters after low-rank decomposition are updated to reduce the model error and make the model output closer to the preset answer in the sample.

[0062] Specifically, selecting samples from the training data set includes: selecting samples from the training data set compiled and summarized from the natural language descriptions of all subgraphs in a random sampling manner. The selected samples need to contain complete content such as fault background task description, fault diagnosis related questions, subgraph node information, subgraph node connection relationship and preset answer, which are used to input the fault-dependent encoder and the large language model in training to obtain subsequent model output results and calculate errors.

[0063] Specifically, the constructed network is trained using the preprocessed dataset. To ensure the coordination of internal parameters of the network, the training of the fault dependency encoder and the large model needs to be carried out at the same time. In the fault dependency encoder, the BERT model parameters are frozen, and all parameters in the GNN are trainable; in the large model, all parameters are frozen, and Lora trainable units are added at the Q and K matrices of each layer of multi-head self-attention for model fine-tuning. The method of interleaved training is adopted to simultaneously fit the trainable parameters of the fault dependency encoder and the large model. Specifically, in one interleaved training process, the GNN is trained m1 times, and the LLM is fine-tuned m2 times. It is generally recommended that the ratio between m1 and m2 is 100:1, and the training process is repeated n times.

[0064] Feature learning is performed on the input samples by forward propagation, and after all samples are propagated to the output layer, the output is compared with the answer (i.e. the true label) in the training set. The cross-entropy loss function is used as the loss function of the network, and the trainable parameters are updated using the backpropagation algorithm. The loss function is represented as follows: wherein, is the model after a small part of the parameters are reparameterized by Lora. is the model after a small part of the parameters are reparameterized by Lora. represents the fine-tuning dataset, (Dtrain, Dval) = (Dtrain, Dval) \ Dtrain represents one input-output pair thereof. is the result of the combination of Instruction, Query and Graph described in the training dataset prepared in step five, wherein Graph has been represented as by the fault dependency encoder. <graphtoken>in the form of a text. is the Answer in the training data set prepared in Step Five.

[0065] It can be understood that through the Lora unit and the interleaved training mode, the encoder and the large language model are cooperatively optimized while reducing the training cost. The trainable parameters are accurately adjusted, the adaptability of the model to the fault diagnosis task is quickly improved, and the performance and generalization ability of the diagnosis model are ensured.

[0066] Referring to Figure 4 As shown in the figure, in some embodiments of the present application, the fault phenomenon is extracted from the fault question input by the user, and the fault phenomenon entity is matched from the fault knowledge graph data layer using the fault phenomenon. Receive the fault-related question input by the user, input the fault-related question into the large language model, and the large language model completes named entity recognition based on In-Context Learning, extracts the text content related to the fault phenomenon from the fault-related question as the fault phenomenon description. Using the method of mixing BM25 and semantic similarity, the fault phenomenon description is compared with the fault phenomenon entity in the fault knowledge graph data layer; the matching degree of the fault phenomenon description and each fault phenomenon entity is calculated, the matching degree is the weighted average value of the similarity calculated by the BM25 method and the semantic similarity; The fault phenomenon entity with the highest matching degree is selected as the target fault phenomenon entity.

[0067] Specifically, the fault phenomenon description is used for the subsequent operation of matching the fault phenomenon entity from the fault knowledge graph data layer using the fault phenomenon; and the target fault phenomenon entity is used for the subsequent operation of generating a subgraph centered on the fault phenomenon entity.

[0068] Specifically, In-Context Learning (Context Learning) refers to the ability of a large language model to understand and complete a specific task without parameter updating, by only providing task description, examples or related context information in the input. In the above method, it is specifically embodied that after receiving the fault-related question input by the user, the large language model directly completes named entity recognition based on its understanding of language and the task logic (such as extracting fault phenomenon) implied in the input, accurately extracts the text content related to the fault phenomenon from the fault question, and provides a basis for subsequent matching of fault phenomenon entities in the fault knowledge graph.

[0069] Specifically, the "similarity calculated in the BM25 manner" is a retrieval similarity calculation method of the Bag-of-Words model based on word frequency statistics. The core principle is to evaluate the matching degree of the two by measuring the frequency of the keywords in the fault phenomenon description (query text) in the fault phenomenon entity (target text) of the fault knowledge graph data layer, the length of the target text, and the average length of all target texts. When calculating, the adjustment parameters (such as k1 to control the word frequency saturation effect, and b to control the document length normalization degree) are introduced to balance the influence of keyword frequency on similarity and the interference brought by text length difference, and finally output the quantitative score reflecting the closeness of the two word matching. The "semantic similarity" is a similarity evaluation method from the perspective of deep semantic association of text. It converts the fault phenomenon description and the fault phenomenon entity text into low-dimensional dense vector representation (i.e. embedding vector), captures the context meaning, implicit semantics and logical association of words in the text, and then calculates the similarity (such as cosine similarity, Euclidean distance, etc.) between the two vectors to get the quantitative result, which can effectively identify the text association with different words but similar semantics, and make up for the limitations of BM25 which only relies on word frequency statistics. The "similarity calculated in the BM25 manner" needs to go through the following specific operations: first, the fault phenomenon description (query text) and each fault phenomenon entity (target text) in the fault knowledge graph data layer are preprocessed respectively, including word segmentation (continuous text is split into independent words) and removing stop words (such as "de" and "he" which have no actual meaning); then, the key parameters are counted, including the frequency of each keyword in the query text in the target text (tf), the length of the target text (dl), the average length of all target texts (avgdl), and the preset adjustment parameters k1 (usually 1.2) and b (usually 0.75); finally, the above parameters are substituted into the BM25 calculation formula score = Σ [(tf × (k1 + 1)) / (tf + k1 × (1 - b + b × dl / avgdl)) × log ((N - n + 0.5) / (n + 0.5))], where N is the total number of target texts, and n is the number of target texts containing the keyword), and the result calculated is the similarity under the BM25 manner. The "semantic similarity" needs to go through the following specific operations: first, the fault phenomenon description and the fault phenomenon entity text are preprocessed in the same way as BM25; then, the preprocessed text is input into the pre-trained language model (such as BERT, RoBERTa, etc.), and the context semantic information of the text is captured through the encoder layer of the model, and the output vector corresponding to the [CLS] token of the model is taken as the sentence embedding vector of the whole text; finally, the cosine similarity formula similarity = (vector A · vector B) / (||vector A|| × ||vector B||)) is used to calculate the similarity between the two sentence embedding vectors, and the result obtained is the similarity at the semantic level.

[0070] It can be understood that the mixed matching method based on In-Context Learning extraction of fault phenomena combined with BM25 and semantic similarity improves the accuracy of fault phenomenon recognition and the precision of entity matching. It ensures accurate positioning of core information from user questions, provides accurate center nodes for subgraph generation in the subsequent reasoning stage, and improves the relevance of diagnostic answers.

[0071] Referring to Figure 4 As shown in the drawings, in some embodiments of the present application, when a subgraph is generated centered on the fault phenomenon entity, the subgraph is input into the fault dependency encoder to obtain the graph embedding token, and then the graph embedding token is input into the fault diagnosis model to obtain the fault diagnosis answer, including: Based on the preset subgraph depth and the preset subgraph node number, the subgraph is collected from the fault knowledge graph data layer with the target fault phenomenon entity as the center node, and the collection process is consistent with the way of extracting fault objects from the fault knowledge graph data layer and generating a subgraph centered on the fault object; The collected subgraph is input into the trained fault dependency encoder, which is processed by the BERT component, the multi-graph layer neural component, and the domain projection component, and outputs the corresponding graph embedding token; The graph embedding token, the user input fault question, and the fault background task description are input into the trained fault diagnosis model together, and the fault diagnosis model analyzes and processes the input information to output multiple fault diagnosis answers.

[0072] It can be understood that the generation of the graph embedding token and the input format of the diagnosis model are specified to standardize the reasoning process. It ensures that the trained model can stably output multiple fault diagnosis answers in actual application, improving the reliability of the reasoning result.

[0073] Referring to Figure 4 As shown in the drawings, in some embodiments of the present application, when the fault diagnosis answers are evaluated and fault diagnosis replies are generated, including: Input all the fault diagnosis answers into a large language model, the large language model evaluates the usefulness of each answer to answer the user's target question, and scores all the fault diagnosis answers according to the usefulness; Arrange the answers in order from high to low according to the score to form an ordered answer list; Add the fault diagnosis answers in the ordered answer list that are greater than the high score threshold to the new context window in order according to the score from high to low; When the content in the added context window reaches the preset token limit, stop adding; When the content in the post-addition context window does not reach the preset token limit, then continue to add until the content in the context window reaches the preset token limit; Based on the fault diagnosis answers in the context window greater than the high score threshold, integrate key information to generate a comprehensive fault diagnosis reply containing fault cause analysis and repair measure suggestions.

[0074] Specifically, the large language model evaluates the usefulness of each answer to answer the user's target question by preset evaluation dimensions (such as relevance to the user's question, accuracy of answer information, completeness of fault diagnosis logic, and pertinence to solving the fault problem), combined with instruction fine-tuning to let the model clearly define the judgment criteria for each dimension, or by inputting examples with usefulness annotations for the model to imitate through few-shot learning. Subsequently, the model judges whether each answer meets the core needs of the user's target question (such as identifying fault causes and obtaining effective repair measures), whether it contains key information, and whether it has errors or redundant content. Based on the above judgments, the answer is quantitatively scored or qualitatively rated in each evaluation dimension, and the usefulness of each answer is finally synthesized. "Scoring all the fault diagnosis answers according to the usefulness" is performed by the large language model, specifically, the large language model takes the user input fault question as the target benchmark, and from the relevance of the fault diagnosis answer to the target question (i.e. whether the answer is closely related to the core demands of the fault), the completeness of the fault cause analysis and repair measure suggestions contained in the answer (i.e. whether it covers key diagnostic dimensions), the accuracy of the information (i.e. whether it is consistent with the entity and relationship information in the fault knowledge graph), and the clarity of the expression, etc. dimensions, the large language model comprehensively considers and quantitatively scores each answer. Integrating key information to generate a comprehensive fault diagnosis reply is achieved through the following specific means: From the fault diagnosis answers greater than the high score threshold selected from the context window, accurately extract core fault cause related information (such as key factors causing the fault, and the correlation logic between factors) and repair measure related content (such as specific operation steps, priority or applicable conditions of measures, etc.), remove duplicate or redundant content, and then systematically organize the extracted information according to the logical structure of "fault cause analysis-repair measure suggestions", while supplementing the connection between information to ensure the coherence of the content, finally forming a comprehensive fault diagnosis reply covering key fault cause analysis and feasible solutions.

[0075] Specifically, the "high score threshold" is a score threshold for screening high-quality answers in the process of evaluating fault diagnosis answers, that is, after the large language model scores the fault diagnosis answers according to the usefulness, the answers higher than the threshold are identified as high-score answers with higher reference value, which can be used in the subsequent context window for integrated generation of fault diagnosis replies. The "high score threshold" is a critical score value that can effectively distinguish high-quality and low-quality answers, which is determined by analyzing the historical score distribution of preset answers in the training data set, counting the minimum score of high-quality answers in past effective fault diagnosis cases, or adjusting different thresholds through multiple rounds of tests and verifying their impact on reply quality.

[0076] It should be noted that: In the specification provided herein, a large number of specific details are explained. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0077] In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments but not others, the combination of features of different embodiments means within the scope of the present application and forms different embodiments.

[0078] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.< / graphtoken> < / graphtoken>

Claims

1. A method for constructing a fault diagnosis model based on knowledge graph subgraph embedding, characterized in that, The method comprises the following steps: Collecting unstructured text data, and performing cleaning and error correction on the unstructured text data to obtain an initial corpus; Determining entity types and defining relationships between entities to complete the construction of a fault knowledge graph ontology, performing named entity recognition based on a preset method, extracting entities corresponding to the entity types from the initial corpus using a large language model, merging entities with consistent semantics in the extracted entities to obtain semi-structured data, extracting relationships between the entities in the semi-structured data to obtain triples, and storing the triples to obtain a fault knowledge graph data layer; Extracting fault objects from the fault knowledge graph data layer, generating a subgraph with the fault objects as the center nodes, converting the subgraph into a training data set composed of natural language descriptions, constructing a fault dependency encoder, and converting the subgraph using the fault dependency encoder to obtain graph embedding tokens; Synchronously training the fault dependency encoder and the large language model using the training data set to obtain a fault diagnosis model, extracting fault phenomena from user input fault questions, matching fault phenomenon entities from the fault knowledge graph data layer using the fault phenomena, generating a subgraph with the fault phenomenon entities as the center, inputting the subgraph into the fault dependency encoder to obtain the graph embedding tokens, inputting the graph embedding tokens into the fault diagnosis model to obtain fault diagnosis answers, evaluating the fault diagnosis answers, and summarizing to generate fault diagnosis replies. 2.The method of claim 1, wherein, When performing cleaning and error correction on the unstructured text data, the method comprises the following steps: Removing duplicate text, meaningless characters, and text content unrelated to fault diagnosis from the unstructured text data; Identifying misspelled words, syntax errors, and ambiguous content in the unstructured text data, and correcting the identified error content with reference to general terms and standard expressions in the fault diagnosis field; The initial corpus includes fault case summaries, fault phenomena, fault analysis, and treatment information.

3. The fault diagnosis model construction method based on knowledge graph subgraph embedding according to claim 2, wherein: The entity types include fault objects, fault causes, fault phenomena, and maintenance measures; The relationships between entities include the occurrence relationship between fault objects and fault phenomena, the caused relationship between fault phenomena and fault causes, the phenomenon-maintenance measure relationship between fault phenomena and maintenance measures, and the cause-maintenance measure relationship between fault causes and maintenance measures; When performing named entity recognition based on a preset method and extracting entities corresponding to the entity types from the initial corpus using a large language model, the method comprises the following steps: The preset method is a json-like prompt; The json-like prompt includes four fields: instruction, schema, example, and input; A preset number of examples are provided for the large language model, and the large language model extracts fault objects, fault causes, fault phenomena, and maintenance measures corresponding to the entity types from the unstructured text of the initial corpus. When the entities with consistent semantics in the extracted entities are merged to obtain semi-structured data, the method comprises: calculating the edit distance between different entities, screening entities with an edit distance less than a preset distance threshold, comparing the semantic similarity of the screened entities, retaining entities with a semantic similarity greater than a preset similarity threshold, and finally manually screening and confirming entities with consistent semantics and merging them to obtain semi-structured data.

4. The method of claim 3, wherein the method further comprises: When extracting the relationships between the entities in the semi-structured data to obtain triples and storing the triples to obtain a fault knowledge graph data layer, the method comprises: According to the relationships between the entities defined in the constructed fault knowledge graph ontology, the corresponding relationships between the fault objects and the fault phenomena, the fault phenomena and the fault causes, the fault phenomena and the maintenance measures, and the fault causes and the maintenance measures are identified from the semi-structured data, and the triples are formed in the format of head entity-relation-tail entity, each triple contains a head entity, a relationship between entities, and a tail entity, the head entity and the tail entity are entities in the semi-structured data, and the relationship is the relationship between entities defined when the fault knowledge graph ontology is constructed; selecting a graph database as a storage carrier, importing the generated triples into the graph database one by one, establishing an association index of entities and relationships in the graph database, and the fault knowledge graph data layer contains all the triples and entity association information.

5. The method of claim 4, wherein the method further comprises: When extracting fault objects from the fault knowledge graph data layer and generating subgraphs with the fault objects as center nodes and converting the subgraphs into a training data set composed of natural language descriptions, the method comprises: traversing all entities in the fault knowledge graph data layer and screening entities with the entity type of fault objects to obtain a list of all fault objects; traversing each fault object node in the list of fault objects, retrieving each fault object node and obtaining all adjacent nodes within a preset subgraph depth jump range of the fault object node, the relationships between the fault object node and the adjacent nodes, and the relationships between the adjacent nodes; when the number of adjacent nodes obtained by retrieval is greater than a preset subgraph node number, multiple split retrievals are performed to group and combine adjacent nodes, each combination contains the fault object node and part of the adjacent nodes, and multiple subgraphs are generated, each subgraph contains a preset subgraph node number of nodes, and all subgraphs completely cover all adjacent nodes; when the number of adjacent nodes obtained by retrieval is less than or equal to the preset subgraph node number, one combination retrieval is performed to combine the fault object node and all adjacent nodes to generate a subgraph with a node number less than or equal to the preset subgraph node number; the natural language description of each subgraph includes fault background task description, fault diagnosis related question, subgraph node information, subgraph node connection relationship, and preset answer; all natural language descriptions of the subgraphs are sorted and summarized to form the training data set.

6. The method of claim 5, wherein the method further comprises: When a fault dependency encoder is constructed and the graph embedding token is converted from the subgraph by using the fault dependency encoder, the method comprises: The fault dependency encoder is composed of a BERT component, a multi-layer neural component and a domain projection component; The BERT component is used to process the text information of each node in the subgraph, and convert the node text of different lengths into node features of consistent length; The relationship between nodes in the subgraph is represented by an adjacency matrix, and the element value in the adjacency matrix is determined according to whether there is an association relationship between nodes. If there is an association relationship, the element value is 1, and if there is no association relationship, the element value is 0; All the node features are integrated to form a feature matrix, the number of rows of the feature matrix is the number of nodes in the subgraph, and the number of columns is the dimension of the node feature; The feature matrix and the adjacency matrix are input into the multi-layer neural component, which learns the node information and the relationship between nodes through multiple rounds, and outputs the overall feature of the subgraph; The overall feature is input into the domain projection component, which converts the overall feature into a graph embedding token aligned with the text feature of the large language model.

7. The method of claim 6, wherein the method further comprises: When the training data set is used to synchronously train the fault dependency encoder and the large language model to obtain a fault diagnosis model, the following steps are included: The parameters of the BERT component in the fault dependency encoder are fixed and not adjusted. The basic parameters of the large language model are fixed and not adjusted. Trainable Lora units are added to the Q and K matrices of each layer of the multi-head self-attention of the large language model. The multi-layer neural component, the domain projection component and the Lora unit of the fault dependency encoder are used as trainable parameter objects; The trainable parameter objects are synchronously trained using an interleaved training method. In a single interleaved training process, the fault dependency encoder is trained a predetermined number of times using the training data set, and the large language model is trained a predetermined number of times using the training data set. The interleaved training process is repeated a predetermined number of times. In each round of interleaved training, samples are selected from the training data set. The fault background task description, fault diagnosis related question and corresponding graph embedding token in the sample are input into the fault dependency encoder and the large language model in training to obtain model output results. The model output results are compared with the predetermined answers in the sample, and the model error is calculated by a cross-entropy loss function. The model error is fed back to the trainable parameter objects by a back propagation algorithm, and the weight parameters and bias parameters of the multi-layer neural component, the projection matrix parameters of the domain projection component and the low-rank matrix parameters of the Lora unit are adjusted based on the model error. When the model error is greater than a predetermined error threshold, the number of rounds of interleaved training is increased, and the parameter adjustment step is repeated until the model error is less than or equal to the predetermined error threshold. When the model error is less than or equal to the predetermined error threshold, the interleaved training is stopped, and the fault diagnosis model is obtained. 8.The method of claim 7, wherein, When the fault phenomenon is extracted from the user input fault question, and the fault phenomenon entity is matched from the fault knowledge graph data layer, the following steps are included: receiving a user inputted fault-related question, inputting the fault-related question into a large language model, the large language model completing named entity recognition based on In-Context Learning, extracting text content related to a fault phenomenon from the fault-related question as a fault phenomenon description; using a method of mixing BM25 and semantic similarity, comparing the fault phenomenon description with fault phenomenon entities in the fault knowledge graph data layer; calculating the matching degree of the fault phenomenon description and each fault phenomenon entity, the matching degree being a weighted average of the similarity calculated by the BM25 method and the semantic similarity; screening the fault phenomenon entity with the highest matching degree as the target fault phenomenon entity. 9.The method of claim 8, wherein, generating a subgraph centered on the fault phenomenon entity, inputting the subgraph into the fault dependency encoder to obtain the graph embedding token, and then inputting the graph embedding token into the fault diagnosis model to obtain the fault diagnosis answer, including: based on the preset subgraph depth and the preset subgraph node number, collecting the subgraph from the fault knowledge graph data layer with the target fault phenomenon entity as the center node, the collection process being consistent with the way of extracting a fault object from the fault knowledge graph data layer and generating a subgraph with the fault object as the center node; inputting the collected subgraph into the trained fault dependency encoder, processing by the BERT component, multi-graph layer neural component and domain projection component, and outputting the corresponding graph embedding token; inputting the graph embedding token, user inputted fault question and fault background task description into the trained fault diagnosis model, the fault diagnosis model analyzing and processing the input information, and outputting multiple fault diagnosis answers. 10.The method of claim 9, wherein, when evaluating the fault diagnosis answers and summarizing the fault diagnosis reply, including: inputting all the fault diagnosis answers into a large language model, the large language model evaluating the usefulness of each answer to the user's target question, and scoring all the fault diagnosis answers according to the usefulness; arranging the answers in order from high to low according to the scores to form an ordered answer list; adding the fault diagnosis answers in the ordered answer list that are greater than the high score threshold in order from high to low to a new context window; when the content in the added context window reaches the preset token limit, stop adding; when the content in the added context window does not reach the preset token limit, continue adding until the content in the context window reaches the preset token limit; based on the fault diagnosis answers in the context window that are greater than the high score threshold, integrating key information to generate a comprehensive fault diagnosis reply, the fault diagnosis reply including fault cause analysis and maintenance measure suggestion.

Images