Electromechanical equipment knowledge graph construction method based on large language model

Abstract

An electromechanical equipment knowledge graph construction method based on a large language model belongs to the field of electromechanical equipment, constructs an electromechanical equipment knowledge graph in combination with the large language model, and performs data acquisition, unstructured data preprocessing, fine tuning data set construction and triple extraction cue word design. Using the fine tuning data set to respectively construct entity extraction and relationship extraction large language models; entities and relationships are automatically extracted from a large amount of unstructured text data; analyzing and screening the structured data to obtain a corresponding entity relationship; performing information fusion according to triple information extracted from the unstructured data and the structured data; then, constructing an electromechanical equipment knowledge graph, and storing data in a graph database; and embedding representation of electromechanical equipment text data is realized in combination with a large language model, and a heterogeneous electromechanical equipment knowledge graph is constructed. According to the method, the representation capability of the atlas is enhanced, and the adaptability of the atlas to downstream tasks is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electromechanical equipment, and particularly relates to a method for constructing an electromechanical equipment knowledge graph based on a large language model. BACKGROUND

[0002] With the continuous development of industrial informatization, the data generated by electromechanical equipment is also continuously accumulated, including sensor data, maintenance records, fault manuals and other multi-source heterogeneous information. How to efficiently utilize such data has become a problem to be solved. Knowledge graph was proposed by Google in 2012. Knowledge graph stores facts in the real world in the form of triples, forming a structured semantic knowledge base. At present, this technology has been widely applied in intelligent question answering, fault diagnosis, fault prediction and other applications.

[0003] Constructing an electromechanical equipment knowledge graph can systematically present the explicit correlation between different data. Combined with deep learning technology, it can further help to mine the implicit relationship between data, thereby improving the intelligent level of equipment fault diagnosis and fault prediction. For example, Xia et al. constructed a device maintenance knowledge graph by defining a domain ontology, and combined with entity alignment and other technologies for knowledge fusion; Liu et al. combined ontology engineering to construct a 3D printing knowledge graph, and used link prediction technology to complete the knowledge graph, thereby assisting in performing fault analysis tasks. However, the above methods all rely on defining fixed triple paradigm to realize information extraction, and with the continuous development of industrial scene informatization and intelligentization, the complexity of data is continuously improved, and it is difficult to manually count complete triple paradigm.

[0004] In recent years, the development of large language models provides a new paradigm for the construction of electromechanical equipment knowledge graph. Large language models have strong semantic understanding and generation capabilities, which can effectively analyze the complex semantic relationship in device data and generate high-quality knowledge representation. Such models have acquired rich knowledge through massive training data and have shown excellent context reasoning and zero-shot learning capabilities, which can adapt to the use of industrial scenes. For example, Liao et al. combined large language models to process maintenance corpus with complex nested relationships, and constructed a fine-grained fault diagnosis knowledge graph; Cheng et al. used GPT-3.5 for triple extraction, and used PiVe technology to adjust the prompt words based on the extraction results. However, common large language models are usually general large language models, and their training data mainly comes from encyclopedic texts, resulting in deficiencies in understanding professional knowledge and terminology in specific fields. When directly applied to entity and relationship extraction in professional fields, such models cannot achieve good results and may also make professional error judgments. This limitation is mainly due to the lack of targeted domain knowledge training of the model.

[0005] Therefore, the construction of knowledge graphs in the field of electromechanical equipment still faces some challenges: (1) Multi-source heterogeneous data in industrial scenarios have complex interrelationships, and it is difficult to manually define a complete triple paradigm. (2) Professional vocabulary in industrial scenarios is too obscure, and some manually recorded data is colloquial. Traditional models and general large language models cannot deeply mine the semantic information and understand professional terms, resulting in incomplete knowledge graphs. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this paper proposes a method for constructing a knowledge graph for electromechanical equipment based on a large language model. This method first proposes a triple extraction method that fine-tunes the large language model in a chained manner. Through a phased prompt engineering design for entity extraction and relationship extraction, it achieves automatic triple extraction. Based on this, a knowledge graph for electromechanical equipment is constructed to integrate multi-source heterogeneous data. Secondly, the large language model is used to encode text information, constructing a heterogeneous knowledge graph composed of the electromechanical equipment knowledge graph and text information. This further enhances the graph's representational capabilities and improves its adaptability to downstream tasks.

[0007] The present invention provides the following technical solutions: A method for constructing a knowledge graph of electromechanical equipment based on a large language model, the method comprising the following steps: The first step is to build a knowledge graph for electromechanical equipment using a large language model. The process is as follows: Step (1.1) Data collection; Step (1.2) unstructured data preprocessing; Step (1.3) fine-tuning dataset construction; Step (1.4) triple extraction prompt word design; In step (1.5), the fine-tuning dataset is used to fine-tune the large language model based on the LoRA (Low-Rank Adaptation) fine-tuning technology and combined with the prompt words to build the entity extraction and relation extraction large language models respectively; Step (1.6) performs entity and relationship extraction based on the fine-tuned large language model, automatically extracting entities and relationships from large amounts of unstructured text data; Step (1.7) For structured data, its entity relationships are clear. After analysis and screening, the corresponding entity relationships are obtained and directly used to construct the electromechanical equipment knowledge graph; Step (1.8) performs triple information fusion based on the triple information extracted from unstructured data and structured data; then constructs the electromechanical equipment knowledge graph ,in and Represent entity sets and relationship sets respectively, and store data in graph databases; Second, the embedding representation of the mechanical and electrical equipment text data is realized by combining a large language model, and a heterogeneous mechanical and electrical equipment knowledge graph is constructed, as follows: Step (2.1) construction of a text encoder based on a large language model; Step (2.2) construction of a heterogeneous mechanical and electrical equipment knowledge graph .

[0008] Further, the process of step (1.1) is as follows: Step (1.1.1) collect unstructured text data related to mechanical and electrical equipment, including maintenance logs, fault cases, technical manuals, monitoring and statistical data of Internet of Things systems, and store the collected data for processing; Use the crawler technology to obtain the encyclopedia data related to the mechanical and electrical equipment from the Internet, including the composition structure and use introduction of the mechanical and electrical equipment; Among them, the crawler technology (Web Crawling), also known as web spider or web robot, is an automatic program that automatically crawls information on the Internet; Step (1.1.2) collect structured data in the mechanical and electrical equipment Internet of Things system, including data stored in JSON format, tables stored in excel files, and store them in a relational database.

[0009] Further, in step (1.2), for unstructured text data, excessively long text data is truncated to adapt to the input length limit of the large language model: ; Among them, represents the maximum input length that the large language model can accept, represents the collected unstructured text data, represents the text data obtained after truncation.

[0010] Further, in step (1.3), the fine-tuning data set construction method is as follows: Step (1.3.1) construct an entity extraction fine-tuning data set, in which the entities in the original text of the mechanical and electrical equipment and the types of the entities are labeled, and the entity types include subsystem, component, fault category, fault phenomenon, fault reason, maintenance type, maintenance project and repair countermeasures.

[0011] Step (1.3.2) construct a relationship extraction fine-tuning data set, in which, based on the entities labeled in step (1.3.1), further label the complete triples in the original text, but the entity type is not labeled again.

[0012] Preferably, the prompt in step (1.4) contains entity extraction prompts and relation extraction prompts, and is designed as follows: Entity extraction prompts : You are an expert in the field of entity recognition related to mechanical and electrical equipment. You need to refer to the following entity types [subsystem, component, fault category, fault phenomenon, fault reason, maintenance type, maintenance project, repair countermeasure], and extract relevant industrial entities from <text_input>, output format <output_format>. When no entity is found, output "No entity found"; Each line of the output must be in JSON format; Relation extraction prompts : You are an expert in the field of relation extraction related to mechanical and electrical equipment. You need to extract the relationship between entities from <text_input>, output format <output_format>. When there is no relationship, output "No triple found"; Each line of the output must be in JSON format; Wherein, <text_input> is the input original text, i.e. ; <output_format> is the set input format. For entity extraction, the model is required to output entities and entity types; For relation extraction, the <output_format> requires the model to directly output complete triples.

[0013] In step (1.5), LoRA is a parameter effective large language model fine-tuning method, which uses multiple low-rank matrices to adjust the weights of large language models, which can significantly reduce the trainable parameter size in the fine-tuning stage, and maintain the original generalization ability of the model. Through the method of LoRA fine-tuning, the adaptability of large language models in specific domain tasks can be enhanced in low resource conditions, thereby significantly improving their performance in the process of mechanical and electrical equipment knowledge processing, and helping to build a more expressive knowledge graph.

[0014] The process of step (1.5) is as follows: Step (1.5.1) based on LoRA fine-tuning, train entity extraction large language model using entity extraction fine-tuning dataset, reduce the parameter size of training by decomposing the pre-training weight matrix of large language model into two low-rank matrices, while improving the inference ability of the model in specific domain: ; Wherein, represents the weight matrix of the fine-tuned entity extraction large language model, and represent the dimensions of the weight matrix; represents the original pre-training weight matrix of the large language model; denotes the updated weight matrix; , denotes two low-rank matrices, is the rank of a matrix.

[0015] Step (1.5.2) again based on LoRA fine-tuning, using the relationship extraction fine-tuning dataset to train the relationship extraction large language model: ; wherein, denotes the weight matrix of the fine-tuned relationship extraction large language model; denotes the updated weight matrix; , denotes two low-rank matrices obtained by performing low-rank decomposition on ; Step (1.5.3) based on step (1.5.1) and step (1.5.2) respectively using two fine-tuning training sets, fine-tuning training the large language model to obtain the entity extraction large language model and the relationship extraction large language model , used for subsequent unstructured text triple extraction; wherein the pre-training weights of the large language model are all After fine-tuning, and learn different weight matrices and adapted to their respective specific tasks, thereby improving the reasoning ability of the model in the specific task.

[0016] The process of the step (1.6) is as follows: Step (1.6.1) first performs entity extraction, for a given input text, combines it with the pre-defined , and inputs the model to obtain the entity extraction result: ; wherein, denotes the input text, is the entity result of the output text; Step (1.6.2) according to the entity extraction result, performs relationship extraction. The input text, the pre-defined , the entity extraction result are combined and encoded, and the model is input to obtain the final output triple result: ; wherein, represents the triples obtained by reasoning on the text of the first clause.

[0017] The process of step (1.8) is as follows: Step (1.8.1) traverses all the triples information by row, first removes the white space at the end of each row of data, and then splits according to the white space in the data to obtain data stored in a list line, wherein line[0] and line[2] represent the head entity and tail entity in the triples, respectively, and line[1] represents the relationship in the triples; Step (1.8.2) initializes a dictionary named lstEntsRels, which contains key and value fields and is a map data structure. The key value is the head entity name, and the value value is a list vlist, which stores each element as a pair of relationship + tail entity, where + is the separator; Step (1.8.3) reads each row in line in turn, judges whether line[0] already exists in the key of lstEntsRels, if not, stores line[0] as the key of lstEntsRels, connects line[1] and line[2] of the corresponding row by + separator, and then stores it in the vlist of value as an element. If the key exists, read the value corresponding to the key, connect line[1] and line[2] of the corresponding row by + separator, traverse all values in vlist, judge whether the value of current line line[1]+line[2] exists, if all exist, read the next line in line and repeat step (1.8.3) operation, otherwise insert the key and value of the dictionary; Step (1.8.4) reads all the rows in the triples information, and lstEntsRels has fused all the triples in the unstructured data and structured data, which is imported into the graph database for storage.

[0018] The process of step (2.1) is as follows: Step (2.1.1) inputs the unstructured text into the large language model, and extracts the hidden state output of the last layer of the model: ; wherein, is the encoder of the large language model, is the hidden state output of the last layer of the large language model about the text inside the large language model; Step (2.1.2) uses different methods to process the last hidden state of the large language model as the output of the text encoder: 1) For the BERT (Bidirectional Encoder Representations from Transformers) model, the [CLS] token output by the last layer is selected as the text embedding representation. The [CLS] token is the first sequence of hidden state outputs and is typically used as the overall representation of the sequence. The BERT model is a commonly used pre-trained language model. 2) Select the last non-filled token in the last layer output sequence to be embedded as the text embedding representation. This step is represented by the formula express; 3) Calculate the average value of all embedding representations of the last hidden state as the final text embedding representation. This step is expressed as Indicates that Express All embedded representations of are averaged; In step (2.1.3), the output of the text encoder is input into the fully connected layer and the activation function layer to perform feature dimension conversion: ; in, is a linear transfer function, is the activation function, The output Embedded representation of a text item; in, and They represent text representation space and knowledge graph representation space respectively. Fully connected layers and activation functions are commonly used techniques in deep learning.

[0019] The process of step (2.2) is as follows: Step (2.2.1) Electromechanical equipment knowledge graph All entities and relationships in are represented as nodes in the heterogeneous knowledge graph. At this time, the node set in the heterogeneous knowledge graph is ; all Also used as a node in the heterogeneous knowledge graph, Represents a set of text information nodes; Step (2.2.2) The edges in are defined as follows: In the initial HKG, Each text information node in All nodes in the are connected, at this time, The adjacency matrix A is expressed as: ; wherein, represents the number of, represents the number of text information nodes. and respectively represent that there is no link between the internal nodes of the two node sets. and then represent each text information node in is connected to all nodes in

[0020] An electromechanical equipment knowledge graph construction and management system based on a large language model, comprising: a prompt word design module, which adjusts the pre-defined prompt words according to the specific scene of the actual application; a large language model fine-tuning module, which is used to select a large language model and fine-tune training based on the constructed data set and prompt words; a triple extraction module, which is used to extract triples from new electromechanical equipment text data, and the obtained triple results are queried according to the operation of step (1.8.3) whether they already exist in the database, and if not, the triples are stored; a knowledge graph display module, which is used to display the constructed electromechanical equipment knowledge graph, and can specifically query a specific node and its related neighbor nodes, and has the function of modifying node information.

[0021] The beneficial effects of the present application mainly manifest in: 1. For unstructured data of electromechanical equipment, a method based on fine-tuning of a large language model is used for triple extraction, and the method of prompt engineering and chain extraction is combined to further improve the quality of triple extraction, thereby providing data support for the construction of electromechanical equipment knowledge graph.

[0022] 2. Combined with a large language model encoder, the powerful generation capability of the large language model is used to perform semantic coding on text data, providing a semantic embedding basis for cross-modal interaction of heterogeneous data.

[0023] 3. An electromechanical equipment heterogeneous knowledge graph is constructed by fusing electromechanical equipment triple information and heterogeneous text embedding information, and by jointly modeling structured relationships and unstructured semantic features, a unified representation framework is provided for realizing feature interaction between text embedding and knowledge graph embedding. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is the electromechanical equipment knowledge graph construction process.

[0025] Figure 2It is an electromechanical equipment information extraction large language model training and inference process.

[0026] Figure 3 It is a structure diagram of a large language model-based electromechanical equipment knowledge graph construction and management system. DETAILED DESCRIPTION

[0027] The application will be further described below with reference to the accompanying drawings.

[0028] Reference Figures 1-3 A large language model-based electromechanical equipment knowledge graph construction method, the method comprising the following steps: Step 1: Constructing an electromechanical equipment knowledge graph based on a large language model, the process being as follows: Step (1.1) Data collection, the process being as follows: Step (1.1.1) Collecting unstructured text data related to electromechanical equipment, including maintenance logs, fault cases, technical manuals, monitoring and statistical data of Internet of Things systems, and storing the collected data for processing; Using a crawler technology to obtain encyclopedia data related to electromechanical equipment from the Internet, including the composition structure and use introduction of the electromechanical equipment; Among them, the crawler technology (Web Crawling) is also called web spider or web robot, which is an automatic program that automatically captures information on the Internet; Step (1.1.2) Collecting structured data in the electromechanical equipment Internet of Things system, including data stored in JSON format and tables stored in excel files, and storing them in a relational database.

[0029] Step (1.2) Unstructured data preprocessing: for unstructured text data, long text data is truncated to adapt to the input length limit of the large language model: ; Among them, represents the maximum input length that the large language model can accept, represents the collected unstructured text data, represents the text data obtained after truncation.

[0030] Step (1.3) Fine-tuning dataset construction, the process being as follows: Step (1.3.1) Constructing an entity extraction fine-tuning dataset, in which dataset, entities in the original text of electromechanical equipment and the types of entities are labeled, and the entity types include subsystems, components, fault categories, fault phenomena, fault reasons, maintenance types, maintenance projects, and maintenance countermeasures; Step (1.3.2) constructs a relation extraction fine-tuning dataset, in which, based on the entities labeled in step (1.3.1), the complete triples in the original text are further labeled, but the entity types are not repeated; Step (1.4) triple extraction prompt design; The prompt of step (1.4) contains entity extraction prompts and relation extraction prompts, and is designed as follows: Entity extraction prompts : You are an expert in the field of mechanical and electrical equipment-related entity recognition. You need to refer to the following entity types [subsystem, component, fault category, fault phenomenon, fault reason, maintenance type, maintenance project, repair countermeasure], and extract relevant industrial entities from <text_input>, output format <output_format>, when no entity is found, output "no entity found"; Each line of output must be in JSON format.

[0031] Relation extraction prompts : You are an expert in the field of mechanical and electrical equipment-related relation extraction. You need to extract the relationship between entities from <text_input>, output format <output_format>, when there is no relationship, output "no triple found"; Each line of output must be in JSON format; Wherein, <text_input> is the input original text, i.e. ; <output_format> is the set input format. For entity extraction, the model is required to output entities and entity types; and for relation extraction, the <output_format> requires the model to directly output complete triples.

[0032] Step (1.5) uses the fine-tuning dataset to fine-tune the large language model based on LoRA (Low-Rank Adaptation) fine-tuning technology and combined with prompts, and constructs entity extraction and relation extraction large language models respectively; The LoRA in step (1.5) is a parameter-effective large language model fine-tuning method, which uses multiple low-rank matrices to adjust the weights of the large language model, which can significantly reduce the size of trainable parameters in the fine-tuning stage, while maintaining the original generalization ability of the model. Through the method of LoRA fine-tuning, the adaptability of the large language model in specific domain tasks can be enhanced in the case of low resources, thereby significantly improving its performance in the process of mechanical and electrical equipment knowledge processing, and helping to build a more expressive knowledge graph.

[0033] The process of step (1.5) is: Step (1.5.1) trains the entity extraction large language model using the entity extraction fine-tuning dataset based on LoRA fine-tuning, reduces the parameter scale of training by decomposing the pre-training weight matrix of the large language model into two low-rank matrices, and improves the inference ability of the model in a specific field: ; wherein, represents the weight matrix of the fine-tuned entity extraction large language model, and represent the dimensions of the weight matrix; represents the original pre-training weight matrix of the large language model; represents the updated weight matrix; , represents two low-rank matrices, is the rank of the matrix; Step (1.5.2) trains the relation extraction large language model using the relation extraction fine-tuning dataset based on LoRA fine-tuning again: ; wherein, represents the weight matrix of the fine-tuned relation extraction large language model; represents the updated weight matrix; , represents two low-rank matrices obtained by low-rank decomposition on ; Step (1.5.3) fine-tunes the large language model using two fine-tuning training sets respectively based on steps (1.5.1) and (1.5.2) to obtain an entity extraction large language model and a relation extraction large language model , which are used for subsequent triple extraction of unstructured text; wherein, the pre-training weights of the large language model are all After fine-tuning, and learn different weight matrices and adapted to their respective specific tasks, respectively.

[0034] Step (1.6) automatically extracts entities and relations from a large amount of unstructured text data based on the fine-tuned large language model, as shown in Figure 1 ; The process of the step (1.6) is as follows: Step (1.6.1) first performs entity extraction. For a given input text, combine it with the pre-defined , and then input the model , as shown inFigure 1 As shown in the NER input part, get the entity extraction results: ; in, Indicates the Input text, for Output Entity results for the text; Step (1.6.2) is to extract relations based on the entity extraction results. , entity extraction results Perform combined encoding and input model ,like Figure 1 As shown in the RE input part, the final output triple result is obtained: ; in, Represents the final triplet.

[0035] Step (1.7) is as follows Figure 1 As shown in the figure, for structured data, the entity relationships are clear, and the corresponding entity relationships are obtained after analysis and screening, which are directly used to construct the knowledge graph of electromechanical equipment; Step (1.8) performs triple information fusion based on the triple information extracted from unstructured data and structured data; then constructs the electromechanical equipment knowledge graph ,in and Represent entity sets and relationship sets respectively, and store the data in a graph database; the process is: Step (1.8.1) traverses all triples by line. First, remove the whitespace characters at the end of each line of data, then split the data according to the whitespace characters in the data. The resulting data is stored in a list called line, where line[0] and line[2] represent the head and tail entities in the triple, respectively, and line[1] represents the relationship in the triple. Step (1.8.2) initializes a dictionary named lstEntsRels. The dictionary contains key and value fields and is a map data structure. The key value is the name of the head entity, and the value is a list vlist. Each element in the dictionary is a pair of relation + tail entity, where + is a separator. Step (1.8.3) reads each line in line in turn, judges whether line[0] already exists in the key of lstEntsRels, if not, stores the line[0] as the key of lstEntsRels, connects line[1] and line[2] of the corresponding line by the + separator, and then stores in the vlist of value as an element existing in the list. If the key exists, read the value corresponding to the key, connect line[1] and line[2] of the corresponding line by the + separator, traverse all values in the vlist, judge whether the value of the current line line[1]+line[2] exists, if all exist, re-read the next line in line to perform the operation of step (1.8.3), otherwise, perform the key and value insertion operation of the dictionary; Step (1.8.4) after reading all the lines in the triple information, lstEntsRels has fused all the triples in the unstructured data and structured data, and stores them in the graph database.

[0036] Second step: realize the embedding representation of the mechanical and electrical equipment text data combined with the large language model, and construct the heterogeneous mechanical and electrical equipment knowledge graph, the process is as follows: Step (2.1) based on the text encoder of the large language model; The process of step (2.1) is: Step (2.1.1) input the unstructured text into the large language model, and extract the hidden state output of the last layer of the model: ; Wherein, is the large language model encoder, is the hidden state output of the last layer of the large language model about the text ; Step (2.1.2) processes the last layer hidden state of the large language model using the following processing flow as the output of the text encoder: 1) For the BERT (Bidirectional Encoder Representations from Transformers) model, the [CLS] marker output by the last layer is selected as the text embedding representation, the [CLS] marker is the first sequence of the hidden state output, which is usually used as the overall representation of the sequence, and the BERT model is a commonly used pre-training language model; 2) The last non-padding marker embedding in the output sequence of the last layer is selected as the text embedding representation, which is represented by the formula ; 3) Calculate the average of all embedding representations of the last hidden state as the final text embedding representation, which is denoted by formula , where represents the mean value calculation of all embedding representations of ; Step (2.1.3), input the output result of the text encoder into the full connection layer and the activation function layer to convert the feature dimension: ; wherein, is a linear conversion function, is an activation function, is the embedding representation of the th text output; wherein, and represent the text representation space and the knowledge graph representation space respectively, and the full connection layer and the activation function are common techniques in deep learning.

[0037] Step (2.2) Construction of Heterogeneous Knowledge Graph of Mechanical and Electrical Equipment The process is as follows: Step (2.2.1) All entities and relationships in the mechanical and electrical equipment knowledge graph are represented as nodes in the heterogeneous knowledge graph, and the node set of the heterogeneous knowledge graph at this time is ; all are also nodes in the heterogeneous knowledge graph, and the text information node set is denoted by ; Step (2.2.2) The edges in are defined as follows: in the initial HKG, each text information node in is connected to all nodes in ; At this time, the adjacency matrix A of ; wherein, represents the number of , and represents the number of text information nodes. and represent that there is no link between the internal nodes of the two node sets, and represent that each text information node in is connected to all nodes in .

[0038] The application also provides a large language model-based mechanical and electrical equipment knowledge graph construction and management system, as shown in Figure 3 comprises: A prompt word design module adjusts the pre-defined prompt word according to the specific scene of the actual application; A large language model fine-tuning module is used to select a large language model and fine-tune the training based on the constructed data set and the prompt word; A triple extraction module is used to extract triples from new mechanical and electrical equipment text data. The obtained triple results are queried according to the operation of step (1.8.3) to determine whether they already exist in the database. If not, the triples are stored; A knowledge graph display module is used to display the constructed mechanical and electrical equipment knowledge graph, and can specifically query a specific node and its related neighbor nodes, and has the function of modifying node information. For example, a fault node can be queried to find the correlation pattern of the fault, find the cause of the fault and seek a solution and maintenance strategy.

[0039] The large language model-based mechanical and electrical equipment knowledge graph construction method of the embodiment is described as follows: Step one, five entity extraction models are given: BiLSTM-CRF: a bidirectional long short-term memory network is combined with a conditional random field to perform entity extraction; BERT-BiLSTM-CRF: BERT is further combined with BiLSTM-CRF to perform entity extraction; LLM (DeepSeek-7B): the entity extraction method proposed by the application selects DeepSeek-7B as a baseline large language model for fine-tuning; LLM (Llama3-8B): the entity extraction method proposed by the application selects Llama3-8B as a baseline large language model for fine-tuning; LLM (GLM4-9B): the entity extraction method proposed by the application selects GLM4-9B as a baseline large language model for fine-tuning.

[0040] Step two, the experimental data set is the unstructured corpus of mechanical and electrical equipment constructed by the application, which contains a total of 5996 text information, covering maintenance records, fault cases and technical manuals.

[0041] Step three, define evaluation indexes: in order to verify the performance of entity extraction, the following evaluation indexes are used for experiments: Precision: this index is used to reflect the ability of the model to accurately identify effective entities, ; wherein, the number of samples representing entities successfully extracted by the model, the number of samples representing entities that are misidentified as mechanical and electrical equipment entities by the model but do not actually exist; Recall: This index is used to evaluate the ability of the model to capture relevant entities, wherein, the number of samples representing mechanical and electrical equipment entities that are not successfully identified by the model; F1Score: This index provides a single, balanced performance measure, Step four, analysis and comparison of results, as follows: Referring to Table 1, the comparison results of three indexes of the mechanical and electrical equipment entity extraction task are shown, , and the optimal experimental index is highlighted in Table 1;

[0042] As can be seen from Table 1, the method for entity extraction based on fine-tuning of large language models proposed by the present application has achieved excellent performance, and the effect is better than that of the traditional model, which is about 4%-6% higher than that of the traditional model. Among the three selected large language models, GLM4-9B achieves the most ideal effect, Llama3-8B and DeepSeek-7B have the second best performance, which is 1.33% and 1.07% lower than GLM4-9B, respectively. The performance difference between these three large language models may be caused by the different parameter quantities of the large language models. Therefore, under the condition of sufficient resources, it is relatively reasonable to select a large language model with more parameters for the triple extraction task.

[0043] The large language model-based mechanical and electrical equipment knowledge graph construction method of the present embodiment is compared as follows: Step 1, large language model text encoder selection, based on the text encoder proposed in step (2.1), different pre-trained language models and large language models are used as text encoders, and link prediction experiments are conducted to evaluate the application value of the HKG constructed by the present application in downstream tasks.

[0044] Among them, the link prediction experiment is a task of predicting the missing triplets by known entities and relationships in the knowledge graph, and the link prediction of the knowledge graph can be further applied to downstream tasks such as fault prediction, fault diagnosis and intelligent question answering.

[0045] ​​The application selects BERT, RoBERTA, GLM4-9B, Flan-t5-large, Llama3-8B, Qwen2.5-7B and DeepSeek-7B as text encoders; wherein, BERT and RoBERTA belong to the category of pre-training language models, and the parameter quantity thereof is significantly different from that of large language models, and the remaining models all belong to the category of large language models.

[0046] Step two, the experimental data set is the mechanical and electrical equipment knowledge graph and the mechanical and electrical equipment corpus constructed by the application, which contains 7673 entity pairs and 5996 unstructured texts.

[0047] Step three, define evaluation indexes, as follows: In order to verify the performance of the large language model text encoder, the following evaluation indexes are adopted, and the link prediction experiment is selected to evaluate the performance of the model: MRR: represents the average reciprocal result of the number of true triples in the test set, and the larger the value is, the better. The formula is as follows: ; Wherein, represents the ranking number of triples, N is the total number of candidate entities.

[0048] Hits@n: represents the proportion of triples with a ranking less than in the link prediction, and the larger the value is, the better, ; Wherein, represents the number of triples satisfying the ranking less than , and the application selects Hits@1, Hits@3 and Hits@10 for evaluation.

[0049] Step four, analyze and compare the results, and table 2 is the link prediction result table under the selection of different large language model encoders,

[0050] Table 2 shows the link prediction results under different large language model encoder selections, wherein the optimal result is marked in bold and the suboptimal result is marked in underline. As can be seen from Table 2, Qwen2.5-7B achieves the optimal result in the four indicators, DeepSeek-7B, GLM4-9B and Llama3-8B are only inferior to Qwen2.5-7B, and the three have advantages and disadvantages among each other, while Flan-t5-large is inferior to the above three models, but the gap is less. Compared with the rest of the large language models, when using BERT and RoBERTA for text initialization embedding, the performance of the model is lower. The reason may be that BERT and the like are released earlier, and they still belong to the category of pre-training language models. The data size used during training cannot be compared with the above large language models. After a large amount of corpus training, the large language model is superior to the pre-training language model in generating text embedding, and therefore better results can be achieved.

[0051] The large language model-based mechanical and electrical equipment knowledge graph construction method of the embodiment has the following implementation process: Step one, fine-tuning of the large language model, as follows: Step (1.1) fine-tuning dataset construction, referring to the training part of Figure 2 For the entity extraction task, the entity and entity type are labeled by the application; and for the relation extraction task, the complete triple is directly labeled by the application; Step (1.2) referring to the entity extraction model comparison result, selecting GLM4-9B as the baseline large language model, and using the LoRA method for fine-tuning training, wherein the rank of LoRA is set to 8, 32 is set. Among them, GLM4 is a large language model developed by Zhipu AI, GLM4-9B represents a version with 9 billion parameters, and the rank of LoRA and are parameters set during fine-tuning.

[0052] Step two, triple extraction, as follows: Step (2.1) input the data in the mechanical and electrical equipment corpus into the entity extraction large language model obtained by fine-tuning to obtain the entity extraction result; Step (2.2) input the mechanical and electrical equipment corpus and the entity extraction result into the relation extraction large language model to obtain the triple extraction result, as shown in the reasoning part of Figure 2 .

[0053] Step three, triple data storage, as follows: Step (3.1) judge whether the triple extraction result is repeated, if not, store it in the graph database; The final obtained mechanical and electrical equipment knowledge graph of step (3.2) contains 7673 entity pairs, 8 entity types and 1756 different relations.

[0054] The embodiments of the present specification are merely a list of implementation forms of the inventive concept, and are only for the purpose of description. The protection scope of the present application should not be regarded as being limited to the specific forms presented in the embodiments, and the protection scope of the present application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A method for constructing a knowledge graph of electromechanical equipment based on a large language model, characterized in that: The method The following steps are involved: The first step is to build a knowledge graph for electromechanical equipment using a large language model. The process is as follows: Step (1.1) Data collection; Step (1.2) unstructured data preprocessing; Step (1.3) fine-tuning dataset construction; Step (1.4) triple extraction prompt word design; In step (1.5), the fine-tuning dataset is used to fine-tune the large language model based on the LoRA fine-tuning technology and combined with the prompt words to build the entity extraction and relation extraction large language models respectively; Step (1.6) performs entity and relationship extraction based on the fine-tuned large language model, automatically extracting entities and relationships from large amounts of unstructured text data; Step (1.7) For structured data, its entity relationships are clear. After analysis and screening, the corresponding entity relationships are obtained and directly used to construct the electromechanical equipment knowledge graph; Step (1.8) performs triple information fusion based on the triple information extracted from unstructured data and structured data; then constructs the electromechanical equipment knowledge graph ,in and Represent entity sets and relationship sets respectively, and store data in graph databases; The second step is to combine the large language model to achieve embedded representation of electromechanical equipment text data and build a heterogeneous electromechanical equipment knowledge graph. The process is as follows: Step (2.1) builds a text encoder based on a large language model; Step (2.2) Heterogeneous knowledge graph of electromechanical equipment Build.

2. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1, characterized in that: The process of step (1.1) is: Step (1.1.1) collects unstructured text data related to electromechanical equipment, including maintenance logs, fault cases, technical manuals, monitoring and statistical data of the Internet of Things system, and stores the collected data for processing; Use crawler technology to obtain encyclopedia data related to electromechanical equipment from the Internet, including the composition structure and usage introduction of the electromechanical equipment; Step (1.1.2) collects structured data from the electromechanical equipment IoT system, including data stored in JSON format and tables stored in Excel files, and stores them in a relational database.

3. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1 or 2, characterized in that: In step (1.2), for unstructured text data, the overly long text data is truncated to fit the input length limit of the large language model: ； in, Indicates the maximum input length that the large language model can accept. Represents the collected unstructured text data, Represents the text data obtained after truncation.

4. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1 or 2, characterized in that: The process of step (1.3) is: Step (1.3.1) constructs an entity extraction fine-tuning dataset, in which entities and entity types in the original text of electromechanical equipment are labeled, where entity types include subsystem, component, fault category, fault phenomenon, fault cause, maintenance type, maintenance item, and maintenance countermeasures; Step (1.3.2) constructs a relation extraction fine-tuning dataset. In this dataset, based on the entities marked in step (1.3.1), the complete triples in the original text are further marked, but the entity types are no longer marked repeatedly.

5. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 3, wherein: The prompt words in step (1.4) include entity extraction prompt words and relationship extraction prompt words, and are designed as follows: Entity extraction prompt words :You are an expert in the field of entity recognition related to electromechanical equipment. You need to refer to the following entity types [subsystem, component, fault category, fault phenomenon, fault cause, maintenance type, maintenance item, maintenance countermeasure] and<text_input> Extract related industrial entities, the output format is<output_format> When no entity is found, the output is "No entity found". Each line of the output must be in JSON format. Relation extraction prompt words :You are an expert in the field of electromechanical equipment relationship extraction.<text_input> Extract the relationship between entities, the output format is<output_format> When there is no relationship, the output is "No triples found". Each line of the output must be in JSON format. in,<text_input> is the original text input, that is ;<output_format> For the input format set, for entity extraction, the model is required to output entities and entity types; for relationship extraction, its<output_format> The model is required to directly output complete triples.

6. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1 or 2, characterized in that: The process of step (1.5) is as follows: Step (1.5.1) Based on LoRA fine-tuning, use the entity extraction fine-tuning dataset to train the entity extraction large language model, and decompose the pre-trained weight matrix of the large language model into two low-rank matrices; Step (1.5.2) is again based on LoRA fine-tuning, using the relation extraction fine-tuning dataset to train the relation extraction large language model; Step (1.5.3) uses two fine-tuning training sets based on steps (1.5.1) and (1.5.2) to fine-tune the large language model to obtain the entity extraction large language model. Large language model with relation extraction , used for subsequent triple extraction of unstructured text; Among them, the pre-training weights of the large language model are , after fine-tuning, and Learn different weight matrices adapted to their respective specific tasks and , thereby improving the model's reasoning ability in specific tasks.

7. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1 or 2, characterized in that: The process of step (1.6) is as follows: Step (1.6.1) first performs entity extraction. For a given input text, compare it with the predefined Combine and then input the model , get entity extraction results ; Step (1.6.2) performs relation extraction based on the entity extraction results, and converts the input text, predefined , entity extraction results Perform combined encoding and input model , get the final output triple result.

8. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1 or 2, characterized in that: The process of step (1.8) is: Step (1.8.1) traverses all triples by line. First, remove the whitespace characters at the end of each line of data, then split the data according to the whitespace characters in the data. The resulting data is stored in a list called line, where line[0] and line[2] represent the head and tail entities in the triple, respectively, and line[1] represents the relationship in the triple. Step (1.8.2) initializes a dictionary named lstEntsRels. The dictionary contains key and value fields and is a map data structure, where the key value is the name of the head entity and the value is a list vlist. Each element stored in the dictionary is a pair of relation + tail entity, where + is a separator. Step (1.8.3) reads each line in line in turn, and determines whether line[0] already exists in the key of lstEntsRels. If not, stores line[0] as the key of lstEntsRels, concatenates line[1] and line[2] of the corresponding line with the + separator, and then stores it in vlist of value as an element of the list. If the key exists, reads the value corresponding to the key, concatenates line[1] and line[2] of the corresponding line with the + separator, traverses all values ​​in vlist, and determines whether the value of line[1]+line[2] in the current line exists. If both exist, reread the next line in line and repeat step (1.8.3). Otherwise, insert the key and value into the dictionary. After reading all the rows of triple information in step (1.8.4), lstEntsRels has merged all the triples in the unstructured data and structured data and imported them into the graph database for storage.

9. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 1 or 2, characterized in that: The process of step (2.1) is as follows: Step (2.1.1) inputs the unstructured text into the large language model and extracts the hidden state output of the last layer of the model; Step (2.1.2) uses the following processing flow to process the last hidden state of the large language model as the output of the text encoder: 1) For the BERT model, the [CLS] token output by the last layer is selected as the text embedding representation. The [CLS] token is the first sequence of the hidden state output and is used as the overall representation of the sequence. 2) Select the last non-filled token in the last layer output sequence to be embedded as the text embedding representation; 3) Calculate the average of all embedded representations of the last hidden state as the final text embedding representation; Step (2.1.3) inputs the output of the text encoder into the fully connected layer and the activation function layer to perform feature dimension conversion to obtain , is the embedding representation of the k-th text.

10. The method for constructing a knowledge graph of electromechanical equipment based on a large language model according to claim 9, characterized in that: The process of step (2.2) is as follows: Step (2.2.1) Electromechanical equipment knowledge graph All entities and relationships in are represented as nodes in the heterogeneous knowledge graph. At this time, the node set in the heterogeneous knowledge graph is ; all Also used as a node in the heterogeneous knowledge graph, Represents a set of text information nodes; Step (2.2.2) The edges in are defined as follows: In the initial HKG, Each text information node in All nodes in are connected; at this time, The adjacency matrix A is expressed as: ； in, express The number of Indicates the number of text information nodes, and Respectively indicate that there are no links between the internal nodes of the two node sets, and It means Each text information node in All nodes in are connected.

Images