Complex semantic relationship extraction method for industrial knowledge graph construction

Abstract

A complex semantic relationship extraction method for industrial knowledge graph construction belongs to the technical field of natural language processing, and comprises the following steps: step 1, constructing a dynamic prompt in combination with an industrial text data context to obtain relationship characterization between industrial entities; 2, enhancing intra-class aggregation and inter-class separation of relation characteristics in the relation storage queue by adopting comparative learning; and 3, dynamically optimizing the training process, and reasoning the relationship between the industrial entities in the industrial knowledge graph by using the converged model. According to the method, semantics of industrial text data contexts and entity type information in an entity labeling set are considered, intra-class aggregation and inter-class separation are carried out on different semantic relationships by utilizing the relationship storage queue, the relationship extraction accuracy is high, and the knowledge graph construction performance is good.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a complex semantic relationship extraction method for constructing an industrial knowledge graph. Background Art

[0002] With the in-depth advancement of informatization and intelligence in the industrial field, knowledge graph technology has been widely used in industrial production, quality management, equipment maintenance and other fields. Industrial knowledge graphs effectively present entities and their relationships in the industrial field in the form of structured data expression. Combined with real-time data analysis and reasoning, they help decision makers quickly identify problems, predict risks, and optimize processes, which is conducive to providing important data support for decision-making on complex issues such as equipment fault diagnosis and supply chain management. However, the complexity and diversity of industrial data and the richness of relational semantic expressions have brought huge challenges to the construction of knowledge graphs, especially in the extraction of complex semantic relationships.

[0003] Traditional rule-based relationship extraction methods are difficult to adapt to the diverse and long-tailed relationships in industrial text data; the data-driven method's demand for large-scale annotated data is in conflict with the scarcity of industrial corpus and the high cost of annotation; deep learning-based methods are sensitive to industrial professional terminology and contextual noise, and it is difficult to fully utilize the semantic information of complex relationship labels. Existing methods are insufficient in extracting complex semantic relationships from small sample industrial text data, and a more efficient and robust solution is urgently needed to support the construction of industrial knowledge graphs. Summary of the invention

[0004] In order to overcome the shortcomings of existing industrial knowledge graph construction, such as scarce data annotation, complex semantic expression of relationships between industrial entities, and the diverse relationship types and semantic ambiguity in industrial domain knowledge, the present invention proposes a complex semantic relationship extraction method for industrial knowledge graph construction, which uses dynamic prompts and implicit structural constraints to obtain the relationship representation between industrial entities, and infers the relationship categories between entities in industrial domain text data by clustering.

[0005] The technical solution adopted by the present invention to solve the technical problem is: A complex semantic relationship extraction method for constructing an industrial knowledge graph, the method comprising the following steps: Step 1: Combine the industrial text data context to construct dynamic prompts to obtain the relationship representation between industrial entities; Step 2: Use contrastive learning to enhance the intra-class aggregation and inter-class separation of relation features in the relation storage queue; Step 3: Dynamically optimize the training process and use the converged model to infer the relationship between industrial entities in the industrial knowledge graph.

[0006] Furthermore, in step 1, the process of constructing dynamic prompts in combination with the industrial text data context to obtain the representation of the relationship between industrial entities includes the following steps: Step 11: Introduce industry-specific vocabulary as entity annotation set ,analyze Equipment name, fault type, process entity type, build entity type collection , using the strong correlation and temporal characteristics of industrial domain knowledge to Enhanced; based on industrial entity sets , using rules to identify the head and tail industrial entities in the given industrial field text data, and construct dynamic prompts ,in , Represents an identified industrial entity, , Is the pseudo type of the corresponding industrial entity, consisting of the collection initialization, The complex semantic relationships that may exist between industrial entities in the context of industrial text data; Step 12: Dynamic prompt Text data from industry merge, The number of samples in each batch is then mapped word by word into a word vector sequence through the embedding layer, with a unified sequence length of , if the length of the mapping sequence is less than , then fill the end with 0; if the sequence length is greater than , then directly truncate the redundant characters at the end and obtain the encoded representation of the input sequence through the encoder ,exist Add absolute position encoding , ; in, Equal to the dimension of the embedding layer; Step 13, select the hidden layer as A deep learning model for industrial fields, select layers 2 to 3 Layer Embedding and building industrial entity relationship matrix , , ; Step 14: Calculate the industrial entity relationship matrix of , ; Step 15: Calculation and contextual representation of text data in industrial fields The similarity between ; The importance of the predicted industrial entity relationship representation of each feature layer of the model compared to the industrial field text data is measured using As The global weighted summation on the final industrial entity relationship representation is generated , .

[0007] Furthermore, in step 2, the process of using contrastive learning to enhance intra-class aggregation and inter-class separation of relational features in the relational storage queue includes the following steps: Step 21: Construct instance-level contrastive loss for a small number of manually annotated and unannotated industrial text datasets. ; Among them, through rules and industrial entity annotation sets Identify business entities , , and the relationship between the model prediction Constitute a positive sample pair ; Select two different arbitrary span segments from the current industrial text data as pseudo-entities , Among them, for a small number of manually annotated industrial text datasets, select the real annotated relationship labels; for unlabeled industrial text datasets, select the central representation of the relationship cluster to which it belongs Constitute a negative sample pair ; Step 22: Based on Compute the probability distribution of possible relationships between industrial entities, ; in, is the classification weight, is the bias parameter; Step 23: Calculate the relationship category of the industrial entity in the industrial data context: ; Step 24: Construct a classification cross entropy loss for annotating industrial text datasets. ; Step 25: Build a relational queue set for unlabeled industrial text data , o represents the number of unlabeled categories, queue The size is , where b is the batch size. For the pseudo queue label The positive sample relationship representation , the comparison set is ; Step 26: Calculate the semantic similarity between the cluster center of the industrial text data and each relational queue, and minimize the semantic similarity based on the relational queue. and the classification probability calculated based on the classification weight parameter The cross entropy between , ; ; in, is the temperature coefficient. After each round of training, the maximum likelihood estimation is used to update the industrial entity relationship in the industrial text data in the iteration cycle. The queue tag , ; After each back propagation, Add to .

[0008] Furthermore, in step 3, the process of dynamically optimizing the training process and using the converged model to infer the relationship between industrial entities in the industrial knowledge graph includes the following steps: Step 31: In adjacent training rounds, record the number of changes in the predicted categories of industrial entity relationships in the industrial text data to assign weights to the samples. , collaboratively optimize instance-level contrast loss, ; ; in For the The relationship between the predicted industrial entities in the given industrial field text data, for In the Adjust parameters in the round; Step 32: Calculate the overall loss , ; in, is a hyperparameter; Step 33 If the loss is less than the specified minimum value or the maximum number of training rounds is reached, the training is terminated. As the final result of the industrial entity relationship in the industrial domain text, otherwise repeat steps 13 to 32.

[0009] The technical concept of the present invention is: to use prompt learning to integrate the knowledge features of annotation sets in industrial fields such as equipment, production processes and quality control; to use contrast loss to reduce the interference of noise in industrial text data; to build a relationship repository for unlabeled industrial text instances, collaboratively optimize annotation sets for dynamic clustering, improve the quality of industrial domain relationship extraction, and provide reliable data support for the construction of high-quality industrial knowledge graphs.

[0010] The beneficial effects of the present invention are: it can process multi-level and multi-dimensional correlation information between entities in industrial text data, overcome the limitations of traditional methods in processing complex domain data in low-resource environments, and improve the accuracy and efficiency of industrial knowledge graph construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 The figure is a flowchart of a complex semantic relationship extraction method for building industrial knowledge graphs. DETAILED DESCRIPTION

[0012] The present invention is further described below with reference to the accompanying drawings.

[0013] Reference Figure 1 , a complex semantic relationship extraction method for industrial knowledge graph construction, comprising the following steps: Step 1: Combine the industrial text data context to build dynamic prompts to obtain the relationship representation between industrial entities. The processing process includes the following steps: Step 11: Introduce industry-specific vocabulary as entity annotation set ,analyze Equipment name, fault type, process entity type, build entity type collection , using the strong correlation and temporal characteristics of industrial domain knowledge to Enhanced; based on industrial entity sets , using rules to identify the head and tail industrial entities in the given industrial field text data, and construct dynamic prompts ,in , Represents an identified industrial entity, , Is the pseudo type of the corresponding industrial entity, consisting of the collection initialization, The complex semantic relationships that may exist between industrial entities in the context of industrial text data; In this embodiment, the data text in the automotive field is annotated "Comprehensive situation of the fault report of car No. 62: Fault phenomenon: after accelerating, the accelerator was released and the engine stalled", use GPT or car manuals to build an entity annotation set containing automobile domain nouns such as "engine" and "accelerator" and entity type sets including "fault", "parts" , and The average embedding of entity types in initializes the virtual type tag . Use rule matching to identify entities with annotation sets Build dynamic prompts for the entities "engine" and "flame-out" ; Step 12: Dynamic prompt Text data from industry merge, The number of samples in each batch is then mapped word by word into a word vector sequence through the word segmenter, and the unified sequence length is , if the length of the mapping sequence is less than , then fill the end with 0; if the sequence length is greater than , then directly truncate the redundant characters at the end and obtain the encoded representation of the input sequence through the encoder ,exist Add absolute position encoding , ; in, Equal to the dimension of the embedding layer; In this embodiment, and The combined input sequence is .use For the deep learning model, if the instance in step 11 is preprocessed and the length after the tokenizer is less than 256, the tail of the sequence is padded with "0" tokens to the maximum length, and finally the word vector encoding and position encoding generated by the model are summed; Step 13, select the hidden layer as A deep learning model for industrial fields, select layers 2 to 3 Layer Embedding and building industrial entity relationship matrix , , ; In this embodiment, the sample used in step 12 is The hidden layer size of the deep learning model is 25, and the hidden layers from the 2nd to the 25th are selected. The embedding of positions builds a relationship matrix; Step 14: Calculate the industrial entity relationship matrix of , ; Step 15: Calculation and contextual representation of text data in industrial fields The similarity between ; The importance of the predicted industrial entity relationship representation of each feature layer of the model compared to the industrial field text data is measured using As The global weighted summation on the final industrial entity relationship representation is generated , ; In this embodiment, for In the last hidden layer Position embedding, calculated for each of the 2 to 25 hidden layers selected The similarity of position embeddings, such as ,by For the selected hidden layer The embedding weights will As a representation of the relationship between "engine" and "flame-out"; Step 2: Use contrastive learning to enhance the intra-class aggregation and inter-class separation of relational features in the relational storage queue. The processing process includes the following steps: Step 21: Construct instance-level contrastive loss for a small number of manually annotated and unannotated industrial text datasets. ; Among them, through rules and industrial entity annotation sets Identify business entities , , and the relationship between the model prediction Constitute a positive sample pair ; Select two different arbitrary span segments from the current industrial text data as pseudo-entities , Among them, for a small number of manually annotated industrial text datasets, select the real annotated relationship labels; for unlabeled industrial text datasets, select the central representation of the relationship cluster to which it belongs Constitute a negative sample pair ; In this embodiment, the annotation set "fault phenomenon of vehicle No. 795 fault report, abnormal noise at the right front wheel when driving on uneven road" is a positive sample and a negative sample pair is ; Unlabeled set "Comprehensive situation of car fault report No. 62: Fault phenomenon: After acceleration, the accelerator is released and the engine stalls", the positive sample pair is , select any span in the instance to generate a pseudo entity, such as is a negative sample pair; Step 22: Based on Compute the probability distribution of possible relationships between industrial entities, ; in, is the classification weight, is the bias parameter; Step 23: Calculate the relationship category of the industrial entity in the industrial data context: ; In this embodiment, the relationship label annotation set , the probability distribution calculated in step 22 is , then the final industrial entity’s relationship category in the industrial data context is “component failure”; Step 24: Construct a classification cross entropy loss for annotating industrial text datasets. ; Step 25: Build a relational queue set for unlabeled industrial text data , o represents the number of unlabeled categories, queue The size is , where b is the batch size, and the pseudo queue label is The positive sample relationship representation , the comparison set is ; In this embodiment, an empty relation queue set of size 7 is constructed for the unlabeled data set. , the maximum queue length is 240, the batch size is 8, and the size of each batch is 30. Among them, the relation representation belonging to queue 2, its comparison set is the set of all relation queues except queue 2; Step 26: Calculate the semantic similarity between the cluster center of the industrial text data and each relational queue, and minimize the semantic similarity based on the relational queue. and the classification probability calculated based on the classification weight parameter The cross entropy between , ; ; in, is the temperature coefficient. After each round of training, the maximum likelihood estimation is used to update the industrial entity relationship in the industrial text data in the iteration cycle. The queue tag , ; After each back propagation, Add to ; In this embodiment, the unlabeled data "Comprehensive situation of the fault report of car No. 62: Fault phenomenon: after acceleration, the accelerator was released and the engine stalled" is used to obtain the relationship embedding between the entity pairs "engine" and "stall" through steps 11-15, and the cluster center embedding of the relationship queue is calculated by the clustering algorithm. Then, the similarity between the relationship embedding and the center embedding of different queues is calculated respectively. If the similarity with queue 2 is the largest, the relationship label between the entity pairs is the result after the decoding of the cluster center embedding of queue 2; Step 3: Dynamically optimize the training process and use the converged model to infer the relationship between industrial entities in the industrial knowledge graph. The processing process includes the following steps: Step 31: In adjacent training rounds, record the number of changes in the predicted categories of industrial entity relationships in the industrial text data to assign weights to the samples. , collaboratively optimize instance-level contrast loss, ; ; in For the The relationship between the predicted industrial entities in the given industrial field text data, for In the Adjust parameters in the round; In this embodiment, taking the unlabeled data in step 26 as an example, if the entity pair "engine" and "off" are in the queue of relationship labels in the first round to the second round of training, The change has arrived , then the second round cross entropy coefficient is , otherwise ; Step 32: Calculate the overall loss , ; in, is a hyperparameter; Step 33 If the loss is less than the specified minimum value or the maximum number of training rounds is reached, the calculation ends. As the final result of the industrial entity relationship in the industrial domain text, otherwise repeat steps 13 to 32.

[0014] In this embodiment, taking the unlabeled data in step 26 as an example, the trained Cluster the relationship prediction results between the entity pairs "engine" and "stall" into queues , then the relationship between entity pairs is the result of decoding the embedding of the center of queue 2 cluster.

[0015] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A complex semantic relationship extraction method for industrial knowledge graph construction, characterized in that: The method comprises the following steps: Step 1: Combine the industrial text data context to construct dynamic prompts to obtain the relationship representation between industrial entities; Step 2: Use contrastive learning to enhance the intra-class aggregation and inter-class separation of relation features in the relation storage queue; Step 3: Dynamically optimize the training process and use the converged model to infer the relationship between industrial entities in the industrial knowledge graph.

2. A complex semantic relationship extraction method for industrial knowledge graph construction according to claim 1, characterized in that: In step 1, the process of constructing dynamic prompts in combination with industrial text data context to obtain the representation of the relationship between industrial entities includes the following steps: Step 11: Introduce industry-specific vocabulary as entity annotation set ,analyze Equipment name, fault type, process entity type, build entity type collection , using the strong correlation and temporal characteristics of industrial domain knowledge to Enhanced; based on industrial entity sets , using rules to identify the head and tail industrial entities in the given industrial field text data, and construct dynamic prompts ,in , Represents an identified industrial entity, , Is the pseudo type of the corresponding industrial entity, consisting of the collection initialization, The complex semantic relationships that may exist between industrial entities in the context of industrial text data; Step 12: Dynamic prompt Text data from industry merge, The number of samples in each batch is then mapped word by word into a word vector sequence through the embedding layer, and the unified sequence length is , if the length of the mapping sequence is less than , then fill the end with 0; if the sequence length is greater than , then directly truncate the redundant characters at the end and obtain the encoded representation of the input sequence through the encoder ,exist Add absolute position encoding , ； in, Equal to the dimension of the embedding layer; Step 13, select the hidden layer as A deep learning model for industrial fields, select layers 2 to 3 Layer Embedding and building industrial entity relationship matrix , ， ； Step 14: Calculate the industrial entity relationship matrix of , ； Step 15: Calculation and contextual representation of industrial text data The similarity between ； The importance of the predicted industrial entity relationship representation of each feature layer of the model compared to the industrial field text data is measured using As The global weighted summation on the final industrial entity relationship representation is generated , 。 3. A complex semantic relationship extraction method for industrial knowledge graph construction as claimed in claim 2, characterized in that: In step 2, the process of using contrastive learning to enhance intra-class aggregation and inter-class separation of relational features in the relational storage queue includes the following steps: Step 21: Construct instance-level contrastive loss for a small number of manually annotated and unannotated industrial text datasets. ； Among them, through rules and industrial entity annotation sets Identify business entities , , and the relationship between the model prediction Constitute a positive sample pair ; Select two different arbitrary span segments from the current industrial text data as pseudo-entities , Among them, for a small number of manually annotated industrial text datasets, select the real annotated relationship labels; for unlabeled industrial text datasets, select the central representation of the relationship cluster to which it belongs Constitute a negative sample pair ; Step 22: Based on Compute the probability distribution of possible relationships between industrial entities, ； in, is the classification weight, is the bias parameter; Step 23: Calculate the relationship category of the industrial entity in the industrial data context: ； Step 24: Construct a classification cross entropy loss for annotating industrial text datasets. ； Step 25: Build a relational queue set for unlabeled industrial text data , o represents the number of unlabeled categories, queue The size is , where b is the batch size, and the pseudo queue label is The positive sample relationship representation , the comparison set is ; Step 26: Calculate the semantic similarity between the cluster center of the industrial text data and each relational queue, and minimize the semantic similarity based on the relational queue. and the classification probability calculated based on the classification weight parameter The cross entropy between , ； ； in, is the temperature coefficient. After each round of training, the maximum likelihood estimation is used to update the industrial entity relationship in the industrial text data in the iteration cycle. The queue tag , ； After each back propagation, Add to .

4. A complex semantic relationship extraction method for industrial knowledge graph construction as claimed in claim 3, characterized in that: In step 3, the process of dynamically optimizing the training process and using the converged model to infer the relationship between industrial entities in the industrial knowledge graph includes the following steps: Step 31: In adjacent training rounds, record the number of changes in the predicted categories of industrial entity relationships in the industrial text data to assign weights to the samples. , collaboratively optimize instance-level contrast loss, ； ； in For the The relationship between the predicted industrial entities in the given industrial field text data, for In the Adjust parameters in the round; Step 32: Calculate the overall loss , ； in, is a hyperparameter; Step 33 When the loss is less than the specified minimum value or the maximum number of training rounds is reached, the training is terminated. As the final result of the industrial entity relationship in the industrial domain text, otherwise repeat steps 13 to 32.

Images