A knowledge graph construction method and system for continuous casting quality engineering field

Abstract

The application discloses a kind of knowledge graph construction method and system for continuous casting quality engineering field, utilize the method of deep learning, from the related data of the description billet generates defect, extract continuous casting quality entity and the relationship between entity, and according to continuous casting quality knowledge graph ontology model, determine continuous casting quality entity class and relationship class.The method proposed in the application can consider the close relationship between continuous casting quality entity and relationship, can avoid the error influence of entity recognition to the error propagation problem of relationship extraction, finally according to the method of deep learning, convert continuous casting quality knowledge alignment problem into pair continuous casting quality entity matching problem, utilize the method of semi-supervised incremental learning, the training process is divided into two stages, to reduce the demand amount of continuous casting quality field data, the knowledge graph formed by the application can reduce the difficulty of analyzing the root cause of continuous casting quality defect, is beneficial to quickly locate defect reason and carry out process optimization.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph construction, and relates to a method and system for constructing a knowledge graph in the field of continuous casting quality engineering. Background Technology

[0002] In the continuous casting steel production process, factors such as the quality of raw materials, the values ​​of process operating parameters, the operating status of equipment, and the experience and condition of operators and inspectors all affect the quality of continuously cast steel billets. Root cause analysis of defects in continuously cast steel billets involves analyzing data and event information from the continuous casting process to determine the causes of quality defects. Quality traceability enables continuous product quality improvement and optimization of production processes.

[0003] In current production environments, there are challenges such as complex production processes and difficulties in identifying the causes of quality defects. The causes of quality defects in continuously cast steel billets are mainly categorized into parameter anomalies and event anomalies. The production process of continuously cast steel billets involves numerous data types and large volumes, easily leading to data redundancy and hindering data analysis. Furthermore, quality defects may be caused by abnormal events, making them difficult to detect and reducing the efficiency of manual quality traceability analysis. Simultaneously, steel companies have low utilization rates of quality traceability knowledge, lacking knowledge accumulation and sharing.

[0004] Currently, the experience and knowledge of continuous casting billet quality traceability are held by only a few people, and there is a lack of sufficient application of this knowledge, resulting in a waste of knowledge and an inability to conduct traceability analysis efficiently and accurately.

[0005] These fragmented pieces of knowledge should be mined and organized into standardized, structured, and systematic knowledge to guide the traceability of continuously cast steel billet quality and improve the accuracy and comprehensiveness of the traceability. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of complex production processes and difficulty in finding the causes of quality defects in the existing continuous casting steel production process. However, due to excessive reliance on manual experience and lack of mastery of experience and knowledge on the traceability of continuous casting billet quality, the utilization rate of quality traceability knowledge is low, traceability analysis cannot be carried out efficiently and accurately, and data in the continuous casting steel production process cannot be analyzed in a timely manner. This invention provides a knowledge graph construction method and system for the field of continuous casting quality engineering.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] A method for constructing a knowledge graph in the field of continuous casting quality engineering includes the following steps:

[0009] Obtain relevant data describing defects in steel billets during continuous casting;

[0010] Construct an ontology model for a knowledge graph of continuous casting quality, and define entity categories and relation categories for continuous casting quality based on the ontology model;

[0011] The relevant data describing defects in steel billets are labeled based on entity category and relationship category, and the labeled data is obtained.

[0012] A deep learning model for joint extraction of continuous casting quality based on segmentation and recombination is constructed. The labeled data is used as the input of the deep learning model for joint extraction of continuous casting quality based on segmentation and recombination to obtain entity-relationship-entity triple data of continuous casting quality.

[0013] Semantic annotation is performed on the acquired entity-relationship-entity triple data to obtain semantically annotated data;

[0014] A knowledge alignment model for continuous casting quality is constructed. Semantically labeled data is used as input to the knowledge alignment model for continuous casting quality, and aligned triple data is obtained. A knowledge graph for the continuous casting quality engineering domain is constructed based on the aligned triple data.

[0015] A further improvement of the present invention is that:

[0016] The entity categories include: continuous casting quality defect name, continuous casting quality event, cause of continuous casting quality defect, characteristics of continuous casting quality defect, solutions to continuous casting quality problems, and characteristics of continuous casting events;

[0017] The relationship categories include: cause, induce, describe, take, and feature.

[0018] The deep learning model for joint extraction of continuous casting quality based on segmentation and recombination includes an input layer, a continuous casting quality sequence segmentation and recombination module, a continuous casting quality entity recognition module, and a continuous casting quality relationship extraction module.

[0019] The labeled data is used as the input to the input layer, and the output of the input layer is connected to the input of the continuous casting quality sequence segmentation and recombination module.

[0020] The continuous casting quality sequence segmentation and recombination module includes a continuous casting quality sequence segmentation module, a self-attention mechanism module, and a continuous casting quality sequence recombination module connected in sequence.

[0021] The output of the continuous casting quality sequence recombination module is connected to the input of the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module. The output of the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module is the continuous casting quality entity relationship extraction result.

[0022] The calculation result of the continuous casting quality sequence segmentation and recombination module is as follows:

[0023]

[0024] In the formula: ρ e ρ s ρ r These represent the segmented entity recognition sequence, shared sequence, and relation extraction sequence results, respectively. This represents the entity recognition sequence, shared sequence, and relation extraction sequence of the current word.

[0025] The calculation process of the self-attention mechanism module includes:

[0026] ρ e ρ s ρ r As input to the self-attention mechanism module, the following is calculated:

[0027] Q = W q P,K=W k P,V=W v P

[0028] In the formula: Q, K, and V represent the Query, Key, and Value matrices in the self-attention mechanism, respectively; W q W k W v These represent the weight matrices for Query, Key, and Value, respectively; P represents the input sequence.

[0029] The calculation results for each part of the self-attention mechanism module are as follows:

[0030] B = V(softmax(K) T Q))

[0031] In the formula: B represents the output of each head of the multi-head self-attention mechanism; Q, K, and V represent the Query matrix, Key matrix, and Value matrix in the self-attention mechanism, respectively;

[0032] The final calculation result of the self-attention mechanism module is:

[0033] O = W b (Linear([B e B s B r ]))

[0034] In the formula: O represents the final output of the self-attention mechanism module; W b B represents the weight matrix of the spliced ​​continuous casting quality entity identification sequence, relation extraction sequence, and shared sequence; e B s and B rThese represent the results of the continuous casting quality entity identification sequence, relation extraction sequence, and shared sequence after passing through the corresponding self-attention mechanism, respectively.

[0035] Both the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module include a fully connected layer, an activation function layer, and a feedforward neural network layer.

[0036] The output results of the continuous casting quality entity identification module and the continuous casting quality relationship extraction module include:

[0037] Given a set of relations R, for each relation l, construct a relation table whose elements Indicates w i and w j This is the probability of the first and last entities of the continuous casting quality of the relation. Extract all triples related to relation l. For each triple (w) i ,l,w j ), through connection and Obtain the feature representation of the triples:

[0038]

[0039] In the formula: The word w i And the word w j Representation of the continuous casting quality ternary group as the head and tail solids; and Features representing the word level; Represents global features at the sentence level;

[0040] Based on the feature representation of the triplet, the probability that the triplet for the continuous casting quality is correct is obtained through a feedforward neural network:

[0041]

[0042] In the formula: The word w i And the word w j The probability that the continuous casting quality triplet, which is the head and tail entity, is a type l relation; The word w i And the word w j The continuous casting mass triplet is represented as the head and tail entity; σ represents the sigmoid activation function.

[0043] A knowledge graph construction system for the field of continuous casting quality engineering includes:

[0044] The raw data acquisition module is used to acquire relevant data describing defects in steel billets during the continuous casting process.

[0045] The ontology model construction module is used to construct the ontology model of the continuous casting quality knowledge graph, and to define the entity categories and relation categories of continuous casting quality based on the ontology model;

[0046] The data annotation module is used to annotate the acquired data describing defects in steel billets based on entity categories and relationship categories, and obtain the annotated data.

[0047] The triplet acquisition module is used to construct a deep learning model for joint extraction of continuous casting quality based on segmentation and recombination. The labeled data is used as the input of the deep learning model for joint extraction of continuous casting quality based on segmentation and recombination to obtain entity-relationship-entity triplet data of continuous casting quality.

[0048] The semantic annotation module is used to perform semantic annotation on the acquired entity-relationship-entity triple data to obtain semantically annotated data;

[0049] The knowledge graph acquisition module is used to construct a continuous casting quality knowledge alignment model. It takes semantically labeled data as input to the continuous casting quality knowledge alignment model, obtains aligned triple data, and constructs a knowledge graph for the continuous casting quality engineering field based on the aligned triple data.

[0050] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described in this invention.

[0051] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described in this invention.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] This invention discloses a knowledge graph construction method for the field of continuous casting quality engineering. An ontology model is constructed based on relevant data describing defects in steel billets during the continuous casting process. Entity categories and relationship categories between entities are defined in the knowledge graph. The entities and relationships defined in the ontology model can more concisely describe continuous casting events, helping to highlight and grasp the key points of the events, thus serving as the basis for constructing the knowledge graph. Using deep learning methods, continuous casting quality entities and relationships between entities are extracted from the relevant data describing defects in steel billets, and the continuous casting quality entity categories and relationship categories are determined based on the continuous casting quality knowledge graph ontology model. Compared to the pipeline approach of first identifying continuous casting quality entities and then extracting continuous casting quality relationships based on the entity identification results, the method proposed in this invention takes into account the close relationship between continuous casting quality entities and relationships. This avoids the problem of error propagation in relationship extraction caused by errors in entity identification. Finally, by using deep learning, the continuous casting quality knowledge alignment problem is transformed into a pairwise continuous casting quality entity matching problem. Using semi-supervised incremental learning, the training process is divided into two stages to reduce the amount of data required in the continuous casting quality domain. At the same time, in order to reduce the impact of pseudo-samples on model performance, the knowledge graph constructed in this invention can reduce the difficulty of analyzing the root causes of continuous casting quality defects, reduce the reliance on human experience, facilitate the rapid location of defect causes and process optimization, and improve the quality of the cast billet, which is of great significance and value. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a diagram illustrating the construction of the knowledge graph ontology model for continuous casting quality in this invention.

[0056] Figure 2 This is a diagram showing the transformation of entity relationships in continuous casting quality according to the present invention.

[0057] Figure 3 This is a design drawing of the continuous casting quality joint extraction model of the present invention;

[0058] Figure 4 This is a detailed design diagram of the self-attention mechanism module described in this invention;

[0059] Figure 5 This is the overall design drawing of the continuous casting quality knowledge alignment model described in this invention;

[0060] Figure 6 This is a design diagram of the pre-trained language model fine-tuning module described in this invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0063] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0064] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0065] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0066] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0067] The present invention will now be described in further detail with reference to the accompanying drawings:

[0068] See Figures 1 to 6 This invention discloses a knowledge graph construction method for the field of continuous casting quality engineering, filling the gap in knowledge graph construction methods for continuous casting billet defects. It addresses the complexity and over-reliance on experience in defect querying and root cause analysis of continuous casting billets during production. Utilizing deep learning, it identifies continuous casting quality entities and relationships from natural language descriptions of continuous casting quality. A knowledge alignment method for continuous casting quality is proposed to eliminate ambiguous pairs of continuous casting quality entities, thereby enabling the construction of a knowledge graph for the continuous casting quality engineering field. This innovative approach helps to quickly locate defect causes, optimize production processes, and improve continuous casting quality during continuous casting production.

[0069] Specifically, the following steps are included:

[0070] Step 1: Obtain relevant data describing the defects of steel billets during continuous casting. That is, based on professional books on continuous casting quality control and core standards for continuous casting steel, obtain relevant natural language descriptions of defects of steel billets during continuous casting, and organize and preprocess them.

[0071] The natural language sources mentioned in this step include professional books on continuous casting quality control and defects, and compilations of core standards for continuous casting. Tables and images are removed during natural language extraction, and the sentences are then segmented according to periods. Next, the sentences undergo preprocessing, specifically including:

[0072] Convert traditional Chinese characters to simplified Chinese characters, remove rare characters, remove the '#' and '*' symbols, verify the closure of closed punctuation marks, unify the symbolic and Chinese character representations of chemical elements, and unify the symbolic and Chinese character representations of size comparisons.

[0073] This step, based on professional books and industry standards in the field of continuous casting, constructs an ontology model and defines the entity categories and relationship categories between entities in the knowledge graph. Compared to entire statements, the entities and relationships defined by the ontology model can more concisely describe continuous casting events, helping to highlight and grasp the key points of the events, thus serving as the foundation for constructing the knowledge graph.

[0074] Step 2: Construct the ontology model of the continuous casting quality knowledge graph, defining the entity categories and relation categories of continuous casting quality. See [link to relevant documentation]. Figures 1 to 2 .

[0075] See Figure 1First, by analyzing the content of continuous casting quality text, we define continuous casting quality entity categories and relation categories, and design the schema layer of the continuous casting quality knowledge graph using a top-down approach. Then, guided by the schema layer, we construct the data layer of the continuous casting quality knowledge graph using a bottom-up approach. Considering the characteristics of continuous casting domain datasets, we design a joint extraction method for continuous casting quality entity and relation knowledge, extracting entity and relation knowledge to form entity-relation-entity triples, which serve as the foundation for knowledge graph construction.

[0076] See Figure 2 Specifically, the ontology model defines six entity categories: continuous casting quality defect name, continuous casting quality event, cause of continuous casting quality defect, characteristics of continuous casting quality defect, solutions to continuous casting quality problems, and characteristics of continuous casting events.

[0077] The ontology model defines five relation categories: cause, lead to, take, describe, and feature.

[0078] Among these, continuous casting events can cause other continuous casting events, solutions, and defect causes, while defect causes lead to defect names; defects have defect characteristics and require corresponding solutions; characteristics describe the conditions and specific requirements for the occurrence of continuous casting events, solutions, and defect causes.

[0079] Step 3: Based on the continuous casting quality entities and relationship categories defined in Step 2, annotate the preprocessed continuous casting quality data from Step 1.

[0080] Step 4: Construct a deep learning model for joint extraction of continuous casting quality based on segmentation and recombination. Use the dataset labeled in Step 3 to train the model for joint extraction of entities and relations. See [link to relevant documentation]. Figures 3 to 4 .

[0081] See Figure 3 The deep learning model for joint extraction of continuous casting quality based on segmentation and recombination includes an input layer, a continuous casting quality sequence segmentation and recombination module, a continuous casting quality entity recognition module, and a continuous casting quality relationship extraction module.

[0082] The input layer:

[0083] The input to the input layer is the labeled continuous casting quality natural language text obtained in step 3.

[0084] The continuous casting quality sequence segmentation and recombination module:

[0085] The continuous casting quality sequence segmentation and recombination module includes a continuous casting quality sequence segmentation module, a self-attention mechanism module, and a continuous casting quality sequence recombination module.

[0086] The output of the continuous casting quality sequence segmentation module is connected to three self-attention modules. The output of the self-attention modules is connected to the input of the continuous casting quality sequence recombination module. The output of the continuous casting quality sequence recombination module is connected to the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module. The outputs of the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module are the continuous casting quality entity relationship extraction results.

[0087] The self-attention module includes an entity gate, a relation gate, and a three-head self-attention mechanism layer;

[0088] The continuous casting quality sequence recombination module includes a connection layer and an activation function layer;

[0089] The continuous casting quality entity recognition module includes a fully connected layer, an activation function layer, and a feedforward neural network layer.

[0090] The continuous casting quality relationship extraction module includes a fully connected layer, an activation function layer, and a feedforward neural network layer.

[0091] Specifically, the structure of the continuous casting quality sequence segmentation module is as follows:

[0092] The continuous casting quality sequence segmentation module includes a solid gate, a relation gate, and a connection layer. The specific calculation process includes:

[0093] The continuous casting quality sequence segmentation module contains a solid gate, as shown in the following formula:

[0094]

[0095] In the formula: Represents the segmentation sequence of solid gates; x t This represents the current continuous casting quality field being processed; h represents the output of the hidden layer in the previous round, used as the input for the current round of training; cummax(·) is defined as follows:

[0096] cummax(x)=cumsum(softmax(x))

[0097] In the formula: cumsum(·) is the cumulative summation function, and cummax(·) can be used to serialize the data into (0,…,0,1,…,1).

[0098] Furthermore, the relation gate formula is as follows:

[0099]

[0100] In the formula: The sequence represents the partitioning sequence of the relation gate; the definition of cummax(·) is as described above; x tThe current continuous casting quality field is represented; h represents the hidden layer output of the previous round, which is used as the input for the current round of training.

[0101] Entity gates divide a continuous casting quality corpus into two parts: a part relevant to entity recognition and a part irrelevant to entity recognition. Similarly, relation gates divide a continuous casting quality corpus into a part relevant to relation extraction and a part irrelevant to relation extraction.

[0102] Furthermore, according to the formula:

[0103]

[0104] In the formula: This represents the shared sequence of the current continuous casting quality field; x i The word indicating the current process; e i Indicates the result of solid gate partitioning; r i Indicates the result of relational gate partitioning; (x i ·e i (x) represents a sequence related to entity recognition; i ·r i ) represents the sequence related to relational extraction; operators This indicates taking the intersection of the two;

[0105]

[0106] In the formula: This represents the entity recognition portion of the current word sequence; (x i ·e i ) represents a sequence related to entity recognition; This represents the shared sequence of the current word; the - operator represents the difference between the two sets.

[0107]

[0108] In the formula: This indicates the partial sequence extracted to represent the relation of the current word; (x i ·r i ) represents the sequence related to the relation extraction; This represents the shared sequence of the current word; the - operator represents the difference between the two.

[0109] Furthermore, each input data point is divided into three parts: continuous casting quality sequence entity recognition part. Continuous casting quality sequence relationship identification section Shared with continuous casting quality sequence Next, using the formula:

[0110]

[0111] In the formula: ρ e ρ s ρ r These represent the segmented entity recognition sequence, shared sequence, and relation extraction sequence results, respectively. This represents the entity recognition sequence, shared sequence, and relation extraction sequence of the current word.

[0112] Furthermore, the input to the self-attention mechanism module is the ρ obtained above. e ρ s ρ r See Figure 3 ,

[0113] The self-attention mechanism module is a structure similar to a three-headed self-attention mechanism, divided into three groups. The input of each group corresponds to the entity recognition part, the relation recognition part, and the shared part, respectively. The self-attention mechanism for each group is calculated as follows:

[0114] Q = W q P,K=W k P,V=W v P

[0115] In the formula: Q, K, and V represent the Query, Key, and Value matrices in the self-attention mechanism, respectively; W q W k W v represents the weight matrices for Query, Key, and Value, respectively; P represents the input sequence.

[0116] The result for each part is shown in the formula:

[0117] B = V(softmax(K) T Q))

[0118] In the formula: B represents the output of each head of the multi-head self-attention mechanism; Q, K, and V represent the Query, Key, and Value matrices in the self-attention mechanism, respectively.

[0119] The final output of the self-attention mechanism module is shown in the formula:

[0120] O = W b (Linear([B e B s B r ]))

[0121] In the formula: O represents the final output of the self-attention mechanism module; W b B represents the weight matrix of the spliced ​​continuous casting quality entity identification sequence, relation extraction sequence, and shared sequence; e B sB r These represent the results of the continuous casting quality entity identification sequence, relation extraction sequence, and shared sequence after passing through the corresponding self-attention mechanism, respectively.

[0122] Furthermore, the continuous casting quality entity identification sequence, relation extraction sequence, and shared sequence are respectively processed through a self-attention mechanism module, and the outputs are fed into the continuous casting quality sequence reconstruction module. The reconstruction process is as follows:

[0123] μ e =o e +o s ,μ s =o s ,μ r =o r +o s

[0124] Where: μ e μ s μ r These represent the entity identification sequence, shared sequence, and relation extraction sequence of the recombined continuous casting quality field; o e o s o r These represent the results of the continuous casting quality entity identification sequence, shared sequence, and relation extraction sequence through the self-attention mechanism module, respectively.

[0125] The final output of the continuous casting quality sequence segmentation and recombination module is shown in the formula:

[0126] h = tanh(Linear([μ e μ s ,μ r ]))

[0127] h e =tanh(μ e )

[0128] h r =tanh(μ r )

[0129] In the formula: h is the hidden layer output, used to aid in the next round of training; h e and h r These represent the outputs used for continuous casting quality entity identification and continuous casting quality relationship extraction, respectively; μ e μ s μ r These represent the transformed and recombined continuous casting quality entity identification sequence, shared sequence, and relation extraction sequence, respectively.

[0130] Furthermore, the output layer of the continuous casting quality sequence segmentation and reorganization module is connected to the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module. The continuous casting quality entity recognition module needs to identify and classify the entities in the given continuous casting quality sequence.

[0131] Specifically:

[0132] Given a set of entity types ε, for each type k, fill in the continuous casting quality entity identification table, whose elements The word w i And the word w j The probabilities of the start and end positions for an entity of type k. For each group (w) i ,w j ), features at the connective level and sentence-level global features Then, it is passed through a fully connected layer and an ELU activation function to obtain the entity representation of the word group, as shown in the formula:

[0133]

[0134] In the formula: The word w i And the word w j As a representation of the continuous casting quality entity at the beginning and end; and Features representing the word level; Represents global features at the sentence level.

[0135] according to The probability that the continuously cast quality entity belongs to the k-th class is obtained through a feedforward neural network, as shown in the formula:

[0136]

[0137] In the formula: The word w i And the word w j The probability that the entities at the beginning and end are of type k continuous casting quality entities; The word w i And the word w j The continuous casting mass entities are represented at the beginning and end; σ represents the sigmoid activation function.

[0138] The continuous casting quality relation extraction module aims to identify all triples in a continuous casting quality sentence. This module only needs to predict the starting position of the entity, as the entire entity's range has already been identified by the continuous casting quality entity identification module. Similar to the continuous casting quality entity identification module, the continuous casting quality relation extraction module is also viewed as a table-filling problem for specific relations. Given a set of relations R, for each relation l, a relation table is constructed, whose elements... Indicates w i and w j This represents the probability of the first and last entities of the continuous casting quality relation. Therefore, we can extract all triples related to relation l using a table. For each triple (w) i ,l,w j ), through connection and Obtain the characteristic representation of the triples, as shown in the formula:

[0139]

[0140] In the formula: The word w i And the word w j Representation of the continuous casting quality ternary group as the head and tail solids; and Features representing the word level; Represents global features at the sentence level.

[0141] Furthermore, based on the triplet feature representation, the probability that the continuous casting quality triplet is correct is obtained through a feedforward neural network, as shown in the formula:

[0142]

[0143] In the formula: The word w i And the word w j The probability that the continuous casting quality triplet, which is the head and tail entity, is a type l relation; The word w i And the word w j The continuous casting mass triplet is represented as the head and tail entity; σ represents the sigmoid activation function.

[0144] The loss used in this embodiment of the invention is BCELoss, and the formula is as follows:

[0145] BCELoss(x,y)=-(ylogx+(1-y)log(1-x))

[0146] The objective of this invention is to perform entity and relation recognition in the field of continuous casting using natural language. Therefore, loss functions are defined for both the continuous casting quality entity recognition module and the continuous casting quality relation extraction module, as shown in the formula:

[0147]

[0148] In the formula: L ner and L re Let BCELoss represent the loss function for identifying continuous casting quality entities and the loss function for extracting continuous casting quality relationships, respectively; BCELoss is defined as previously. and Labels representing the actual continuous casting quality entities and relationships, respectively; and Labels representing the calculated continuous casting quality entities and relationships.

[0149] The overall loss function is shown in the formula:

[0150] L = L ner +L re

[0151] In the formula: L is the total loss function; L ner L is the loss function for identifying quality entities in continuous casting; re The loss function extracted for continuous casting quality relationships

[0152] This invention utilizes deep learning to extract continuous casting quality entities and their relationships from the natural language representation of continuous casting quality. Based on a knowledge graph ontology model of continuous casting quality, the entity and relationship categories are determined. Compared to a pipeline approach that first identifies continuous casting quality entities and then extracts relationships based on those entities, the proposed method considers the close relationships between entities and relationships, thus avoiding the propagation of errors in relationship extraction caused by entity identification errors.

[0153] Step 5: Based on the continuous casting quality entity-relationship-entity triplet obtained in Step 4, label it according to whether the semantics are the same, and prepare data for entity alignment. Based on this, the present invention discloses a specific embodiment, see Table 1.

[0154] The entity pairs of the continuous casting quality triplet obtained in step 4 are labeled to construct positive samples; any entity in the positive samples is replaced, and negative samples are constructed through undersampling.

[0155] Table 1

[0156]

[0157]

[0158] By using deep learning, the continuous casting quality knowledge alignment problem is transformed into a pairwise continuous casting quality entity matching problem. Semi-supervised incremental learning is used to divide the training process into two stages to reduce the amount of data required in the continuous casting quality domain. At the same time, in order to reduce the impact of spurious samples on model performance, this invention also introduces adversarial training to improve the robustness of the model.

[0159] Step 6: Based on the data obtained in Step 5, construct a continuous casting quality knowledge alignment model based on semi-supervised incremental learning, perform knowledge alignment, and construct a knowledge graph for the continuous casting quality engineering domain based on the aligned triples. See [link to details] for further information. Figures 5 to 6 .

[0160] The continuous casting quality knowledge alignment model based on semi-supervised incremental learning includes an input layer, a pre-trained language model fine-tuning module, a continuous casting quality knowledge pseudo-label generation module, and a continuous casting quality knowledge alignment incremental learning module. Figure 5 This is the overall flowchart of the entity alignment model based on semi-supervised incremental learning.

[0161] The input to the continuous casting knowledge alignment method is the labeled continuous casting quality data obtained in step 5. First, the BERT (CHINESE) model is fine-tuned using the labeled data. The entity-to-data input sequence is shown in the formula:

[0162] I = {[CLS], x1, x2, ..., x} n [SEP], y1, y2, ..., y m [SEP]

[0163] In the formula: I represents the input continuous casting quality sequence; [CLS] and [SEP] are special separators for the BERT model input; x i and y j The characters represent entity 1 and entity 2 respectively.

[0164] Furthermore, the pre-trained language model fine-tuning module includes a BERT (CHINESE) module and a result prediction module, such as... Figure 6 As shown. The BERT (CHINESE) module contains 12 Transformer encoders; each Transformer encoder includes a multi-head self-attention mechanism, a feedforward neural network, and a normalization layer. The continuous casting quality knowledge alignment result prediction module includes an aggregation module and a softmax classifier, as shown in the formula:

[0165]

[0166] In the formula: H is the output of the fine-tuning module of the pre-trained language model, i.e., the prediction result; This represents the parameter aggregation of the 12th and final layer of the BERT (CHINESE) model.

[0167] The continuous casting quality knowledge pseudo-label generation module includes an input layer, a prediction layer, and an output layer. The input layer takes unlabeled continuous casting quality entity pair samples as input. The prediction layer is the model fine-tuned by the pre-trained language model fine-tuning module. The output layer outputs whether the unlabeled continuous casting quality entity pair samples need to be fused and provides a confidence probability.

[0168] The incremental learning module for continuous casting quality knowledge alignment first selects pseudo-labeled continuous casting quality samples as incremental data and adds them to the labeled samples based on confidence probabilities, and then fine-tunes BERT(CHINESE). This module includes a BERT(CHINESE) module and a result prediction module. To overcome the negative impact of random noise on model performance and further enhance the generalization ability and robustness of the entity alignment model during continuous casting quality knowledge fusion, this invention introduces adversarial training in the incremental learning module. Adversarial training uses the BERT ATTACK model to generate perturbation samples. The BERT ATTACK model is a BERT-based adversarial sample generation technique that generates adversarial samples for continuous casting quality by finding vulnerable words in the continuous casting quality text and replacing them. After obtaining the adversarial samples of continuous casting quality, the adversarial samples of continuous casting quality, along with the labeled continuous casting quality data and the pseudo-labeled continuous casting quality data, are used as inputs for incremental learning of continuous casting quality knowledge alignment. The pre-trained language model is further fine-tuned using incremental learning to improve the robustness and accuracy of the continuous casting quality entity alignment model of this invention. The final prediction result is obtained through the continuous casting quality knowledge alignment result prediction module.

[0169] Based on the aligned triplet of continuous casting quality entity pairs and relationships, they can be used as the head and tail entities and relationships of the continuous casting quality knowledge graph, thus constructing a knowledge graph for the continuous casting quality engineering field.

[0170] This invention can reduce the difficulty of analyzing the root causes of continuous casting quality defects, facilitate the rapid location of defect causes and process optimization, and improve the quality of cast billets, which is of great significance and value.

[0171] This embodiment also discloses a knowledge graph construction system for the field of continuous casting quality engineering, including:

[0172] The raw data acquisition module is used to acquire relevant data describing defects in steel billets during the continuous casting process.

[0173] The ontology model construction module is used to construct the ontology model of the continuous casting quality knowledge graph, and to define the entity categories and relation categories of continuous casting quality based on the ontology model;

[0174] The data annotation module is used to annotate the acquired data describing defects in steel billets based on entity categories and relationship categories, and obtain the annotated data.

[0175] The triplet acquisition module is used to construct a deep learning model for joint extraction of continuous casting quality based on segmentation and recombination. The labeled data is used as the input of the deep learning model for joint extraction of continuous casting quality based on segmentation and recombination to obtain entity-relationship-entity triplet data of continuous casting quality.

[0176] The semantic annotation module is used to perform semantic annotation on the acquired entity-relationship-entity triple data to obtain semantically annotated data;

[0177] The knowledge graph acquisition module is used to construct a continuous casting quality knowledge alignment model. It takes semantically labeled data as input to the continuous casting quality knowledge alignment model, obtains aligned triple data, and constructs a knowledge graph for the continuous casting quality engineering field based on the aligned triple data.

[0178] A schematic diagram of a terminal device according to an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the various method embodiments described above. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0179] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention.

[0180] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0181] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0182] The memory can be used to store the computer program and / or module. The processor implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.

[0183] If the modules / units integrated into the terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0184] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a knowledge graph in the field of continuous casting quality engineering, characterized in that, Includes the following steps: Obtain relevant data describing defects in steel billets during continuous casting; Construct an ontology model for a knowledge graph of continuous casting quality, and define entity categories and relation categories for continuous casting quality based on the ontology model; The relevant data describing defects in steel billets are labeled based on entity category and relationship category, and the labeled data is obtained. A deep learning model for joint extraction of continuous casting quality based on segmentation and recombination is constructed. The labeled data is used as the input of the deep learning model for joint extraction of continuous casting quality based on segmentation and recombination to obtain entity-relationship-entity triple data of continuous casting quality. Semantic annotation is performed on the acquired entity-relationship-entity triple data to obtain semantically annotated data; A knowledge alignment model for continuous casting quality is constructed. Semantically labeled data is used as input to the knowledge alignment model for continuous casting quality, and aligned triple data is obtained. A knowledge graph for the continuous casting quality engineering domain is constructed based on the aligned triple data. The entity categories include: continuous casting quality defect name, continuous casting quality event, cause of continuous casting quality defect, characteristics of continuous casting quality defect, solutions to continuous casting quality problems, and characteristics of continuous casting events; The relationship categories include: cause, induce, describe, take, and feature is; The deep learning model for joint extraction of continuous casting quality based on segmentation and recombination includes an input layer, a continuous casting quality sequence segmentation and recombination module, a continuous casting quality entity recognition module, and a continuous casting quality relationship extraction module. The labeled data is used as the input to the input layer, and the output of the input layer is connected to the input of the continuous casting quality sequence segmentation and recombination module. The continuous casting quality sequence segmentation and recombination module includes a continuous casting quality sequence segmentation module, a self-attention mechanism module, and a continuous casting quality sequence recombination module connected in sequence. The output of the continuous casting quality sequence recombination module is connected to the input of the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module. The output of the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module is the continuous casting quality entity relationship extraction result.

2. The knowledge graph construction method for continuous casting quality engineering according to claim 1, characterized in that, The calculation result of the continuous casting quality sequence segmentation and recombination module is as follows: In the formula: , These represent the segmented entity recognition sequence, shared sequence, and relation extraction sequence results, respectively. , , This represents the entity recognition sequence, shared sequence, and relation extraction sequence of the current word.

3. The method for constructing a knowledge graph in the field of continuous casting quality engineering according to claim 2, characterized in that, The calculation process of the self-attention mechanism module includes: Will , As input to the self-attention mechanism module, the following is calculated: In the formula: , These represent the Query, Key, and Value matrices in the self-attention mechanism, respectively. , , These represent the weight matrices for Query, Key, and Value, respectively. Represents the input sequence; The calculation results for each part of the self-attention mechanism module are as follows: In the formula: This represents the output of each head in the multi-head self-attention mechanism; , and These represent the Query matrix, Key matrix, and Value matrix in the self-attention mechanism, respectively. The final calculation result of the self-attention mechanism module is: In the formula: This represents the final output of the self-attention mechanism module; The weight matrix represents the sequence of entities identified in continuous casting, the sequence of relationships extracted, and the sequence of shared data. , and These represent the results of the continuous casting quality entity identification sequence, relation extraction sequence, and shared sequence after passing through the corresponding self-attention mechanism, respectively.

4. The knowledge graph construction method for continuous casting quality engineering according to claim 1, characterized in that, Both the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module include a fully connected layer, an activation function layer, and a feedforward neural network layer.

5. A method for constructing a knowledge graph in the field of continuous casting quality engineering according to claim 4, characterized in that, The output results of the continuous casting quality entity recognition module and the continuous casting quality relationship extraction module include: Given a set of relations For each relation Construct a relational table, whose elements express and This is the probability of the first and last solids of the continuous casting quality in this relation; extract all sums of the relation. The relevant triples, for each triple , through connection , and Obtain the feature representation of the triples: In the formula: Representing words and words Representation of the continuous casting quality ternary group as the head and tail solids; and Features representing the word level; Represents global features at the sentence level; Based on the feature representation of the triplet, the probability that the triplet for the continuous casting quality is correct is obtained through a feedforward neural network: In the formula: Representing words and words The continuous casting quality ternary group, which consists of the head and tail solids, is the first... The probability of class relationships; Representing words and words Representation of the continuous casting quality ternary group as the head and tail solids; This represents the sigmoid activation function.

6. A knowledge graph construction system for the field of continuous casting quality engineering that implements the method of claim 1, characterized in that, include: The raw data acquisition module is used to acquire relevant data describing defects in steel billets during the continuous casting process. The ontology model construction module is used to construct the ontology model of the continuous casting quality knowledge graph, and to define the entity categories and relation categories of continuous casting quality based on the ontology model; The data annotation module is used to annotate the acquired data describing defects in steel billets based on entity categories and relationship categories, and obtain the annotated data. The triplet acquisition module is used to construct a deep learning model for joint extraction of continuous casting quality based on segmentation and recombination. The labeled data is used as the input of the deep learning model for joint extraction of continuous casting quality based on segmentation and recombination to obtain entity-relationship-entity triplet data of continuous casting quality. The semantic annotation module is used to perform semantic annotation on the acquired entity-relationship-entity triple data to obtain semantically annotated data; The knowledge graph acquisition module is used to construct a continuous casting quality knowledge alignment model. It takes semantically labeled data as input to the continuous casting quality knowledge alignment model, obtains aligned triple data, and constructs a knowledge graph for the continuous casting quality engineering field based on the aligned triple data.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5.

Images