A tin smelting process hyper-relational knowledge graph embedding representation method
By constructing a hyper-relational knowledge graph of the tin smelting process and using a large language model and graph neural network for embedded representation learning, the problem of insufficient knowledge representation in the tin smelting process was solved and the intelligence level of the tin smelting process was improved.
Patent Information
- Application Number
- CN202510984116.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-17
- Publication Date
- 2025-10-24
- Estimated Expiration
- 2045-07-17
AI Technical Summary
Existing knowledge graph embedding representation methods are difficult to effectively model additional information of tin smelting, and there is a lack of hyper-relational knowledge graph embedding representation methods for the tin smelting process flow, resulting in insufficient knowledge representation capabilities and affecting the intelligent development of the tin smelting process flow.
Construct a hyper-relational knowledge graph of the tin smelting process, extract hyper-relational facts through a large language model, combine graph neural networks and Transformer models to perform global and local embedding representation learning, aggregate hyper-relational information, and learn potential knowledge features.
It realizes the structured representation of tin smelting process knowledge, improves the embedding representation capability of knowledge graph, and promotes the intelligent development of tin smelting process.
Smart Images
Figure CN120470134B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to a tin smelting process super-relation knowledge graph embedding representation method and belongs to the technical field of knowledge representation. BACKGROUND
[0002] In modern industry, tin smelting is a complex industrial process of converting tin ore into pure tin, involving multiple stages and fine operations. By constructing a knowledge graph of the tin smelting process and performing embedding representation learning, unstructured experience knowledge can be converted into a structured knowledge representation form and used in production.
[0003] Currently, most knowledge graph embedding representation methods are based on triples <head entity-relation-tail entity>, which are difficult to model additional information for tin smelting, have limited knowledge representation capabilities, and, for example, for the tin smelting knowledge text “the reduction smelting process includes a weak reduction stage, which takes 20 minutes, and the tin content in the slag decreases from 15% to 5%”, using the triple <reduction smelting process-contains-weak reduction stage> for representation learning ignores the potential knowledge in additional information such as “required time” and “content change”.
[0004] Some related research has proposed a super-relation knowledge graph that can represent more complex information, i.e., introducing auxiliary key-value pairs to enhance the semantic information of simple triples. The super-relation knowledge graph embedding representation method based on super-relation facts <head entity-relation-tail entity-auxiliary key-value pair> has attempted to convert super-relation facts into n-tuple form for learning, but this destroys the structure of super-relation facts, resulting in a suboptimal model. Therefore, some technologies have designed a method of directly learning embeddings from super-relation facts using two convolutional neural network channels, but there are problems with insufficient consideration of local and global information.
[0005] Due to the complexity of the tin smelting process and the high degree of specialization of the knowledge in the tin smelting field, there is currently a lack of super-relation facts for the tin smelting process, and there is little technical research on super-relation knowledge graph embedding representation methods, and it has not been widely used in practice. How to construct a super-relation knowledge graph for the tin smelting process for super-relation knowledge graph embedding representation and use it for subsequent downstream tasks is a technical problem that needs to be solved. SUMMARY
[0006] The technical problem to be solved by the present application is to provide a tin smelting process super-relation knowledge graph embedding representation method for extracting tin smelting process knowledge, learning the potential feature information of the knowledge for downstream tasks, and promoting the digitalization and intelligentization development of the tin smelting industry.
[0007] The technical solution of the present application is: a tin smelting process super-relation knowledge graph embedding representation method, the specific steps of which are as follows:
[0008] Step1: Collect and organize the tin smelting process field knowledge text to be constructed based on each stage of the tin smelting process, including the stages of tin concentrate roasting, reduction smelting, slag fuming, fire refining and electrolytic refining;
[0009] Step2: Set the prompt word for information extraction of the tin smelting process, including the entity type information set, the relationship information set, and the additional information set in the tin smelting process super-relationship knowledge graph;
[0010] Step3: The large language model extracts super-relationship facts in the tin smelting process field knowledge text to be constructed according to the prompt word, including the main triple and auxiliary key-value pairs, taking the head entity of the main triple as the first node, the tail entity as the second node, the relationship as the edge between the first node and the second node, and the auxiliary key and auxiliary value in the auxiliary key-value pair as the attribute and attribute value attached to the corresponding relationship, to construct the tin smelting process super-relationship knowledge graph;
[0011] Step4: Take the super-relationship facts as samples in the training set, perform global embedding representation learning and local embedding representation learning, train the representation model, and obtain the final embedding of the tin smelting process knowledge graph for downstream tasks.
[0012] The Step2 is specifically:
[0013] Step2.1: Construct a prompt template, and the information structure extracted in the prompt word is defined as a super-relationship fact structure, each super-relationship fact consists of a main triple and n auxiliary key-value pairs that modify the main triple, where n is an integer greater than or equal to 0;
[0014] Step2.2: According to prior knowledge, add the entity type information set, the relationship information set, and the additional information set in the tin smelting process to the prompt word, the entity type information set includes entity type and entity type definition, for example, "[raw material]: the starting material involved in the tin smelting process, mainly including tin ore and tin concentrate"; the relationship information set is the logical connection between the stages of tin smelting and the interaction and conversion between raw materials, products and impurities within the stage, including "enter", "remove", "produce" and other relationships. The additional information set contains additional information supplemented with entities and relationships, such as "temperature", "impurity content";
[0015] Step2.3: Add m examples of tin smelting process field knowledge text to extract information;
[0016] Step2.4: Add the format requirements of the output and the method to handle non-existent information, and complete the prompt word setting.
[0017] The Step 4 is specifically as follows:
[0018] Step 4.1: Obtain hyper-relational facts in the hyper-relational knowledge graph of the tin smelting process as samples in the training set, and set initial embedding vectors for each target to be learned in the hyper-relational facts. The target to be learned includes the main triples Head entity in ,relation , tail entity and auxiliary key-value pairs Auxiliary keys in and auxiliary values , the embedding representation learning of the representation model includes global embedding representation learning and local embedding representation learning, and the embedding vector size is ,in is the dimension size;
[0019] Step 4.2: Perform global embedding representation learning. On the hyper-relational knowledge graph of the tin smelting process, use a graph neural network to pass messages and aggregate information between primary triples and auxiliary key-value pairs.
[0020] In the In the hidden layer, the node and neighbor node information are aggregated, and the auxiliary key-value pair level information is aggregated using the following formula:
[0021]
[0022] in, For the The auxiliary key-value pair embedding vector of the layer, For the Layer auxiliary key embedding vectors, For the Layer Auxiliary value embedding vectors, For the The auxiliary key-value pair weight matrix of the layer is , A set of auxiliary key-value pairs in a super-relational fact, including Auxiliary keys, for The i-th auxiliary key-value pair in ;
[0023] The relationship-secondary key-value pair level information is aggregated using the following formula:
[0024]
[0025] in, For the Layer-wise relationship-assisted key-value pair embedding vectors, for the i-th Layer-wise relationship embedding vectors in triplets, denotes the Hadamard product, is a hyper-parameter, ;
[0026] The entity-relationship-assisted key-value pair layer information is aggregated by the following formula, and the node embedding vectors are updated by aggregating the representation from the neighbor nodes, and the message passing is performed on the hyper-relational knowledge graph:
[0027]
[0028] wherein, for the i-th Head entity embedding vectors of the i-th layer, for the i-th Head entity embedding vectors of the i-th layer, for the i-th Tail entity embedding vectors of the i-th layer, for the i-th Head entity weight matrix of the i-th layer, for the i-th Information aggregation weight matrix of the i-th layer, with size , is a set of relationship-tail entity pairs connected with the head entity, is a relationship-tail entity pair in , represents the relationship, represents the tail entity, is a nonlinear activation function ReLU;
[0029] Step4.3: After global embedding representation learning, local embedding representation learning is performed according to each hyper-relational fact, each hyper-relational fact is converted into an input sequence, the dependence relationship between different positions in the sequence data is captured using a masked language modeling and attention mechanism-based model, and the model is trained to obtain the final embedding of the hyper-relational fact;
[0030] The hyper-relational fact is queried in sequence, converted into an input sequence , the corresponding embedding vector is , the data of the masked input sequence is represented by [MASK], and the information between the contexts is learned using an attention mechanism-based model:
[0031]
[0032] wherein, is a Transformer encoder, the number of layers of the Transformer encoder is C, , is the output of the first layer,
[0033] is the embedding vector of the i-th [MASK] mask position, The model outputs a probability distribution :
[0034]
[0035] wherein, and are learnable parameters, and the sizes are and , is the number of candidate labels of the mask position, is an activation function;
[0036] The prediction loss of the mask position is calculated using a cross-entropy loss function :
[0037]
[0038] wherein, is the number of mask positions, is the true label of the i-th mask position;
[0039] Step 4.4: Train the model parameters by backpropagating the loss and gradient descent algorithm, iterate the multiple hyper-relation fact samples in the training data set, until the preset iteration number is reached to obtain the trained representation model and the final independent embedding vector of each learning target, for downstream tasks.
[0040] The beneficial effects of the present application are: the present application fills the blank of the tin smelting process hyper-relation knowledge graph, realizes the automatic construction of the tin smelting process hyper-relation knowledge graph by means of the large language model, aggregates hyper-relation fact information, learns potential knowledge characteristics, improves the embedding representation ability of the hyper-relation knowledge graph, and promotes the intelligent development of the tin smelting process. BRIEF DESCRIPTION OF DRAWINGS
[0041] Figure 1 is the step flowchart of the present application;
[0042] Figure 2 is a part of the hyper-relation fact schematic diagram of the pyrorefining stage of the present application;
[0043] Figure 3 is the embedding representation learning model structure diagram of the present application. DETAILED DESCRIPTION
[0044] The application will be further described below in conjunction with the accompanying drawings and specific embodiments.
[0045] Embodiment 1: As shown in a tin smelting process flow super-relation knowledge graph embedding representation method, the specific steps are: Figure 1
[0046] Step 1: Based on each stage of the tin smelting process flow, collect and organize the tin smelting process flow field knowledge text to be constructed, and the stages include tin concentrate roasting, reduction smelting, slag fuming, fire refining and electrolytic refining stages.
[0047] Specifically, based on each stage, the tin smelting process flow field knowledge text to be constructed is collected from the Metallurgical Industry Publishing House's book "Tin Metallurgy", such as "Tin concentrate is boiled to remove arsenic, desulfurization, and then magnetic separation, so that the tin grade in tin concentrate is increased to more than 50%, arsenic content is less than 0.45%, and sulfur content is less than 0.5%, and placed in the stockyard" "The melting method is to heat solid crude tin containing iron and arsenic (called ethyl crude tin in production), and the smelting plant equipment mainly uses a reverberatory furnace; on the contrary, the condensation method is to melt the crude tin containing low iron and arsenic into liquid (called methyl crude tin), and the equipment is composed of a refining pot and a stirrer";
[0048] Step 2: Set the prompt words for information extraction of the tin smelting process flow, including the entity type information set, the relationship information set, and the additional information set in the tin smelting process flow super-relation knowledge graph.
[0049] Specifically, one or more examples of tin smelting process field knowledge text are added to extract information, the format requirement of the output is added, and the method for processing non-existent information is added, and the constructed prompt words are as follows:
[0050] "You are an intelligent model that can analyze context and extract information from it. The structure of the extracted information is a super-relation fact structure, each super-relation fact consists of a main triple and n auxiliary key-value pairs that modify the triple, where n is an integer greater than or equal to 0. Your task is to extract entity, relationship and auxiliary key-value pair information from the given sentence.
[0051] The head entity and tail entity in the extracted triple can refer to the entity type information set. The possible entity type information set is as follows: [`raw material`: the starting material involved in the tin smelting process, mainly including tin ore and tin concentrate, `impurities`: unwanted metal or non-metal elements and their compounds that exist in tin raw materials and affect the purity of tin, which need to be removed through the smelting process, such as lead, bismuth, copper, iron, sulfur, etc., `smelting equipment`: a series of industrial equipment used to heat metals or alloys above the melting point, with high-temperature heating capacity and the ability to control temperature and atmosphere conditions during the heating process to meet the smelting requirements of different metals or alloys, such as reverberatory furnace, top-blown furnace, electric furnace, etc., `chemical reagents`: chemicals used in the smelting process, such as reducing agents and fluxes, `smelting products`: products produced during the smelting process, including crude tin, refined tin, etc., `smelting by-products`: non-main products produced during the smelting process, such as slag and dust, etc.].
[0052] The relationship in the extracted triple can refer to the relationship information set. The possible relationship set is as follows: [`into`, `contains impurities`, `produces`, `needs`, `uses`, `next step`, `goes through`, `generates`].
[0053] The extracted auxiliary key-value pair can refer to the additional information set. The possible additional information set is as follows: [`content change`, `temperature change`, `reaction conditions`, `furnace grade`, `required time`, `heating time`, `operation conditions`, `product purity`, `detection method`, `impurity removal method`, `use equipment`].
[0054] From the input sentence text, all possible triples and their corresponding auxiliary key-value pair information are extracted, and the following conditions are met: the head entity should not be equal to the tail entity, the entity should not appear in the relationship set, the extraction format is {head entity, relationship, tail entity, [auxiliary key-value pair list]}, and NULL is used if there is no auxiliary key-value pair information. The following is an extraction example:
[0055] Input sentence text: The reduction smelting process includes a weak reduction stage, which takes 20 minutes, and the slag tin content decreases from 15% to 5%.
[0056] Output hyper-relation fact: {reduction smelting process, contains, weak reduction stage, [required time: 20 minutes, content change: slag tin content decreases from 15% to 5%]}
[0057] Step 3: The large language model extracts the hyper-relational facts in the knowledge text of the tin smelting process to be constructed based on the prompt words, including the main triples and auxiliary key-value pairs. The head entity of the main triple is used as the first node, the tail entity as the second node, the relationship as the edge between the first and second nodes, and the auxiliary key and auxiliary value in the auxiliary key-value pair as attributes and attribute values are attached to the corresponding relationship to construct the hyper-relational knowledge graph of the tin smelting process.
[0058] Specifically, the Tongyi Qianwen language model is used to extract super-relational facts in the knowledge text of the tin smelting process to be constructed based on the prompt words. Some super-relational facts are as follows: Figure 2 As shown in the figure, a hyper-relational knowledge graph of tin smelting process is constructed.
[0059] Step 4: Use hyper-relational facts as samples in the training set, perform global embedding representation learning and local embedding representation learning, train the representation model, and obtain the final embedding of the tin smelting process knowledge graph for downstream tasks.
[0060] Step 4.1: Obtain hyper-relational facts in the hyper-relational knowledge graph of the tin smelting process as samples in the training set, and set initial embedding vectors for each target to be learned in the hyper-relational facts. The target to be learned includes the main triples Head entity in ,relation , tail entity and auxiliary key-value pairs Auxiliary keys in and auxiliary values , the embedding representation learning of the representation model includes global embedding representation learning and local embedding representation learning, and the embedding vector size is ,in is the dimension size; Figure 3 As shown in the figure, the embedding representation learning of the representation model includes two parts: global embedding representation learning based on graph convolutional neural network and local embedding representation learning based on Transformer;
[0061] Step 4.2: Perform global embedding representation learning. On the hyper-relational knowledge graph of the tin smelting process, use a graph neural network to pass messages and aggregate information between primary triples and auxiliary key-value pairs. The number of graph neural network layers is set to 3.
[0062] In the In the hidden layer, the node and neighbor node information are aggregated, and the auxiliary key-value pair level information is aggregated using the following formula:
[0063]
[0064] in, For the The auxiliary key-value pair embedding vector of the layer, For the Layer auxiliary key embedding vectors, For the Layer Auxiliary value embedding vectors, For the The auxiliary key-value pair weight matrix of the layer is , A set of auxiliary key-value pairs in a super-relational fact, including Auxiliary keys, for The i-th auxiliary key-value pair in ;
[0065] The relationship-secondary key-value pair level information is aggregated using the following formula:
[0066]
[0067] in, For the The relation-auxiliary key-value pair embedding vector of the layer, For the The relation embedding vector in the triplet of the layer, represents the Hadamard product, is a hyperparameter, ;
[0068] Aggregate entity-relationship-auxiliary key-value pair hierarchical information through the following formula, and update the node embedding vector by aggregating representations from neighboring nodes, and perform message passing on the hyper-relational knowledge graph:
[0069]
[0070] in, For the The head entity embedding vector of the layer, For the The head entity embedding vector of the layer, For the The tail entity embedding vector of the layer, For the The head entity weight matrix of the layer, For the The information aggregation weight matrix of the layer is , is the set of relationship-tail entity pairs connected to the head entity, for A relation-tail entity pair in Representative relationship, Represents the tail entity, is the nonlinear activation function ReLU;
[0071] Step 4.3: After learning the global embedding representation, we learn local embedding representations for each super-relational fact. We convert each super-relational fact into an input sequence, use masked language modeling and an attention-based model to capture the dependencies between different positions in the sequence data, and train the model to obtain the final embedding of the super-relational fact.
[0072] Perform sequence queries on hyper-relational facts and convert them into input sequences , the corresponding embedding vector is , mask the input sequence data, and the masked part is represented by [MASK], for example , using an attention-based model to learn information between contexts:
[0073]
[0074] in, is the Transformer encoder, , the number of layers of the Transformer encoder is C, C is 2, , For the Layer output, is the output of the last layer;
[0075] Embedding vector for the i-th [MASK] mask position , the model outputs a probability distribution :
[0076]
[0077] in, and are learnable parameters, and their sizes are and , is the number of candidate labels for the mask position, is the activation function;
[0078] The prediction loss of the mask position is calculated using the cross entropy loss function :
[0079]
[0080] in, is the number of mask positions, is the true label of the i-th mask position;
[0081] Step 4.4: training the model parameters by the back propagation loss and gradient descent algorithm, iteratively training a plurality of hyper-relation fact samples in the training data set until a preset iteration number is reached to obtain a trained representation model and final independent embedding vectors of each learning target for downstream tasks.
[0082] The specific embodiments of the application are described in detail above with reference to the accompanying drawings, but the application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.
Claims
1. A tin smelting process hyper-relational knowledge graph embedding representation method, characterized in that, Comprise the following steps: Step1: based on tin smelting process of each stage, collect and organize the tin smelting process field knowledge text to be constructed, the each stage contains tin concentrate roasting, reduction smelting, slag fuming, fire refining and electrolytic refining stage; Step2: set the prompt word of tin smelting process information extraction, including entity type information set, relationship information set, additional information set in tin smelting process super relationship knowledge graph; Step3: the large language model extracts the super relationship fact in the tin smelting process field knowledge text to be constructed according to the prompt word, including the main triple and auxiliary key value pair, taking the head entity of the main triple as the first node, the tail entity as the second node, the relationship as the edge between the first node and the second node, and the auxiliary key and auxiliary value in the auxiliary key value pair as the attribute and attribute value attached to the corresponding relationship, and constructing the tin smelting process super relationship knowledge graph; Step4: the super relationship fact is used as the sample in the training set, global embedding representation learning and local embedding representation learning are carried out, the representation model is trained, and the final embedding of the tin smelting process knowledge graph is obtained, which is used for downstream tasks; The Step2 specifically is: Step2.1: a prompt template is constructed, and the information structure extracted in the prompt word is defined as a super relationship fact structure, each super relationship fact consists of a main triple and n auxiliary key value pairs modifying the main triple, wherein n is an integer greater than or equal to 0; Step2.2: according to prior knowledge, add the entity type information set, relationship information set and additional information set in the prompt word, the entity type information set includes entity type and entity type definition, the relationship information set is the logical connection between the raw materials, products and impurities in each stage of tin smelting and the mutual action and conversion between them, and the additional information set includes additional information supplemented with entities and relationships; Step2.3: m tin smelting process field knowledge texts are added to extract information examples; Step2.4: the format requirement of output and the method of processing non-existent information are added, and the prompt word setting is completed; The Step4 specifically is: Step 4.1: Obtain hyper-relational facts in the hyper-relational knowledge graph of the tin smelting process as samples in the training set, and set initial embedding vectors for each target to be learned in the hyper-relational facts. The target to be learned includes the main triples Head entity in ,relation , tail entity and auxiliary key-value pairs Auxiliary keys in and auxiliary values , the embedding representation learning of the representation model includes global embedding representation learning and local embedding representation learning, and the embedding vector size is ,in is the dimension size; Step4.2: global embedding representation learning is carried out, and graph neural network is used to perform message passing and information aggregation on the main triple and auxiliary key value pair in the tin smelting process super relationship knowledge graph; In the first In the layer of hidden layers, the aggregate node and neighbor node information, and aggregate the auxiliary key-value pair hierarchical information by the following formula: ； in, For the The auxiliary key-value pair embedding vector of the layer, For the Layer auxiliary key embedding vectors, For the Layer auxiliary value embedding vectors, For the The auxiliary key-value pair weight matrix of the layer is , A set of auxiliary key-value pairs in a super-relational fact, including Auxiliary keys, for The i-th auxiliary key-value pair in ; The relationship-auxiliary key value pair level information is aggregated by the following formula: ； wherein, is the layer relationship-assisted key-value pair embedding vector, is the layer relationship embedding vector in a triple, denotes the Hadamard product, is a hyperparameter, ; The entity-relationship-auxiliary key value pair level information is aggregated by the following formula, and the node embedding vector is updated by aggregating the representation from the neighbor nodes, and the message passing is carried out on the super relationship knowledge graph: ； wherein, is the head entity embedding vector for the layer, is the head entity embedding vector for the layer, is the tail entity embedding vector for the layer, is the head entity weight matrix for the layer, is the information aggregation weight matrix for the layer of size , is a set of relation-tail entity pairs connected with the head entity, is a relation-tail entity pair in , represents a relation, represents a tail entity, is a nonlinear activation function ReLU; Step4.3: after global embedding representation learning, local embedding representation learning is carried out according to each super relationship fact, each super relationship fact is converted into an input sequence, the mask language modeling and attention mechanism based model are used to capture the dependency relationship between different positions in the sequence data, and the model is trained to obtain the final embedding of the super relationship fact. Sequence query is performed on the hyper-relation fact, and converted into an input sequence The corresponding embedding vector is The data of the mask input sequence, the masked part is represented by [MASK], and a model based on an attention mechanism is used to learn the information between contexts: ； wherein, is a Transformer encoder, the number of layers of the Transformer encoder is C, , is the first layer output; For the i-th [MASK] mask position the embedding vector , the model outputs a probability distribution : ； wherein, and are learnable parameters with sizes and , is the number of candidate labels for the mask position, is an activation function; The prediction loss for the mask positions is calculated using a cross-entropy loss function : ； wherein, is the number of mask positions, is the true label of the i-th mask position; Step4.4: training the model parameters by back-propagating the loss and gradient descent algorithm, iteratively training multiple hyper-relation fact samples in the training data set until a preset number of iterations is reached to obtain a trained representation model and final independent embedding vectors of each learning goal for downstream tasks.
Citation Information
Patent Citations
Global data space link prediction method and system based on super relation
CN115757822A