A Method for Entity Relationship Extraction in the Automotive Parts Supply Chain Based on Heterogeneous Graph Neural Networks

Abstract

This invention discloses a method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks. The method includes: S1: acquiring automotive parts supply chain information to be processed and predefined relationship types; S2: performing word information embedding and high-dimensional word vector embedding on the automotive parts supply chain information and relationship types respectively, obtaining initial representations of word nodes and relationship nodes; S3: inputting the initial representations of word nodes and relationship nodes into a trained relationship extraction model, outputting several subject-relationship-object triples; S4: using all subject-relationship-object triples as the entity relationship extraction result of the automotive parts supply chain information. This invention first models entities and their relationships as nodes in a graph, then iteratively fuses and updates node representations through a multi-layer heterogeneous graph neural network and integrates subject features into the node representations through an attention mechanism, finally extracting subject-relationship-object triples all at once.

Description

Technical Field

[0001] This invention relates to the fields of Internet big data and next-generation information technology, specifically to a method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks. Background Technology

[0002] In a cloud service environment, the automotive parts supply chain is undergoing a trend towards increasing complexity and diversification. The multi-entity, multi-link structure of the automotive parts supply chain results in diverse information types, wide-ranging sources, massive volumes, and complex relationships. This heterogeneous information is difficult to describe uniformly and adaptively identify, posing significant challenges to the management and optimization of the automotive parts supply chain. If this multi-source heterogeneous information cannot be effectively integrated and utilized, the collaborative efficiency of the automotive parts supply chain will be severely affected, hindering product maintenance and parts supply under large-scale manufacturing models.

[0003] Knowledge graphs, as an effective tool, can integrate multi-source heterogeneous information into a unified structure, revealing the complex relationships between elements in the supply chain and thus forming a comprehensive view to support cross-domain collaborative interaction. However, accurately extracting entities and relationships from multi-source heterogeneous information is the key to building a knowledge graph and also the main technical challenge currently faced. Traditional entity and relationship extraction methods often fail to achieve ideal results when dealing with complex domains such as the automotive parts supply chain, mainly because they cannot fully utilize the deep-level connections between entities and relationships.

[0004] Most existing research on entity relation extraction is based on pipeline methods. This involves first identifying entities in a sentence, and then classifying relations for each entity pair. This approach ignores the dependency between entity identification and relation classification, making it prone to error propagation. To preserve the correlation between entity identification and relation classification, subsequent research has proposed joint training methods for entities and relations. However, most existing methods cannot correctly handle scenarios where sentences contain multiple overlapping relation triples and generate a large amount of redundant information. A segment of information from an automotive parts supply chain contains multiple entities with overlapping triples. In such a segment, an entity can belong to multiple triples simultaneously, and some entities may be subjects in one triple but objects in another, exhibiting a complex link structure. Existing entity relation extraction methods cannot effectively identify the entities and the complex relationships between them in automotive parts supply chain information. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem this invention aims to solve is: how to provide a method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks. First, entities and their relationship types are uniformly modeled as nodes in a graph. Then, the node representations are iteratively fused and updated using a multi-layer heterogeneous graph neural network, and a subject feature fusion module based on an attention mechanism is used to fuse subject features into word node representations. Finally, subject-relationship-object triplets are extracted in one step using a subject marker and a relation-object pair marker, thereby improving the efficiency and accuracy of entity relationship extraction in the automotive parts supply chain.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks includes:

[0008] S1: Obtain the automotive parts supply chain information to be processed, as well as the predefined relationship types;

[0009] S2: Perform word information embedding and word vector embedding on the automotive parts supply chain information and relationship types respectively to obtain the initial representations of word nodes and relationship nodes;

[0010] S3: Input the initial representations of word nodes and relation nodes into the trained relation extraction model, and output several subject-relation-object triples;

[0011] The processing steps when training the relation extraction model are as follows:

[0012] S301: Iteratively fuse word nodes and relation nodes through a multi-layer heterogeneous graph neural network to update the word node representation, and use the word node representation output by the last layer of the heterogeneous graph neural network as the updated word node representation;

[0013] S302: Find several subjects by marking the start and end positions of the subjects in the updated word node representation using the subject marker;

[0014] S303: Randomly select a subject, and fuse the features of the current subject with the updated word node representation through the subject feature fusion module to obtain the word node representation fused with subject features;

[0015] S304: By using the relation-object pair marker to mark the position of the relation and the start and end positions of the object in the word node representation of the fused subject features, find the relation-object pair corresponding to the subject;

[0016] S305: Combine the subject and the corresponding relation-object pair to generate a subject-relation-object triple;

[0017] S306: Repeat S303 to S305 until all subjects are selected and all subject-relationship-object triples are extracted;

[0018] S307: Calculate the loss function based on the extracted subject-relation-object triples and (pre-labeled) real subject-relation-object triples, and back-optimize the parameters of the relation extraction model;

[0019] S308: Repeat steps S301 to S307 to iteratively train the relation extraction model until the model converges or reaches the preset number of iterations;

[0020] S4: Take all the subject-relationship-object triples output as the entity relation extraction results of the automotive parts supply chain information to be processed.

[0021] Preferably, in step S2, the processing steps for word information embedding and word vector embedding are as follows:

[0022] S201: Perform word segmentation on automotive parts supply chain information to obtain the corresponding one-hot vector representation;

[0023] S202: Input the one-hot vector representation of the automotive parts supply chain information into the pre-trained BERT model to embed word information and obtain the hidden features of the automotive parts supply chain information;

[0024] S203: Use the hidden features of the last layer of the BERT model as the initial representation of word nodes;

[0025] The formula is expressed as:

[0026]

[0027] In the formula: This represents the initial representation of the i-th word node; w i This represents the i-th word in the original automotive parts supply chain information; n represents the number of words in a segment of automotive parts supply chain information.

[0028] S204: High-dimensional word vector embedding is performed on relation types through a relation type embedding layer to obtain relation node representations;

[0029] The formula is expressed as:

[0030]

[0031] In the formula: This represents the initial representation of the j-th relation node; r j represents the predefined j-th relation type; E represents the relation type embedding layer; m represents the number of predefined relation types; W and b are trainable parameters.

[0032] Preferably, in step S301, the processing steps of the multilayer heterogeneous graph neural network are as follows:

[0033] S3011: Use the initial representations of word nodes and relation nodes as input to a multilayer heterogeneous graph neural network;

[0034] S3012: Iteratively fuse and update the representations of word nodes and relation nodes through a multi-layer heterogeneous graph neural network;

[0035] The formula is expressed as:

[0036]

[0037] In the formula: This represents the word node representation of the output of the (l+1)th layer heterogeneous graph neural network; This represents the relation node representation of the output of the (l+1)th layer heterogeneous graph neural network; HGNN represents a heterogeneous graph neural network. This represents the word node h in the l-th layer of the heterogeneous graph neural network. i The set of all neighbor relationship nodes; Represents the relation node r in the l-th layer heterogeneous graph neural network. j The set of all neighboring word nodes; This represents the word node representation of the output of the l-th layer heterogeneous graph neural network.

[0038] Preferably, in step S3012, the processing steps for each layer of the heterogeneous graph neural network are as follows:

[0039] 1) The initial representations of word nodes and relation nodes are converted into word node vectors and relation node vectors through a linear layer;

[0040] The formula is expressed as:

[0041] α i =W h h i +b i ;

[0042] β j =W r r j +b r ;

[0043] In the formula: α i β i W represents the word node vector and the relation node vector. h b h W r b r These are trainable parameters;

[0044] 2) Concatenate the word node vector and the relationship node vector and calculate the similarity. Then, calculate the attention weight between the word node and the relationship node based on the similarity.

[0045] γ ij =Linear(W a [α i :β j ]);

[0046]

[0047] In the formula: Represents word node h i and relation node r j Attention weights between; r ij Represents the word node vector α i and relation node vector β i Similarity between them; [:] indicates vector concatenation; Liner indicates a linear layer; f indicates softmax; W a Indicates the parameters that can be trained;

[0048] 3) Attention weights based on word nodes and relationship nodes Update the representation of word nodes using a gating mechanism;

[0049] The formula is expressed as:

[0050]

[0051] In the formula: This represents the word node representation after processing by a heterogeneous graph neural network.

[0052] 4) Represent the word nodes after processing by the heterogeneous graph neural network. The word node representation is added to the input word node and used as the output word node representation of the heterogeneous graph neural network;

[0053] The formula is expressed as:

[0054]

[0055] Preferably, in step S302, the processing steps of the main marker are as follows:

[0056] S3021: Map each word node in the updated word node representation to a space K with a two-dimensional output by linear mapping. The two dimensions of each word node in space K represent the probability of being the start position and the end position of the topic, respectively.

[0057] The formula is expressed as:

[0058] K = (σ(Linear(Dropout(X)))) 2 ;

[0059] In the formula: X∈R n×d This represents the updated word node representation; Dropout is used to prevent overfitting; σ represents the sigmoid activation function; Liner represents a linear layer;

[0060] S3022: Determine if each word node is the start and end position of the main body: If the probability of a word node being the start and end position of the main body is greater than the set threshold, its dimension is set to 1; otherwise, it is set to 0, thus obtaining the start and end position indexes of the main body.

[0061] S3023: Find the actual subject S by using the start and end position indices of the subject.

[0062] Preferably, in step S303, the processing steps of the main feature fusion module are as follows:

[0063] S3031: Calculate the main features based on all word nodes between the start and end positions of the main body;

[0064] S3032: The main features are fused with each word node in the updated word node representation through a gating mechanism to generate a fused representation of the main features of the word node;

[0065] The formula is expressed as:

[0066]

[0067] In the formula: The main feature fusion representation of the i-th word node; h i The i-th word node in the updated word node representation; gate represents the gating mechanism; s f Represents the main feature; σ represents the sigmoid activation function; W g b g The linear layer parameters of the gating mechanism are represented; cue_len represents the effective length of the information.

[0068] S3033: Add the fused representation of the main features of the word node to the updated word node representation to obtain the word node representation with fused main features;

[0069] The formula is expressed as:

[0070]

[0071] In the formula: This represents the i-th word node representing the fused main features; The main feature fusion representation of the i-th word node; h i This represents the updated representation of the i-th word node.

[0072] Preferably, in step S3031, the main features are calculated using the following formula:

[0073]

[0074] In the formula: s f Indicates the main features of the current training; h x and h y represents the updated word node representations of the current training subject at the x-th and y-th positions, respectively; b and e represent the start and end positions of the current training subject, respectively; a x Let x represent the attention weight of the x-th word node in the main body, where x,y∈(b,e); It is a trainable parameter.

[0075] Preferably, in step S304, the processing steps for the relation-object pair marker are as follows:

[0076] S3041: The d-dimensional representation of each word node in the word node representation of the fused subject features is mapped from the d-dimensional representation to the (2×m×d)-dimensional representation through a linear layer, where m represents the number of predefined relation types; the value of the first dimension 2 represents the probability that each word node is the starting and ending position of the object, and the value of the second dimension m represents the probability that each word node is the position of the relation.

[0077] The formula is expressed as:

[0078] (r,o)=(σ(Linear(Dropout(X′)))) 4 ;

[0079] In the formula: (r,o) represents the starting position index, ending position index, and relation category index of the object in the relation-object pair; X′∈R n×d The word node representation represents the fused main features; σ represents the sigmoid activation function; Liner represents a linear layer;

[0080] S3042: Determine whether each word node is the start and end position of an object, and whether it is one of the m relation types: if the probability of the word node's relation position is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0; if the probability of the word node's start and end position of an object is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0.

[0081] S3043: Obtain the relation object pair (R,O) using the relation category index and the object start and end position indexes.

[0082] Preferably, a subject-relation-object triple is represented as (S, R, O), where S represents the subject, R represents the relation, and O represents the object.

[0083] Preferably, in step S3, the loss function used when training the relation extraction model is as follows:

[0084]

[0085] In the formula: U is the set of all relations in the training set; I represents a labeled segment of automotive parts supply chain information; T represents the triples contained in this segment of information; S∈T represents the subject in the triple; T|S represents the set of all triples with S as the subject; (R,O)∈T|S represents the relation-object pair with S as the subject; U / T∈S represents all relations other than those already contained in the triple set. The empty relation-object pair is used to indicate that there is no corresponding relation or object; Indicates the probability of subject prediction. This represents the predicted probability of a pair of relational objects.

[0086] Compared with existing technologies, the method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks in this invention has the following advantages:

[0087] This invention embeds word information and high-dimensional word vectors into automotive parts supply chain information and relationship types, enabling the unified modeling of entities and relationships within the supply chain as nodes in a graph. This not only preserves the richness of the original information but also provides a data foundation for subsequent relationship extraction. Secondly, this invention uses a multi-layer heterogeneous graph neural network to iteratively fuse and update the representation of word nodes and relationship nodes, capturing deep-level connections within the information. This approach performs exceptionally well when handling complex supply chain information containing multiple overlapping triples (i.e., multiple entities sharing the same or similar relationships). Furthermore, through continuous iteration, the representation of each node is fused with the representations of its neighbors, gradually refining the features of each node. This allows for faster location of key information, thereby improving the efficiency of entity relationship extraction in the automotive parts supply chain. Finally, the subject feature fusion module fuses the features of the subject with the updated word node representation, enhancing the node's representational capabilities. This allows the model to better focus on the relationships between subject-relationship-object triples within the information, enabling a deeper understanding of the meaning within the automotive parts supply chain information and more accurate judgments, thus improving the model's entity relationship extraction performance. Then, this invention marks the positions of the subject, relation, and object through the subject marker and the relation-object pair marker, which not only ensures the structural integrity of the extracted triples, but also ensures their semantic accuracy. At the same time, the subject marker and the relation-object pair marker can extract the subject-relationship-object triples from the automotive parts supply chain information in one go, instead of dividing entity relationship extraction into two sub-tasks, entity extraction and relationship classification, as in traditional entity relationship extraction models and methods. The method of this invention can better focus on the potential connections between relations and entities, and is more suitable for extracting entity relationships from automotive parts supply chain information with multiple subjects and multiple overlapping triples, thereby improving the accuracy of entity relationship extraction in the automotive parts supply chain. Attached Figure Description

[0088] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0089] Figure 1 This is a network architecture diagram of the relation extraction (AER-HGNN) model.

[0090] Figure 2 Examples of triples included in actual automotive parts supply chain information.

[0091] Figure 3 This describes the operational flow of the iterative update mechanism for heterogeneous graph neural networks.

[0092] Figure 4 The steps of the main feature fusion mechanism.

[0093] Figure 5 Statistical analysis of data from two publicly available datasets, NYT and WebNLG.

[0094] Figure 6 The results are from ablation experiments under different information complexities.

[0095] Figure 7 This study compares the performance of various heterogeneous layer number models under different information complexity conditions.

[0096] Figure 8 A simple knowledge graph constructed using triples extracted from the AER-HGNN model. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0098] The following detailed explanation illustrates the specific implementation methods:

[0099] Example:

[0100] The applicant found that in the task of extracting entity relationships in the automotive parts supply chain, the relationships between entities are often determined by contextual information rather than the target entities themselves. Integrating relationship information beforehand into the entity relationship extraction task can effectively reduce the extraction of semantically irrelevant entities, decrease the generation of redundant triples, and improve extraction efficiency. Furthermore, existing models separate entity extraction and relationship extraction into two tasks, and the final model evaluation only assesses the performance of entity extraction, rather than focusing on the overall triples. Applying these models to the practical entity relationship extraction in the automotive parts supply chain is impractical.

[0101] In response to the above findings, this embodiment discloses a method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks.

[0102] like Figure 1 As shown, the method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks includes:

[0103] S1: Obtain the automotive parts supply chain information to be processed, as well as the predefined relationship types;

[0104] S2: Perform word information embedding and high-dimensional word vector embedding on the automotive parts supply chain information and relationship types respectively to obtain the initial representations of word nodes and relationship nodes (on the graph);

[0105] S3: Input the initial representations of word nodes and relation nodes into the trained relation extraction (AER-HGNN) model, and output the subject-relation-object (entity relation) triples;

[0106] The processing steps when training the relation extraction model are as follows:

[0107] S301: Iteratively fuse word nodes and relation nodes through a multi-layer heterogeneous graph neural network to update the word node representation. The word node representation output by the last layer of the heterogeneous graph neural network is used as the updated word node representation (for subsequent entity relation extraction).

[0108] S302: Find several subjects by marking the start and end positions of the subjects in the updated word node representation using the subject marker;

[0109] S303: (During each training session) Randomly select a subject, and fuse the features of the current subject with the updated word node representation through the subject feature fusion module to obtain the word node representation fused with subject features;

[0110] S304: By using the relation-object pair marker to mark the position of the relation and the start and end positions of the object in the word node representation of the fused subject features, find the relation-object pair corresponding to the subject;

[0111] S305: Combine the subject and the corresponding relation-object pair to generate a subject-relation-object triple;

[0112] S306: Repeat S303 to S305 until all subjects are selected and all subject-relationship-object triples are extracted;

[0113] S307: Calculate the loss function based on the extracted subject-relation-object triples and (pre-labeled) real subject-relation-object triples, and back-optimize the parameters of the relation extraction model;

[0114] S308: Repeat steps S301 to S307 to iteratively train the relation extraction model until the model converges or reaches the preset number of iterations;

[0115] S4: Take all the subject-relationship-object triples output as the entity relation extraction results of the automotive parts supply chain information to be processed.

[0116] This invention embeds word information and high-dimensional word vectors into automotive parts supply chain information and relationship types, enabling the unified modeling of entities and relationships within the supply chain as nodes in a graph. This not only preserves the richness of the original information but also provides a data foundation for subsequent relationship extraction. Secondly, this invention uses a multi-layer heterogeneous graph neural network to iteratively fuse and update the representation of word nodes and relationship nodes, capturing deep-level connections within the information. This approach performs exceptionally well when handling complex supply chain information containing multiple overlapping triples (i.e., multiple entities sharing the same or similar relationships). Furthermore, through continuous iteration, the representation of each node is fused with the representations of its neighbors, gradually refining the features of each node. This allows for faster location of key information, thereby improving the efficiency of entity relationship extraction in the automotive parts supply chain. Finally, the subject feature fusion module fuses the features of the subject with the updated word node representation, enhancing the node's representational capabilities. This allows the model to better focus on the relationships between subject-relationship-object triples within the information, enabling a deeper understanding of the meaning within the automotive parts supply chain information and more accurate judgments, thus improving the model's entity relationship extraction performance. Then, this invention marks the positions of the subject, relation, and object through the subject marker and the relation-object pair marker, which not only ensures the structural integrity of the extracted triples, but also ensures their semantic accuracy. At the same time, the subject marker and the relation-object pair marker can extract the subject-relationship-object triples from the automotive parts supply chain information in one go, instead of dividing entity relationship extraction into two sub-tasks, entity extraction and relationship classification, as in traditional entity relationship extraction models and methods. The method of this invention can better focus on the potential connections between relations and entities, and is more suitable for extracting entity relationships from automotive parts supply chain information with multiple subjects and multiple overlapping triples, thereby improving the accuracy of entity relationship extraction in the automotive parts supply chain.

[0117] Combination Figure 2 As shown, this segment of automotive parts supply chain information contains nine triples. An entity may belong to multiple triples simultaneously, and some entities may be subjects in one triple but objects in another, resulting in a complex link structure. Existing entity relationship extraction methods cannot effectively identify the entities and the complex relationships between them in the automotive parts supply chain information.

[0118] To better illustrate the technical solution of the present invention, this embodiment is described in the following parts.

[0119] I. Word Information Embedding and Word Vector Embedding

[0120] In this embodiment, the processing steps for word information embedding and word vector embedding are as follows:

[0121] S201: The WordPiece method is used to segment the automotive parts supply chain information to obtain the one-hot vector representation of the sub-words of the automotive parts supply chain information.

[0122] S202: Input the one-hot vector representation of the automotive parts supply chain information into the pre-trained BERT model to embed word information and obtain the hidden features of the automotive parts supply chain information;

[0123] S203: Use the hidden features of the last layer of the BERT model as the initial representation of word nodes;

[0124] The formula is expressed as:

[0125]

[0126] In the formula: This represents the initial representation of the i-th word node; w i This represents the i-th word in the original automotive parts supply chain information; n represents the number of words in a segment of automotive parts supply chain information.

[0127] S204: High-dimensional word vector embedding is performed on relation types through a relation type embedding layer to obtain relation node representations;

[0128] The formula is expressed as:

[0129]

[0130] In the formula: This represents the initial representation of the j-th relation node; r j represents the predefined j-th relation type; E represents the relation type embedding layer, which is responsible for embedding each relation type into a high-dimensional vector; m = |R| represents the number of predefined relation types; W and b are trainable parameters.

[0131] This invention utilizes a pre-trained BERT model to embed words into automotive parts supply chain information. Based on the relationships existing in the current automotive parts supply chain information, predefined relationship types are defined, and high-dimensional word vectors are embedded into these relationship types. The hidden features output from the last layer of BERT are used as the initial representation of word nodes, and the high-dimensional relationship type word vectors are used as the initial representation of relationship nodes. This format of node representation provides good input for the model, thus improving the training effect and extraction accuracy.

[0132] II. Multilayer Heterogeneous Graph Neural Networks

[0133] In this embodiment, combined with Figure 3 As shown, the processing steps of a multilayer heterogeneous graph neural network are as follows:

[0134] S3011: Use the initial representations of word nodes and relation nodes as input to a multilayer heterogeneous graph neural network;

[0135] S3012: Iteratively fuse and update the representations of word nodes and relation nodes through a multi-layer heterogeneous graph neural network; after obtaining the new word node representation, the relation nodes are updated through the new word nodes, and at this time all word nodes are considered as neighbors of a relation node, and a new relation node representation is obtained through iterative fusion;

[0136] The formula is expressed as:

[0137]

[0138] In the formula: This represents the word node representation of the output of the (l+1)th layer heterogeneous graph neural network; This represents the relation node representation of the output of the (l+1)th layer heterogeneous graph neural network; HGNN represents a heterogeneous graph neural network. This represents the word node h in the l-th layer of the heterogeneous graph neural network. i The set of all neighbor relationship nodes; Represents the relation node r in the l-th layer heterogeneous graph neural network. j The set of all neighboring word nodes; This represents the word node representation of the output of the l-th layer heterogeneous graph neural network, and is also the input of the (l+1)-th layer heterogeneous graph neural network.

[0139] Specifically, the processing steps for each layer of the heterogeneous graph neural network are as follows:

[0140] 1) The initial representations of word nodes and relation nodes are converted into word node vectors and relation node vectors through a linear layer;

[0141] The formula is expressed as:

[0142] α i =W h h i +b i ;

[0143] β j =W r r j +b r ;

[0144] In the formula: α i β i W represents the word node vector and the relation node vector. hb h W r b r These are trainable parameters;

[0145] 2) Concatenate the word node vector and the relationship node vector and calculate the similarity. Then, calculate the attention weight between the word node and the relationship node based on the similarity.

[0146] γ ij =Linear(W a [α i :β j ]);

[0147]

[0148] In the formula: Represents word node h i and relation node r j Attention weights between; r ij Represents the word node vector α i and relation node vector β i Similarity between them; [:] indicates vector concatenation; Liner indicates a linear layer; f indicates softmax; W a Indicates the parameters that can be trained;

[0149] 3) Attention weights based on word nodes and relationship nodes The representation of word nodes is updated using a gating mechanism; to ensure that the dimension of each node remains unchanged and to maintain non-linearity, a gating mechanism is used instead of an activation function.

[0150] The formula is expressed as:

[0151]

[0152] In the formula: This represents the word node representation after processing by a heterogeneous graph neural network.

[0153] 4) Represent the word nodes after processing by the heterogeneous graph neural network. The word node representation is added to the input word node and used as the output word node representation of the heterogeneous graph neural network;

[0154] The formula is expressed as:

[0155]

[0156] III. Main Marker

[0157] In this embodiment, the processing steps of the main marker are as follows:

[0158] S3021: Map each word node in the updated word node representation to a space K with a two-dimensional output by linear mapping. The two dimensions of each word node in space K represent the probability of being the start position and the end position of the topic, respectively.

[0159] The formula is expressed as:

[0160] K = (σ(Linear(Dropout(X)))) 2 ;

[0161] In the formula: X∈R n×d This represents the updated word node representation; Dropout is used to prevent overfitting; σ represents the sigmoid activation function; Liner represents a linear layer;

[0162] S3022: Simplify the main body labeling task into a binary classification task, that is, determine whether each word node is the start position and end position of the main body: if the probability of the word node being the start position and end position of the main body is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0, thus obtaining the start position index and end position index of the main body (the dimension with 1 is used as the start position and end position index).

[0163] S3023: Find the actual subject S by using the start and end position indices of the subject.

[0164] IV. Main Feature Fusion Module

[0165] In this embodiment, the processing steps of the main feature fusion module are as follows:

[0166] S3031: Calculate the main features based on all word nodes between the start and end positions of the main body;

[0167] Combination Figure 4 As shown, the main features are calculated using the following formula:

[0168]

[0169] In the formula: s f Indicates the main features of the current training; h x and h y represents the updated word node representations of the current training subject at the x-th and y-th positions, respectively; b and e represent the start and end positions of the current training subject, respectively; a x Let x represent the attention weight of the x-th word node in the main body, where x,y∈(b,e); These are trainable parameters.

[0170] S3032: The main features are fused with each word node in the updated word node representation through a gating mechanism to generate a fused representation of the main features of the word node;

[0171] The formula is expressed as:

[0172]

[0173]

[0174] In the formula: The main feature fusion representation of the i-th word node; h i The i-th word node in the updated word node representation; gate represents the gating mechanism; s f Represents the main feature; σ represents the sigmoid activation function; W g b g The linear layer parameters of the gating mechanism are represented; cue_len represents the effective length of the information.

[0175] S3033: Add the fused representation of the main features of the word node to the updated word node representation to obtain the word node representation with fused main features;

[0176] The formula is expressed as:

[0177]

[0178] In the formula: This represents the i-th word node representing the fused main features; The main feature fusion representation of the i-th word node; h i This represents the updated representation of the i-th word node.

[0179] V. Relation-Object Pair Markers

[0180] In this embodiment, the processing steps for the relation-object pair marker are as follows:

[0181] S3041: The d-dimensional representation of each word node in the word node representation of the fused subject features is mapped from the d-dimensional representation to the (2×m×d)-dimensional representation through a linear layer, where m represents the number of predefined relation types; the value of the first dimension 2 represents the probability that each word node is the starting and ending position of the object, and the value of the second dimension m represents the probability that each word node is the position of the relation.

[0182] The formula is expressed as:

[0183] (r,o)=(σ(Linear(Dropout(X′)))) 4 ;

[0184] In the formula: (r,o) represents the starting position index, ending position index, and relation category index of the object in the relation-object pair; X′∈R n×d The word node representation represents the fused main features; σ represents the sigmoid activation function; Liner represents a linear layer;

[0185] S3042: Simplify the relation and object labeling task into a classification task, that is, determine whether each word node is the start and end position of an object, and whether it is one of the m relation types: if the probability of the word node being a relation position is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0; if the probability of the word node being the start and end position of an object is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0; the position with dimension 1 is used as the index of the relation category or the index of the start and end position of the object.

[0186] S3043: Obtain the relation object pair (R, O) using the relation category index and the object start and end position indexes.

[0187] Finally, a subject-relation-object triple is represented as (S, R, O), where S represents the subject, R represents the relation, and O represents the object.

[0188] VI. Loss Function

[0189] In this embodiment, the loss function for training the relation extraction model is as follows:

[0190]

[0191] In the formula: U is the set of all relations in the training set; I represents a labeled segment of automotive parts supply chain information; T represents the triples contained in this segment of information; S∈T represents the subject in the triple; T|S represents the set of all triples with S as the subject; (R,O)∈T|S represents the relation-object pair with S as the subject; U / T∈S represents all relations other than those already contained in the triple set. The empty relation-object pair is used to indicate that there is no corresponding relation or object; Indicates the probability of subject prediction. This represents the predicted probability of a pair of relational objects.

[0192] This invention improves the performance of entity relation extraction by optimizing the loss function during the training of the relation extraction model to maximize the joint probability of entity relation extraction for all sentences in the training set.

[0193] VII. Experimental Instructions

[0194] To better illustrate the advantages of the technical solution of the present invention, the following experiment is disclosed in this embodiment.

[0195] This experiment tested the proposed Automotive Parts Supply Chain Information Entity Relationship Extraction (AER-HGNN) method based on heterogeneous graph neural networks using actual automotive parts supply chain information.

[0196] 1. Dataset

[0197] Combination Figure 5 As shown, existing entity relationship extraction models are tested on these public datasets. Statistical results show that the information structure of these public datasets is too simple and the number of triples is small, which is very different from the actual automotive parts supply chain information.

[0198] We evaluated our model based on automotive parts supply chain information. This information encompasses the design, manufacturing, and operation phases of the automotive parts supply chain, covering numerous nodes such as manufacturers, dealers, logistics, and after-sales service. We defined 23 relationship types based on this information. Since the information in the automotive parts supply chain consists of overlapping triples, and some information contains many triples, we categorized the information into four types based on the number of triples: simple, general, relatively complex, and complex. Simple information contains 10 or fewer overlapping triples; general information contains 10-20 triples; relatively complex information contains 20-30 triples; and complex information contains more than 30 triples. The statistics for automotive parts data are shown in Tables 1 and 2.

[0199] Table 1 shows the dataset statistics. The information at each node is divided into training set, validation set, and test set.

[0200]

[0201]

[0202] Table 2. Statistics on the Quantity of Four Types of Information

[0203]

[0204] Previous tasks have separated entity extraction and relation classification, meaning a prediction is considered correct as long as both the predicted subject and object are correct, ignoring relation classification. Our proposed model extracts the subject-relation-object triplet in one step, eliminating the need for separate relation classification. Therefore, a prediction is considered correct only when every element in the subject-relation-object triplet is correct. We evaluate the model's performance using Precision, Recall, and F1 score, calculated as follows:

[0205]

[0206] Where TP is the number of samples correctly predicted as positive by the model, FP is the number of samples incorrectly predicted as positive by the model, and FN is the number of samples incorrectly predicted as negative by the model (i.e., positive samples missed by the model).

[0207] 2. Training details and parameter settings

[0208] For entity relationship extraction in the automotive parts supply chain, we built our model on top of a pre-trained BERT model, using its default configuration for word embedding. We used stochastic gradient descent (SGD) to train and optimize the model. The training batch size was 32, the maximum length of the automotive parts supply chain information (i.e., sentences) was 128, and the learning rate was set to 0.000001. The word embedding dimension was set to 768, and the number of layers in the heterogeneous graph neural network was set to 2. We performed a validation run after each epoch of training, and manually stopped training if there was no improvement on the validation set for 10 consecutive epochs. The remaining hyperparameters were determined based on the validation set.

[0209] Since automotive parts supply chain information has the characteristics of multiple entities, in order to avoid interference problems that may occur when multiple entities are trained at the same time, only a single entity is considered each time during the training process to prevent the model from learning incorrect features and improve the model training effect.

[0210] 3. Experimental Results

[0211] 1) Model test results

[0212] For the task of extracting entity relationships in the automotive parts supply chain, we evaluated our proposed AER-HGNN on four types of information: simple, general, relatively complex, and complex.

[0213] Table 3 shows the application of our method to automotive parts supply chain data, and the results obtained for extracting information entity relationships of different complexities.

[0214] Table 3 shows the test results of the model on information with different levels of complexity.

[0215]

[0216] The experimental results show that our model exhibits excellent performance for relatively simple automotive parts supply chain information. However, the model's performance gradually declines as the number of triples in the information increases. When the number of triples exceeds 30, the model's recall is significantly affected. In contrast, the AER-HGNN model maintains good performance when the number of triples is less than 30, especially when the number of triples is less than 20. According to the data in Table 2, most automotive parts supply chain information contains fewer than 20 overlapping triples. Therefore, our method can maintain high precision and recall when handling entity relation extraction tasks for such information, validating the effectiveness of our proposed method in current automotive parts supply chain information entity relation extraction tasks.

[0217] The experimental results show that our model exhibits excellent performance for relatively simple automotive parts supply chain information. However, the model's performance gradually declines as the number of triples in the information increases. When the number of triples exceeds 30, the model's recall is significantly affected. In contrast, the AER-HGNN model maintains good performance when the number of triples is less than 30, especially when the number of triples is less than 20. According to the data in Table 2, most automotive parts supply chain information contains fewer than 20 overlapping triples. Therefore, our method can maintain high precision and recall when handling entity relation extraction tasks for such information, validating the effectiveness of our proposed method in current automotive parts supply chain information entity relation extraction tasks.

[0218] 2) Ablation experiment

[0219] To verify the effectiveness of our proposed heterogeneous graph neural network for fusing entity relationships and subject feature fusion, we also tested two variants of AER-HGNN: AER-HGNN(NG) and AER-HGNN(NS). AER-HGNN(NG) does not use a heterogeneous graph neural network for iterative fusion of node representations, while AER-HGNN(NS) does not use subject feature fusion. To ensure a fair comparison, when validating AER-HGNN(NG), we used the hidden features processed by the last layer of BERT as input to word nodes; when validating AER-HGNN(NS), we directly used the updated nodes for relation-object pair extraction. Furthermore, we tested these two variants on four different complexities of automotive parts supply chain information. Figure 6 Table 4 shows the performance of AER-HGNN, AER-HGNN(NG), and AER-HGNN(NS) under four different complexity information, while Table 4 shows the performance comparison of these three models under several mixed complexity information.

[0220] Table 4 shows the performance of AER-HGNN, AER-HGNN(NG), and AER-HGNN(NS) under four different levels of information complexity.

[0221]

[0222] The results show that the model's performance significantly decreases when heterogeneous graph neural networks are not used for node updates, indicating that heterogeneous graph neural networks can comprehensively improve model performance. On the other hand, the model's prediction accuracy decreases when subject feature fusion is lacking, but the recall rate improves when processing complex information. This suggests that subject feature fusion not only improves prediction accuracy but also balances precision and recall to some extent. In conclusion, the combination of heterogeneous graph neural networks and subject feature fusion significantly enhances the model's performance in the task of extracting entity relationships in the automotive parts supply chain.

[0223] 3) Performance comparison of different heterogeneous graph neural network layer numbers

[0224] To determine the optimal number of layers in a heterogeneous graph neural network for extracting entity relationships from the automotive parts supply chain, we set the number of layers to l = {1, 2, 3, 4, 5, ...} and analyzed the trend of model performance as the number of layers changes. Figure 7 As shown, when the heterogeneous graph neural network has 2, 3, or 4 layers, the model performs well across all evaluation metrics. With increasing network layers, the recall rate significantly improves when processing complex types of information. However, precision decreases significantly. Furthermore, training time also increases significantly with increasing network layers. Considering the balance between training time cost and model performance, we ultimately chose l=2 as the optimal number of layers for the AER-HGNN heterogeneous graph neural network.

[0225] 4) Knowledge Graph Construction

[0226] We extracted a subset of automotive parts supply chain information triples from the AER-HGNN model and constructed a knowledge graph using SmartKG. Figure 8 The visualization results of this map are shown. Figure 8 The knowledge graph in the image displays triples corresponding to manufacturers and suppliers, with the graph at the center constructed based on this triple information. The question-and-answer function in the lower right corner allows users to input queries such as "Laizhou Yinheng Machinery Co., Ltd.", and the system returns the attribute information of that node and its adjacent nodes. This result demonstrates that the automotive parts supply chain triples extracted using our designed model can effectively support practical knowledge graph construction.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks, characterized in that, include: S1: Obtain the automotive parts supply chain information to be processed, as well as the predefined relationship types; S2: Perform word information embedding and word vector embedding on the automotive parts supply chain information and relationship types respectively to obtain the initial representations of word nodes and relationship nodes; S3: Input the initial representations of word nodes and relation nodes into the trained relation extraction model, and output several subject-relation-object triples; The processing steps when training the relation extraction model are as follows: S301: Iteratively fuse word nodes and relation nodes through a multi-layer heterogeneous graph neural network to update the word node representation, and use the word node representation output by the last layer of the heterogeneous graph neural network as the updated word node representation; S302: By marking the start and end positions of the subjects in the updated word node representation using the subject marker, several subjects are found; In step S302, the processing steps of the main marker are as follows: S3021: Map each word node in the updated word node representation to a space K with a two-dimensional output by linear mapping. The two dimensions of each word node in space K represent the probability of being the start position and the end position of the topic, respectively. The formula is expressed as: K=(σ(Linear(Dropout(X)))) 2 ; In the formula: X∈R n×d This represents the updated word node representation; Dropout is used to prevent overfitting; σ represents the sigmoid activation function; Liner represents a linear layer; S3022: Determine if each word node is the start and end position of the main body: If the probability of a word node being the start and end position of the main body is greater than the set threshold, its dimension is set to 1; otherwise, it is set to 0, thus obtaining the start and end position indexes of the main body. S3023: Find the actual subject S using the subject's start and end position indices; S303: Randomly select a subject, and fuse the features of the current subject with the updated word node representation through the subject feature fusion module to obtain the word node representation with fused subject features; In step S303, the processing steps of the main feature fusion module are as follows: S3031: Calculate the main features based on all word nodes between the start and end positions of the main body; S3032: The main features are fused with each word node in the updated word node representation through a gating mechanism to generate a fused representation of the main features of the word node; The formula is expressed as: In the formula: This represents the fused representation of the main features of the i-th word node; h i The i-th word node in the updated word node representation; gate represents the gating mechanism; s f Represents the main feature; σ represents the sigmoid activation function; W g b g The linear layer parameters of the gating mechanism are represented; cue_len represents the effective length of the information. S3033: Add the fused representation of the main features of the word node to the updated word node representation to obtain the word node representation with fused main features; The formula is expressed as: In the formula: This represents the i-th word node representing the fused main features; This represents the fused representation of the main features of the i-th word node; h i This represents the updated representation of the i-th word node; S304: By using the relation-object pair marker to mark the position of the relation and the start and end positions of the object in the word node representation of the fused subject features, find the relation-object pair corresponding to the subject; In step S304, the processing steps for the relation-object pair marker are as follows: S3041: The d-dimensional representation of each word node in the word node representation of the fused subject features is mapped from the d-dimensional representation to the (2×m×d)-dimensional representation through a linear layer, where m represents the number of predefined relation types; the value of the first dimension 2 represents the probability that each word node is the starting and ending position of the object, and the value of the second dimension m represents the probability that each word node is the position of the relation. The formula is expressed as: (r,o)=(σ(Linear(Droput(X′)))) 4 ; In the formula: (r,o) represents the starting position index, ending position index, and relation category index of the object in the relation-object pair; X′∈R n ×d The word node representation represents the fused main features; σ represents the sigmoid activation function; Liner represents a linear layer; S3042: Determine whether each word node is the start and end position of an object, and whether it is one of the m relation types: if the probability of the word node's relation position is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0; if the probability of the word node's start and end position of an object is greater than the set threshold, its dimension is set to 1, otherwise it is set to 0. S3043: Obtain the relation object pair (R, O) using the relation category index and the object start and end position indexes; S305: Combine the subject and the corresponding relation-object pair to generate a subject-relation-object triple; S306: Repeat S303 to S305 until all subjects are selected and all subject-relationship-object triples are extracted; S307: Calculate the loss function based on the extracted subject-relation-object triples and the pre-labeled real subject-relation-object triples, and back-optimize the parameters of the relation extraction model; S308: Repeat steps S301 to S307 to iteratively train the relation extraction model until the model converges or reaches the preset number of iterations; S4: Take all the subject-relationship-object triples output as the entity relation extraction results of the automotive parts supply chain information to be processed.

2. The method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks as described in claim 1, characterized in that: In step S2, the processing steps for word information embedding and word vector embedding are as follows: S201: Perform word segmentation on automotive parts supply chain information to obtain the corresponding one-hot vector representation; S202: Input the one-hot vector representation of the automotive parts supply chain information into the pre-trained BERT model to embed word information and obtain the hidden features of the automotive parts supply chain information; S203: Use the hidden features of the last layer of the BERT model as the initial representation of word nodes; The formula is expressed as: In the formula: w represents the initial representation of the i-th word node; i This represents the i-th word in the original automotive parts supply chain information; n represents the number of words in a segment of automotive parts supply chain information; S204: High-dimensional word vector embedding is performed on relation types through a relation type embedding layer to obtain relation node representations; The formula is expressed as: In the formula: This represents the initial representation of the j-th relation node; r j represents the predefined j-th relation type; E represents the relation type embedding layer; m represents the number of predefined relation types; W and b are trainable parameters.

3. The method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks as described in claim 1, characterized in that: In step S301, the processing steps of the multilayer heterogeneous graph neural network are as follows: S3011: Use the initial representations of word nodes and relation nodes as input to a multilayer heterogeneous graph neural network; S3012: Iteratively fuse and update the representations of word nodes and relation nodes through a multi-layer heterogeneous graph neural network; The formula is expressed as: In the formula: This represents the word node representation of the output of the (l+1)th layer heterogeneous graph neural network; This represents the relation node representation of the output of the (l+1)th layer heterogeneous graph neural network; HGNN represents a heterogeneous graph neural network. This represents the word node h in the l-th layer of the heterogeneous graph neural network. i The set of all neighbor relationship nodes; Represents the relation node r in the l-th layer heterogeneous graph neural network. j The set of all neighboring word nodes; This represents the word node representation of the output of the l-th layer heterogeneous graph neural network.

4. The method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks as described in claim 3, characterized in that: In step S3012, the processing steps for each layer of the heterogeneous graph neural network are as follows: 1) The initial representations of word nodes and relation nodes are converted into word node vectors and relation node vectors through a linear layer; The formula is expressed as: a i =W h h i +b h ; b j =W r r j +b r ; In the formula: α i β i W represents the word node vector and the relation node vector. h b h W r b r These are trainable parameters; 2) Concatenate the word node vector and the relationship node vector and calculate the similarity. Then, calculate the attention weight between the word node and the relationship node based on the similarity. c ij =Linear(W a [a i :b j ]); In the formula: Represents word node h i and relation node r j Attention weights between them; r ij Represents the word node vector α i and relation node vector β i Similarity between them; [:] indicates vector concatenation; Liner indicates a linear layer; f indicates softmax; W a Indicates the parameters that can be trained; 3) Attention weights based on word nodes and relationship nodes Update the representation of word nodes using a gating mechanism; The formula is expressed as: In the formula: This represents the word node representation after processing by a heterogeneous graph neural network. 4) Represent the word nodes after processing by the heterogeneous graph neural network. The word node is added to the input word node to form the output word node representation of the heterogeneous graph neural network; The formula is expressed as:

5. The method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks as described in claim 1, characterized in that: In step S3031, the main features are calculated using the following formula: In the formula: s f Indicates the main features of the current training; h x and h y represents the updated word node representations of the current training subject at the x-th and y-th positions, respectively; b and e represent the start and end positions of the current training subject, respectively; a x Let x represent the attention weight of the x-th word node in the main body, where x,y∈(b,e); These are trainable parameters.

6. The method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks as described in claim 1, characterized in that: A subject-relation-object triple is represented as (S, R, O), where S represents the subject, R represents the relation, and O represents the object.

7. The method for extracting entity relationships in the automotive parts supply chain based on heterogeneous graph neural networks as described in claim 1, characterized in that: In step S3, the loss function for training the relation extraction model is as follows: In the formula: U is the set of all relations in the training set; I represents a labeled segment of automotive parts supply chain information; T represents the triples contained in this segment of information; S∈T represents the subject in the triple; T|S represents the set of all triples with S as the subject; (R,O)∈T|S represents the relation-object pair with S as the subject; U / T|S represents all other relations besides those already contained in the triple set. The empty relation-object pair is used to indicate that there is no corresponding relation or object; Indicates the probability of subject prediction. This represents the predicted probability of a pair of relational objects.

Images