Anti-noise entity alignment method based on multi-modal industrial chain supply chain knowledge graph

Abstract

The invention is suitable for the technical field of knowledge maps, and provides an anti-noise entity alignment method based on a multi-modal industrial chain supply chain knowledge map. The method comprises the following steps: acquiring a plurality of pseudo tags; the similarity of the entity pairs is determined based on the entity features of the entity pairs of each pseudo label, if the similarity of the entity pairs is larger than a first preset threshold value, the pseudo labels corresponding to the entity pairs serve as clean labels, and if the similarity of the entity pairs is not larger than the first preset threshold value, the pseudo labels corresponding to the entity pairs serve as pseudo labels to be divided; and for each to-be-divided pseudo tag, determining the weight of the entity pair of the to-be-divided pseudo tag based on a preset weight adjustment function, taking the to-be-divided pseudo tag corresponding to the entity pair of which the weight is not less than a second preset threshold as a clean tag, and taking the to-be-divided pseudo tag corresponding to the entity pair of which the weight is less than the second preset threshold as a noise tag. According to the embodiment of the invention, clean labels and noise labels can be distinguished, so that the influence of noise is effectively reduced.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge graphs, and in particular to a noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph. Background Art

[0002] At present, the superposition of the "industrial chain + supply chain" provides a more accurate perspective for analyzing the industrial development trend. Starting from the structural logic of industrial development, it can accurately portray the industrial development trend at the factor level, location level, technology level, competition level, advantage level, etc., to make up for the previous shortcomings of insufficient overall grasp of the industrial development trend.

[0003] Although existing entity alignment methods based on multimodal industrial chain and supply chain knowledge graphs have made some progress in practical applications, there is still the problem that the noise influence in the training data is not fully suppressed, which makes the entity alignment model unable to distinguish between clean labels and noisy labels, resulting in a decrease in alignment performance. Summary of the Invention

[0004] In view of this, an embodiment of the present application provides a noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph to solve the problem that the entity alignment method of the multimodal industrial chain and supply chain knowledge graph in the prior art cannot distinguish between clean labels and noisy labels, thereby resulting in a decrease in alignment performance.

[0005] A first aspect of an embodiment of the present application provides a noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph, comprising:

[0006] Acquire multiple pseudo labels; wherein the pseudo labels are used to represent predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, the entity pairs include entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information;

[0007] Using a preset distance measurement method, the similarity of the entity pairs is determined based on the entity features of each pseudo-labeled entity pair. If the similarity of the entity pairs is greater than a first preset threshold, the pseudo-label corresponding to the entity pair is used as the clean label. If the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as the pseudo-label to be divided.

[0008] For each pseudo-label to be divided, the weight of the entity pair to be divided is determined based on the preset weight adjustment function, and the pseudo-label to be divided corresponding to the entity pair whose weight is not less than the second preset threshold is used as the clean label, and the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold is used as the noise label.

[0009] A second aspect of the embodiments of the present application provides a noise-resistant entity alignment device based on a multimodal industrial chain and supply chain knowledge graph, comprising:

[0010] An acquisition module is configured to acquire multiple pseudo labels, wherein the pseudo labels are used to represent predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, wherein the entity pairs include entities in the industry chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information;

[0011] A first determination module is configured to determine the similarity of entity pairs based on entity features of each pseudo-labeled entity pair using a preset distance measurement method; if the similarity of the entity pairs is greater than a first preset threshold, the pseudo-label corresponding to the entity pair is used as a clean label; if the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a pseudo-label to be divided;

[0012] The second determination module is used to determine the weight of the entity pair to be divided of the pseudo-label based on a preset weight adjustment function for each pseudo-label to be divided, and to use the pseudo-label to be divided corresponding to the entity pair whose weight is not less than the second preset threshold as the clean label, and to use the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold as the noise label.

[0013] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods of the first aspect when executing the computer program.

[0014] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any method of the first aspect are implemented.

[0015] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0016] The first aspect of the embodiment of the present application is a noise-resistant entity alignment method based on a multimodal industrial chain supply chain knowledge graph. The similarity of entity pairs can be determined based on the entity features of each pseudo-labeled entity pair, and the first label division is performed through a first preset threshold. That is, if the similarity of the entity pairs is greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a clean label; if the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a pseudo-label to be divided.

[0017] Then, the embodiment of the present application can determine the weights of the entity pairs to be divided into pseudo-labels based on a preset weight adjustment function, and then continue to divide the pseudo-labels to be divided according to the weights, that is, the pseudo-labels to be divided corresponding to the entity pairs whose weights are not less than the second preset threshold are used as clean labels, and the pseudo-labels to be divided corresponding to the entity pairs whose weights are less than the second preset threshold are used as noise labels.

[0018] At the same time, since the entity pairs in the embodiment of the present application include entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, the entity information includes at least two modal information, and the pseudo-label is used to represent the predicted aligned entity pairs obtained by learning unlabeled entities based on a preset entity alignment model, the embodiment of the present application can be applied to the noise-resistant entity alignment of the multimodal industrial chain supply chain knowledge graph, and can distinguish between clean labels and noisy labels, so that the weight of the noise label can be reduced and the weight of the clean label can be increased, thereby effectively reducing the impact of noise.

[0019] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flowchart of a noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph provided in an embodiment of the present application;

[0022] Figure 2 This is a flowchart of another noise-resistant entity alignment method based on a multimodal industrial chain supply chain knowledge graph provided in an embodiment of the present application;

[0023] Figure 3 This is a framework diagram of a noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a pseudo-label modal influence analysis provided by an embodiment of the present application;

[0025] Figure 5 This is a structural diagram of a noise-resistant entity alignment device based on a multimodal industrial chain and supply chain knowledge graph provided in an embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0028] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0029] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0030] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0031] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0033] The superposition of the "industrial chain + supply chain" provides a more accurate perspective for analyzing the industrial development trend. Starting from the structural logic of industrial development, it can accurately portray the industrial development trend at the factor level, location level, technology level, competition level, advantage level, etc., to make up for the previous shortcomings of insufficient overall grasp of the industrial development trend.

[0034] Knowledge engineering has been revived, characterized by the rapid growth of knowledge graphs. A knowledge graph (KG) is essentially a large-scale semantic network in which data is stored in the form of triples. A basic knowledge graph consists of entities and relationships between entities. Entities refer to things in the objective world, while relationships represent interactions between entities. Knowledge graphs have been widely used in knowledge-driven applications such as recommendation systems, information retrieval, and machine learning, but research on industrial chain and supply chain graphs is still rare. The "dual chain" knowledge graph system realizes the identification, extraction, fusion, reasoning, construction, and visualization of different "dual chain" full graph elements, providing a panoramic view of "dual chain" resilience risk analysis.

[0035] Since most existing KGs are represented by pure symbols in text form, this weakens the ability of machines to describe and understand the real world. Therefore, it is necessary to combine text entities with corresponding image, sound and video data. The multimodal knowledge graph (MMKG) integrates multiple modal information such as vision and text and establishes associations between different modal data, which can provide a richer and more comprehensive semantic representation. The multimodal "dual-chain" knowledge graph adopts a knowledge representation method based on representation learning to convert the existing knowledge in reality such as smart customs platforms, smart cities, domestic and foreign industries, and corporate data into content that can be recognized and processed by computers, and explicitly represents its concepts and relationships through symbols. Among them, the entities in the multimodal "dual-chain" knowledge graph include enterprises, transaction records, commodities, various trademarks, and pictures of commodities.

[0036] MMKGs are typically constructed from a single data source, but different MMKGs can provide information about an entity from different perspectives, improving the graph's coverage. Therefore, integrating MMKGs from different data sources is crucial, making the fusion of MMKGs from different data sources a key stage in MMKG construction. Multimodal entity alignment (MMEA), a key task in MMKG integration and construction, aims to match equivalent entities across different MMKGs using their relationships, attributes, literal, and image features to address the challenges of diverse naming conventions and multiple languages.

[0037] The most commonly used graph representation learning methods for entity alignment (EA) are translation (TransE) models and graph neural network (GNN) models. These models determine whether entities in different knowledge graphs are the same by obtaining entity embeddings and calculating the distance between them as the alignment basis. In entity representation learning, pre-aligned entity pairs are used to guide entity and relation representation learning and to spread entity embeddings. However, in practice, manually annotating aligned entity pairs is very expensive, resulting in a scarcity of data available for model training. Therefore, some EA methods introduce iterative learning strategies to generate semi-supervised data, mixing existing training data with newly generated semi-supervised data and retraining them in the next iteration. Furthermore, unsupervised methods based on entity name translation (e.g., product names, company names) and visual similarity (trademarks, product images) extract useful cues from cross-modal information about entities and contribute to training by generating pseudo-labels. However, both semi-supervised and unsupervised methods inevitably add noisy data, which reduces the robustness of model training and amplifies the impact of erroneous data during training.

[0038] In this context, reducing the generation of noisy data, mitigating the impact of existing noise data, and improving model robustness are key to improving the accuracy of entity alignment. A noise-resistant entity alignment method based on a multimodal "dual-chain" knowledge graph can help improve the accuracy of entity alignment when the number of manually labeled pre-aligned entity pairs is small. This aids the construction of industrial chain and supply chain knowledge graphs and provides a high-coverage, high-quality data foundation for exploring urban "dual-chain" resilience risks.

[0039] Although existing entity alignment methods based on multimodal industrial chain and supply chain knowledge graphs have made some progress in practical applications, these methods still have some shortcomings:

[0040] (1) Existing multimodal unsupervised methods such as EVA and Meaformer mainly rely on extracting visual feature similarity to generate pseudo-labels, ignoring the importance of other modal information of entities. These methods fail to fully explore and integrate the alignment clues hidden in cross-modal information, such as attribute names and attribute values.

[0041] (2) The influence of noise in the training data is not fully suppressed, and the model cannot distinguish between clean labels and noisy labels, resulting in a decrease in alignment performance.

[0042] The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph provided in this application aims to solve the above technical problems of the existing technology.

[0043] The following is a detailed description of the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems with specific embodiments. It should be noted that the following embodiments can refer to, draw on, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0044] See also Figure 1 As shown, the embodiment of the present application provides a flowchart of a noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph. The noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph includes: steps S101 to S103.

[0045] S101. Obtain multiple pseudo labels; wherein the pseudo labels are used to represent predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, the entity pairs include entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information.

[0046] The pseudo-labels of the embodiments of the present application can be implemented in an unsupervised experimental environment by learning side information of unlabeled entities (such as image features, entity names, etc.) and using entity pairs with maximized prediction probabilities as pseudo-labels, thereby realizing a supervised learning process.

[0047] Pseudo-labeling technology generally uses a model trained on labeled data to make predictions on unlabeled data, screens samples based on the prediction results, and then inputs them into the model again for training.

[0048] In some embodiments, the entity information includes visual modality information and attribute modality information; before obtaining multiple pseudo labels, the following steps are performed:

[0049] Input the industrial chain knowledge graph and the supply chain knowledge graph into the entity alignment model, extract the visual features of the entities with images in the industrial chain knowledge graph and the visual features of the entities with images in the supply chain knowledge graph, and obtain the attribute triples of the industrial chain knowledge graph and the attribute triples of the supply chain knowledge graph; the attribute triples are used to represent the correspondence between entities, attribute names and attribute values;

[0050] Based on the visual features of entities with images in the industry chain knowledge graph and the visual features of entities with images in the supply chain knowledge graph, two visual embedding matrices are obtained respectively. Similarity analysis is performed on the two visual embedding matrices to obtain a similarity score matrix of visual modalities.

[0051] Based on the two attribute triples, two attribute modal sets are constructed, and the two attribute modal sets are mapped to corresponding entities to obtain an index mapping matrix. Based on the two attribute modal sets, a similarity matrix of attribute names and a similarity matrix of attribute values ​​are determined. Based on the index mapping matrix, the similarity matrix of attribute names and the similarity matrix of attribute values ​​are mapped to corresponding entities to obtain an attribute modal similarity score matrix; wherein the attribute modal set is used to represent a set of attribute names and corresponding attribute values;

[0052] Perform weighted processing on the similarity score matrix of the visual modality and the similarity score matrix of the attribute modality to obtain a similarity score matrix to be processed, and perform similarity scaling processing on the similarity score matrix to obtain a target similarity score matrix;

[0053] Based on the target similarity score matrix, a preset number of entity pairs corresponding to similarity scores before arrangement are selected as pseudo labels in descending order of similarity scores.

[0054] The embodiment of the present application adopts a cross-modal label generation method to generate high-quality pseudo labels through visual images and attribute modal information, thereby reducing the generation of noise.

[0055] S102. Using a preset distance measurement method, determine the similarity of entity pairs based on the entity features of each pseudo-labeled entity pair; if the similarity of the entity pairs is greater than a first preset threshold, use the pseudo-label corresponding to the entity pair as a clean label; if the similarity of the entity pairs is not greater than the first preset threshold, use the pseudo-label corresponding to the entity pair as a pseudo-label to be divided.

[0056] Optionally, the preset distance measurement method may be to use a CSLS distance measurement to calculate the similarity of the entity pairs. The first preset threshold may be set according to actual application needs, and the first preset threshold may divide the pseudo labels into pseudo labels in a high confidence set and pseudo labels in a low confidence set.

[0057] Specifically, the pseudo labels in the high-confidence set are all clean labels, and the pseudo labels in the low-confidence set need to be further divided and processed.

[0058] In some embodiments, before determining the similarity of entity pairs based on entity features of entity pairs of pseudo-labels using a preset distance measurement method, the method further includes:

[0059] Based on the joint attention coefficient, the structural embedding of the entity is learned through the message propagation mechanism of the preset joint attention graph attention network to obtain the joint structural embedding feature of the entity. The joint structural embedding output feature of the entity is obtained by connecting the joint structural embedding features output by the joint attention graph attention network of all layers. The joint attention coefficient is used to represent the indicator of the interactive attention strength between two different modal information.

[0060] The entity's relational embedding vector is used as the relational attention vector coefficient. The entity's relational structural embedding is learned through a preset relational-aware graph attention network to obtain the entity's relational structural embedding feature. The entity's relational structural embedding output feature is obtained by connecting the relational structural embedding features output by the relational-aware graph attention network at all layers. The entity's relational embedding vector is obtained by initializing the entity's relational vector based on the relational triplet of the knowledge graph in which the entity is located. The relational triplet is used to represent the directed relationship between the entity and other entities.

[0061] The entity's joint structure embedding output feature and the entity's relationship-aware structure embedding output feature are concatenated to obtain the entity structure embedding vector as the entity feature.

[0062] The embodiments of the present application can effectively integrate cross-modal information for structural learning, dynamically generate entity-level weights for each modality, effectively utilize the cross-modal information of the entity, and obtain the entity structure embedding vector by splicing the entity's joint structure embedding output features and the entity's relationship-aware structure embedding output features, thereby more effectively capturing various relationships between entities.

[0063] S103. For each pseudo-label to be divided, determine the weight of the entity pair of the pseudo-label to be divided based on a preset weight adjustment function, and use the pseudo-label to be divided corresponding to the entity pair whose weight is not less than the second preset threshold as the clean label, and use the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold as the noise label.

[0064] Optionally, the weight adjustment function is based on the full negative sampling strategy, and the weight is determined by the difference between the similarity of the positive sample pair and the average similarity of all negative sample pairs; the positive sample pair is an entity pair to be divided into pseudo-labels, and the negative sample pair is an entity pair consisting of one entity of the entity pair to be divided into pseudo-labels and the negative entity corresponding to the other entity.

[0065] Optionally, the second preset threshold can be set according to actual applications, such as a natural number 1.

[0066] The embodiment of the present application is based on a noise-resistant entity alignment method of a multimodal industrial chain and supply chain knowledge graph. It can determine the similarity of entity pairs based on the entity features of the entity pairs of each pseudo-label, and perform the first label division through a first preset threshold. That is, if the similarity of the entity pairs is greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a clean label; if the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a pseudo-label to be divided.

[0067] Then, the embodiment of the present application can determine the weights of the entity pairs to be divided into pseudo-labels based on a preset weight adjustment function, and then continue to divide the pseudo-labels to be divided according to the weights, that is, the pseudo-labels to be divided corresponding to the entity pairs whose weights are not less than the second preset threshold are used as clean labels, and the pseudo-labels to be divided corresponding to the entity pairs whose weights are less than the second preset threshold are used as noise labels.

[0068] At the same time, since the entity pairs in the embodiment of the present application include entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, the entity information includes at least two modal information, and the pseudo-label is used to represent the predicted aligned entity pairs obtained by learning unlabeled entities based on a preset entity alignment model, the embodiment of the present application can be applied to the noise-resistant entity alignment of the multimodal industrial chain supply chain knowledge graph, and can distinguish between clean labels and noisy labels, so that the weight of the noise label can be reduced and the weight of the clean label can be increased, thereby effectively reducing the impact of noise.

[0069] The embodiment of the present application addresses some defects in the existing field of multimodal entity alignment and proposes an entity alignment method based on a multimodal industrial chain and supply chain knowledge graph, which can improve the efficiency and reliability in entity alignment tasks.

[0070] See also Figure 2 As shown, the embodiment of the present application provides a flowchart of another noise-resistant entity alignment method based on a multimodal industrial chain supply chain knowledge graph. Figure 3 As shown, the embodiment of the present application provides a framework diagram of a noise-resistant entity alignment method based on a multimodal industrial chain supply chain knowledge graph. Figure 2 The noise-resistant entity alignment method based on the multimodal industrial chain supply chain knowledge graph shown is applicable to Figure 3 The framework diagram shown.

[0071] Combine Figure 2 and Figure 3As shown, the embodiment of the present application proposes a new unsupervised multimodal entity alignment model NREA, which includes a cross-modal pseudo-label generation module, a joint attention guidance graph structure learning module and a noise self-induction weighting module. The cross-modal pseudo-label generation module can implement the contents of steps S201 to S205 of the embodiment of the present application, the joint attention guidance graph structure learning module can implement the contents of steps S207 to S209 of the embodiment of the present application, and the noise self-induction weighting module can implement the contents of steps S206 and S210 to S211 of the embodiment of the present application. NREA can effectively integrate entity cross-modal information and improve the quality of pseudo-label generation; through noise self-induction weighting, the weight of the noisy label is reduced while the weight of the clean label is increased, thereby effectively improving the model's noise resistance.

[0072] See also Figure 2 As shown, the noise-resistant entity alignment method based on the multimodal industrial chain supply chain knowledge graph includes: steps S201 to S211.

[0073] S201. Input the industrial chain knowledge graph and the supply chain knowledge graph into the entity alignment model, extract the visual features of the entities with images in the industrial chain knowledge graph and the visual features of the entities with images in the supply chain knowledge graph, and obtain the attribute triples of the industrial chain knowledge graph and the attribute triples of the supply chain knowledge graph; the attribute triples are used to represent the correspondence between entities, attribute names and attribute values.

[0074] Since each modality of MMKG provides different entity information, NREA generates high-quality pseudo labels by combining visual image and attribute modality information to reduce noise. In the visual modality, we use SwinTransformer to extract visual features from images.

[0075] S202. Based on the visual features of entities with images in the industrial chain knowledge graph and the visual features of entities with images in the supply chain knowledge graph, two visual embedding matrices are obtained respectively, and similarity analysis is performed on the two visual embedding matrices to obtain a similarity score matrix of the visual modality.

[0076] Through and After extracting visual features from entities with images, two visual embedding matrices can be obtained and Similarity score matrix M of visual modality vis The calculation formula is:

[0077]

[0078] in, Norm means minimum-maximum normalization. Note that if an entity has no corresponding image, then M vis The similarity of the corresponding rows and columns is 0.

[0079] S203. Construct two attribute modal sets based on the two attribute triples, map the two attribute modal sets to corresponding entities to obtain an index mapping matrix, determine the attribute name similarity matrix and the attribute value similarity matrix based on the two attribute modal sets, map the attribute name similarity matrix and the attribute value similarity matrix to the corresponding entities based on the index mapping matrix to obtain the attribute modal similarity score matrix; the attribute modal set is used to represent a set of attribute names and corresponding attribute values.

[0080] In some embodiments, determining a similarity matrix of attribute names and a similarity matrix of attribute values ​​based on two attribute modality sets includes:

[0081] For each attribute modal set, the attribute names and attribute values ​​in the attribute modal set are concatenated and input into the pre-trained model of the entity alignment model to obtain the attribute name embedding matrix. Based on the two attribute name embedding matrices, the attribute name similarity matrix is ​​determined.

[0082] When the pre-trained model determines that two attribute names are the same, the similarity of the attribute values ​​is determined based on the inverse of the difference between the two attribute values ​​corresponding to the two attribute names, and the similarity matrix of the attribute values ​​is determined based on the similarities of the attribute values.

[0083] In the attribute modality information, the similarity between attribute name and attribute value is used to obtain the cross-MMKG similarity matrix of attribute modality. Specifically, we use two MMKG attribute triples Construct two attribute modal sets Entity to The index mapping matrix W in ind . Then, by adding the attribute name and attribute values The concatenation is input as token into the RoBert pre-trained model to obtain attribute name embedding. and The embedding matrices based on attribute names are and The similarity matrix of attribute names is in Regarding the similarity of attribute values, it is believed that for an entity pair, if the attribute names are the same, the smaller the difference between the attribute values, the more similar the attribute values ​​are.

[0084] Therefore, the similarity matrix of attribute values ​​is where ci,j ∈M val , Similarity score matrix M of attribute modality attr for:

[0085]

[0086] in ⊙ represents the Harmand product.

[0087] S204 , performing weighted processing on the similarity score matrix of the visual modality and the similarity score matrix of the attribute modality to obtain a similarity score matrix to be processed, and performing similarity scaling processing on the similarity score matrix to be processed to obtain a target similarity score matrix.

[0088] In the embodiment of the present application, the attribute modality similarity score matrix M is obtained. attr and the visual modality similarity score matrix M vis Finally, the Sinkhorn operator is introduced to scale the similarity. The Sinkhorn operator is an iterative algorithm for solving the optimal transfer problem. The core idea is to approximate the optimal transfer solution by alternately updating the scaling factors of rows and columns. The final target similarity score matrix M final The calculation formula is:

[0089] M final =Sinkhorn(M vis +βM attr ) (3)

[0090] Among them, β is a balancing factor that reflects the importance of similarity between different modalities.

[0091] S205 : Based on the target similarity score matrix, select entity pairs corresponding to a preset number of similarity scores before arrangement as pseudo labels in descending order of similarity scores.

[0092] Specifically, the target similarity score matrix M final The similarity matrix is ​​sorted from high to low according to the score and the top k are taken as pseudo labels.

[0093] S206. Obtain multiple pseudo labels; the pseudo labels are used to represent the predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, the entity pairs include entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information.

[0094] The principle of step S206 in the embodiment of the present application is consistent with that of step S101 and will not be repeated here.

[0095] S207. Based on the joint attention coefficient, the structural embedding of the entity is learned through the message propagation mechanism of the preset joint attention graph attention network to obtain the joint structural embedding feature of the entity, and the joint structural embedding output feature of the entity is obtained by connecting the joint structural embedding features output by the joint attention graph attention network of all layers; the joint attention coefficient is used to represent an indicator of the strength of interactive attention between two different modal information.

[0096] In some embodiments, the joint attention coefficient is obtained as follows:

[0097] For each modal information of an entity, an entity-level weight is generated through a preset self-attention learning mechanism;

[0098] The entity-level weights of each modal information are fused with the embedding vector of the entity corresponding to the modal information to obtain the joint embedding vector of the entity; wherein the embedding vector of the entity is used to represent the structural embedding information of the entity;

[0099] The joint embedding vectors of two adjacent entities are merged and normalized to obtain the joint attention coefficient.

[0100] In order to effectively integrate cross-modal information for structural learning, entity-level weights are dynamically generated for each modality. m Through the transformation matrix W Q ,W K ,W V Projection to modality-aware query Q m ,key K m ,value V m The self-attention coefficient ξ between the modalities m and j of the entity mj The calculation formula is as follows:

[0101]

[0102] in, Represents a set of modalities, a is the attribute modality, and v is the visual modality. The entity-level weight calculation formula for each modality m is:

[0103]

[0104] Get entity-level weight w m Afterwards, the joint embedding h of the entity u Expressed as:

[0105]

[0106] in, Represents entity e iThe traditional GAT represents h by connecting nodes i and h j To learn relational attention weights. The embedding vector of the corresponding entity, The joint embedding vector of the corresponding entities.

[0107] In order to effectively utilize the cross-modal information of entities, we use the joint embedding h u To guide the learning of the joint attention coefficient γ, it can be calculated as follows:

[0108]

[0109] Among them, σ represents the activation function; Represents the connection entity e i and e j The joint embedding of the edges; represents the learnable joint attention weight vector; Represents entity e i The set of one-hop neighbor nodes of . It is worth noting that Normalization is required to ensure After obtaining the joint attention coefficient, the joint structural embedding feature of the entity is calculated as follows:

[0110]

[0111] in, Represents the transformation matrix, which is obtained through the joint-guided relational reflection operation. i The joint structure embedding output features is obtained by concatenating the embeddings from all layers and is calculated as:

[0112]

[0113] S208. Use the relational embedding vector of the entity as the relational attention vector coefficient, learn the relational-aware structural embedding of the entity through a preset relational-aware graph attention network, and obtain the relational-aware structural embedding feature of the entity. Then, obtain the relational-aware structural embedding output feature of the entity by connecting the relational-aware structural embedding features output by the relational-aware graph attention network of all layers. The relational embedding vector of the entity is obtained by initializing the relational vector of the entity based on the relational triple of the knowledge graph where the entity is located, and the relational triple is used to represent the directed relationship between the entity and other entities.

[0114] In order to more effectively capture the various relationships between entities, following the relationship-aware EA method, an anisotropic relationship attention mechanism is used to aggregate the neighbor information around the entity. iThe relation-aware structure embedding at layer l is determined by the following equation:

[0115]

[0116] in, Represents the relation attention vector; Relation embedding vectors representing entities; Represents entity e i and e j The set of relationships between Represents the relationship r k The relational reflection matrix of entity e i Relation-aware structural embedding output features It is obtained by concatenating the embeddings from all layers and is calculated as:

[0117]

[0118] S209: Concatenate the entity's joint structure embedding output feature and the entity's relationship-aware structure embedding output feature to obtain an entity structure embedding vector as the entity feature.

[0119] Finally, the multi-view structural embedding of the entity is concatenated with the relation embedding to obtain the final entity structural embedding vector Expressed as:

[0120]

[0121] in Represents entity e i The collection of all associated relationships.

[0122] S210. Using a preset distance measurement method, determine the similarity of entity pairs based on the entity features of the entity pairs of each pseudo-label. If the similarity of the entity pairs is greater than a first preset threshold, use the pseudo-label corresponding to the entity pair as a clean label. If the similarity of the entity pairs is not greater than the first preset threshold, use the pseudo-label corresponding to the entity pair as a pseudo-label to be divided.

[0123] The principle of step S210 in the embodiment of the present application is consistent with that of step S102 and will not be repeated here.

[0124] For ease of reading, the final output entity structure embedding vector of the entity In the following text, we use entity e i express.

[0125] The goal of NREA is to minimize the L2 distance of each correct entity pair. The GNN-based method maps the embeddings of two multimodal knowledge graphs into the same space and uses pairwise loss to optimize the distance between entities. The specific loss function formula is as follows:

[0126]

[0127] Where θ represents the fixed margin, [x] + Represents the maximum operator Max(0,x), x ′ represents the negative sample of x, e i ′ and e j ′ Respectively represent e i and e j the corresponding negative entity; Represents the set of pre-aligned entity pairs. We follow the full negative sampling strategy and use LogSumExp to identify hard-to-negative samples. The preset initial alignment loss function is defined as:

[0128]

[0129] Among them, τ and ρ are hyperparameters, and ln represents the loss normalization operation. However, in semi-supervised and unsupervised learning, adding to the pre-aligned entity pair set The data used for training often contains a significant amount of noise. Noisy labels can propagate misinformation in the neural network, weakening the effectiveness of full-negative sampling strategies and thus affecting the model's alignment performance. To address this challenge, inspired by HASN, we propose a dynamic weight adjustment function λ to distinguish between clean and noisy labels, effectively suppressing the impact of noisy labels.

[0130] Specifically, the CSLS distance metric is first used to calculate the similarity of entity pairs, and then the entity pairs whose confidence (similarity) exceeds the threshold η (i.e., the first preset threshold) are divided into the high confidence set , as clean labels. The remaining entity pairs are divided into low confidence sets As the label to be divided. In order to improve the alignment performance of pseudo labels in the high confidence set, the cross entropy loss function is used, and the calculation formula is:

[0131]

[0132] where q e Represents the one-hot label vector of entity e, p e The embedding expression similarity between two entities.

[0133] S211. For each pseudo-label to be divided, determine the weight of the entity pair of the pseudo-label to be divided based on a preset weight adjustment function, and use the pseudo-label to be divided corresponding to the entity pair whose weight is not less than the second preset threshold as a clean label, and use the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold as a noise label.

[0134] The principles of step S211 in the embodiment of the present application are consistent with those of step S203 and will not be repeated here.

[0135] In some embodiments, the weight adjustment function is:

[0136]

[0137] Among them, e i and e j Corresponding to the two entities of the entity pair, e j ′ Indicates e j The corresponding negative entity, e i ∈ε1,e j ∈ε2, ε1 and ε2 are the entity sets of the industrial chain knowledge graph and the entity sets of the supply chain knowledge graph, respectively. sim(·) represents the cosine similarity operation. The hyperparameters μ and z are preset focusing factors that jointly determine the noise label weight reduction rate. exp represents an exponential function with the natural constant e as the base.

[0138] Specifically, when sim(·) is operated, it is for e i and e j The corresponding entity feature is the entity structure embedding vector.

[0139] In the embodiment of the present application, for the pseudo labels (entity pairs) to be divided in the low confidence set, a weight adjustment function is formed based on the cosine similarity between the entity pairs.

[0140] The properties of the dynamic weight adjustment function λ are analyzed as follows:

[0141] (1) The focusing factor μ ranges from [-0.8, -0.4] and is responsible for distinguishing whether the pseudo-label is a clean label or a noise label. For example, The pseudo labels of are regarded as noise labels, and is considered a clean label.

[0142] (2) The range of the focusing factor z is [2, 6], which is responsible for controlling the weighting ratio of the noise label and the clean label. Taking the noise sample as an example, if When the values ​​of the hyperparameters z and μ are 5 and -0.6 respectively, the output weight of the weight adjustment function is exp(-0.6+0.6-0.3) 5 =0.2231.

[0143] In some embodiments, after taking the pseudo labels to be divided corresponding to the entity pairs whose weights are not less than the second preset threshold as clean labels and taking the pseudo labels to be divided corresponding to the entity pairs whose weights are less than the second preset threshold as noise labels, the method further includes:

[0144] Based on a preset weight adjustment function and a preset initial alignment loss function, an adjusted alignment loss function is determined; the adjusted alignment loss function and the preset cross entropy loss function are added to obtain a target loss function; based on the target loss function, the parameters of the entity alignment model are updated.

[0145] Based on the weight adjustment function λ, the adjusted alignment loss function is defined as:

[0146]

[0147] The objective loss function is defined as:

[0148]

[0149] And minimize the loss by backpropagation to update the parameters in the solid alignment model.

[0150] The embodiments of the present application use a cross-modal pseudo-label generation module to obtain high-quality pseudo-labels, thereby reducing the generation of noise; through a joint attention-guided graph structure learning module, cross-modal information is effectively integrated to generate embedded expressions of entities; through a noise self-perception weighting module, clean labels and noisy labels are distinguished by confidence, while the weight of the noisy labels is reduced and the weight of the clean labels is increased to effectively reduce the impact of noise.

[0151] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0152] In order to verify the alignment effect of the noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph in the embodiment of the present application, the present application further conducted corresponding experimental tests.

[0153] (1) Experimental design

[0154] a. Dataset and Experimental Settings

[0155] This paper selects two types of multimodal knowledge graph datasets to verify the effectiveness of the model: 1) the monolingual dataset MMKG. FB15K-DB15K and FB15K-YAGO15K were selected from MMKG as benchmark datasets for the MMEA task; 2) the bilingual dataset DBP15K. DBP15K includes three datasets from the multilingual version of DBpedia, namely DBP15K ZH-EN 、DBP15K JA-EN and DBP15K FR-ENIn the supervised experiments, 20% of the pre-aligned entity pairs are used for training on the monolingual dataset MMKG, while 30% of the pre-aligned entity pairs are used for training on the bilingual dataset, and the rest are used as the test set. Some statistics of the dataset are shown in Table 1:

[0156] Table 1 Dataset description

[0157]

[0158]

[0159] The model is based on PyTorch, and the pre-trained language model Roberta is downloaded from Sentence-Transformers. For all datasets, the number of GCN layers is set to 2, and the hidden layer dimension is set to 128.

[0160] Semi-supervised Setting: The model's alignment performance was evaluated in a semi-supervised setting on all datasets. A bidirectional iterative training strategy was used to add new pseudo-labels during training. Specifically, every five epochs, cross-KG entity pairs with their nearest neighbors in the vector space were incorporated into the training data. Unsupervised Setting: The model's alignment performance was evaluated in an unsupervised setting on the MMKG dataset, with the number of pseudo-labels k set to FB15K-DB15K (k=5700) and FB15K-YAGO15K (k=5900).

[0161] Baseline Methods: NREA is compared with classic multimodal entity alignment methods.

[0162] b. Evaluation indicators

[0163] In the field of named entity alignment, Hits@N and MRR (Mean Reciprocal Ranking) are generally used as evaluation indicators.

[0164]

[0165] Among them, rank i Represents the score ranking position of the correct mapping of the i-th query entity if and only if rank i ≤N In other cases Represents the test dataset; in particular, Hits@1 represents the accuracy. Hits@1 and MRR are used to evaluate the alignment performance of the model.

[0166] (2) Experimental results

[0167] a. Semi-supervised experiments

[0168] Table 2 Experimental results of NREA on monolingual datasets.

[0169]

[0170] Table 3 Experimental results of NREA on bilingual datasets.

[0171]

[0172] Tables 2 and 3 list the results of NREA(-) on all datasets in the semi-supervised setting. It is worth noting that NREA(-) does not use the cross-modal pseudo-label generation module to ensure fair experimentation. The experimental results show that the method achieves state-of-the-art performance on both datasets. On monolingual datasets, NREA(-) outperforms the best baseline by 4.5% to 6% in the Hits@1 metric and by 3.8% to 4.6% in the MRR metric.

[0173] b. Unsupervised Experiments

[0174] (3) Table 4 Unsupervised experimental results of NREA on monolingual datasets.

[0175]

[0176] To the best of our knowledge, NREA is the first unsupervised method to attempt to integrate attribute and visual modalities to generate pseudo-labels on the monolingual MMKG dataset. To maintain fairness in unsupervised experiments, the same visual embedding representation is used in the pseudo-label generation process. As shown in Table 4, after effectively integrating attribute modalities, NREA outperforms the best baseline by 33.6% to 40.6% in Hits@1 and 29.9% to 34.9% in MRR, compared to methods using only visual information. This is primarily attributed to the high-quality pseudo-labels provided by the cross-modal pseudo-label generation module by integrating cross-modal information, and the noise suppression capability of the noise self-sensing weighting module.

[0177] c. Pseudo-label analysis experiment

[0178] See also Figure 4 As shown, the embodiment of the present application provides a schematic diagram of pseudo-label modal impact analysis. Figure 4 As shown in , in unsupervised experiments, the performance of the model depends largely on the quantity and quality of pseudo labels. The accuracy of pseudo labels generated by fusing cross-modal information and unimodal information on FB15K-DB15K was evaluated. Figure 1As shown in the figure, we can observe that: 1) As the number of pseudo-labels increases, the accuracy of the three methods, Vis., Attr., and Vis.+Attr., gradually decreases, with Vis. decreasing the fastest. 2) Compared with visual information, attribute information generates more accurate pseudo-labels in this dataset. 3) Compared with using only single-modal information, the pseudo-labels generated by cross-modal information fusion are more accurate and stable, demonstrating the superiority of the proposed method. The experimental results of FB15K-YAGO15K are similar to those of FB15K-DB15K.

[0179] See also Figure 5 As shown, the embodiment of the present application provides a structural diagram of a noise-resistant entity alignment device based on a multimodal industrial chain supply chain knowledge graph. Figure 5 As shown, the noise-resistant entity alignment device 50 based on the multimodal industrial chain supply chain knowledge graph includes: an acquisition module 501, a first determination module 502 and a second determination module 503.

[0180] The acquisition module 501 is used to obtain multiple pseudo labels; wherein the pseudo labels are used to represent predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, the entity pairs include entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information;

[0181] The first determination module 502 is configured to determine the similarity of the entity pairs based on the entity features of each pseudo-labeled entity pair using a preset distance measurement method. If the similarity of the entity pairs is greater than a first preset threshold, the pseudo-label corresponding to the entity pair is used as a clean label. If the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a pseudo-label to be divided.

[0182] The second determining module 503 is configured to determine, for each pseudo-label to be divided, a weight of an entity pair of the pseudo-label to be divided based on a preset weight adjustment function, and to use the pseudo-label to be divided corresponding to the entity pair whose weight is not less than a second preset threshold as a clean label, and to use the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold as a noise label;

[0183] Among them, the weight adjustment function is based on the full negative sampling strategy, and the weight is determined by the difference between the similarity of the positive sample pair and the average similarity of all negative sample pairs; the positive sample pair is the entity pair to be divided into pseudo-labels, and the negative sample pair is the entity pair consisting of one entity of the entity pair to be divided into pseudo-labels and the negative entity corresponding to the other entity.

[0184] Optionally, the acquisition module 501, the first determination module 502 and the second determination module 503 can constitute the noise self-induction weighting module of the embodiment of the present application, and the noise-resistant entity alignment device 50 based on the multimodal industrial chain supply chain knowledge graph also includes a cross-modal pseudo-label generation module and a joint attention guidance graph structure learning module.

[0185] Optionally, a cross-modal pseudo-label generation module is used to input the industrial chain knowledge graph and the supply chain knowledge graph into the entity alignment model, extract the visual features of the entities with images in the industrial chain knowledge graph and the visual features of the entities with images in the supply chain knowledge graph, and obtain the attribute triples of the industrial chain knowledge graph and the attribute triples of the supply chain knowledge graph; the attribute triples are used to represent the correspondence between entities, attribute names and attribute values; based on the visual features of the entities with images in the industrial chain knowledge graph and the visual features of the entities with images in the supply chain knowledge graph, two visual embedding matrices are obtained respectively, and similarity analysis is performed on the two visual embedding matrices to obtain a similarity score matrix of the visual modality; two attribute modal sets are correspondingly constructed based on the two attribute triples, and the two attribute modal sets are combined. Map to the corresponding entity to obtain an index mapping matrix, determine the similarity matrix of the attribute name and the similarity matrix of the attribute value based on the two attribute modal sets, map the similarity matrix of the attribute name and the similarity matrix of the attribute value to the corresponding entity based on the index mapping matrix, and obtain the similarity score matrix of the attribute modality; the attribute modal set is used to represent the set of attribute names and corresponding attribute values; perform weighted processing on the similarity score matrix of the visual modality and the similarity score matrix of the attribute modality to obtain the similarity score matrix to be processed, perform similarity scaling processing on the similarity score matrix to be processed, and obtain the target similarity score matrix; based on the target similarity score matrix, select the entity pairs corresponding to the preset number of similarity scores before arrangement as pseudo labels in descending order of similarity scores.

[0186] Optionally, the joint attention guided graph structure learning module is used to learn the structural embedding of the entity based on the joint attention coefficient through the message propagation mechanism of the preset joint attention graph attention network to obtain the joint structural embedding feature of the entity, and obtain the joint structural embedding output feature of the entity by connecting the joint structural embedding features output by the joint attention graph attention network of all layers; the joint attention coefficient is used to represent an indicator of the intensity of interactive attention between two different modal information; the relationship embedding vector of the entity is used as the relationship attention vector coefficient, and the relationship-aware structural embedding of the entity is learned through the preset relationship-aware graph attention network to obtain the relationship-aware structural embedding feature of the entity, and obtain the relationship-aware structural embedding output feature of the entity by connecting the relationship-aware structural embedding features output by the relationship-aware graph attention network of all layers; the relationship embedding vector of the entity is obtained by initializing the relationship vector of the entity based on the relationship triple of the knowledge graph where the entity is located, and the relationship triple is used to represent the directed relationship between the entity and other entities; the joint structural embedding output feature of the entity and the relationship-aware structural embedding output feature of the entity are spliced ​​to obtain the entity structure embedding vector as the entity feature.

[0187] Optionally, the second determination module 503 is used to determine an adjusted alignment loss function based on a preset weight adjustment function and a preset initial alignment loss function; add the adjusted alignment loss function and the preset cross entropy loss function to obtain a target loss function; and update the parameters of the entity alignment model based on the target loss function.

[0188] The device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, and will not be repeated here.

[0189] An embodiment of the present application also provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and is characterized in that the processor implements the steps of the method of the embodiment of the present application when executing the computer program.

[0190] See also Figure 6 As shown in FIG, a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6Only one is shown in the figure) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on at least one processor 60. When the processor 60 executes the computer program 62, the steps of any of the above method embodiments are implemented.

[0191] The electronic device 6 can be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that Figure 6 This is merely an example of the electronic device 6 and does not constitute a limitation on the electronic device 6 . The electronic device 6 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 6 may also include input and output devices, network access devices, etc.

[0192] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0193] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. In other embodiments, the memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 6. Furthermore, the memory 61 may include both an internal storage unit of the electronic device 6 and an external storage device. The memory 61 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of a computer program. The memory 61 may also be used to temporarily store data that has been output or is about to be output.

[0194] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0196] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0197] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned various method embodiments when executing the computer program product.

[0198] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include at least: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0199] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0200] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0201] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the above modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0202] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A noise-resistant entity alignment method based on a multimodal industrial chain and supply chain knowledge graph, characterized by: include: Acquire multiple pseudo labels; wherein the pseudo labels are used to represent predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, the entity pairs including entities in the industrial chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information; Using a preset distance measurement method, the similarity of the entity pairs is determined based on the entity features of each entity pair of the pseudo-labels. If the similarity of the entity pairs is greater than a first preset threshold, the pseudo-label corresponding to the entity pair is used as a clean label. If the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a pseudo-label to be divided. For each of the pseudo-labels to be divided, the weight of the entity pair of the pseudo-label to be divided is determined based on a preset weight adjustment function, and the pseudo-label to be divided corresponding to the entity pair whose weight is not less than a second preset threshold is used as a clean label, and the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold is used as a noise label.

2. The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph according to claim 1 is characterized in that: The weight adjustment function is: Among them, e i and e j Corresponding to the two entities of the entity pair, e j ' represents e j The corresponding negative entity, e i ∈ε1,e j ∈ε2, ε1 and ε2 are the entity sets of the industrial chain knowledge graph and the entity sets of the supply chain knowledge graph, respectively. sim(·) represents the cosine similarity operation. The hyperparameters μ and z are preset focusing factors that jointly determine the noise label weight reduction rate. exp represents an exponential function with the natural constant e as the base.

3. The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph according to claim 1 is characterized in that: The entity information includes visual modality information and attribute modality information; before obtaining multiple pseudo labels, it includes: Inputting the industrial chain knowledge graph and the supply chain knowledge graph into the entity alignment model, extracting visual features of entities with images in the industrial chain knowledge graph and visual features of entities with images in the supply chain knowledge graph, and obtaining attribute triples of the industrial chain knowledge graph and attribute triples of the supply chain knowledge graph; wherein the attribute triples are used to represent the correspondence between entities, attribute names, and attribute values; Based on the visual features of the entities with images in the industrial chain knowledge graph and the visual features of the entities with images in the supply chain knowledge graph, two visual embedding matrices are obtained respectively, and similarity analysis is performed on the two visual embedding matrices to obtain a similarity score matrix of visual modalities; Constructing two attribute modal sets based on the two attribute triples, mapping the two attribute modal sets to corresponding entities to obtain an index mapping matrix, determining an attribute name similarity matrix and an attribute value similarity matrix based on the two attribute modal sets, and mapping the attribute name similarity matrix and the attribute value similarity matrix to corresponding entities based on the index mapping matrix to obtain an attribute modal similarity score matrix; wherein the attribute modal set is used to represent a set of attribute names and corresponding attribute values; performing weighted processing on the similarity score matrix of the visual modality and the similarity score matrix of the attribute modality to obtain a similarity score matrix to be processed, and performing similarity scaling processing on the similarity score matrix to be processed to obtain a target similarity score matrix; Based on the target similarity score matrix, a preset number of entity pairs corresponding to similarity scores before arrangement are selected as pseudo labels in descending order of similarity scores.

4. The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph according to claim 3 is characterized in that: Determining a similarity matrix of attribute names and a similarity matrix of attribute values ​​based on the two attribute modal sets includes: For each attribute modality set, the attribute names and attribute values ​​in the attribute modality set are concatenated and input into the pre-trained model of the entity alignment model to obtain an attribute name embedding matrix. Based on the two attribute name embedding matrices, a similarity matrix of the attribute names is determined; When the pre-trained model determines that two attribute names are the same, the similarity of the attribute values ​​is determined based on the inverse of the difference between the two attribute values ​​corresponding to the two attribute names, and the similarity matrix of the attribute values ​​is determined based on the similarities of the attribute values.

5. The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph according to claim 1 is characterized in that: Before determining the similarity of entity pairs based on the entity features of each pseudo-labeled entity pair using a preset distance measurement method, the method further includes: Based on the joint attention coefficient, the structural embedding of the entity is learned through the message propagation mechanism of the preset joint attention graph attention network to obtain the joint structural embedding feature of the entity, and the joint structural embedding output feature of the entity is obtained by connecting the joint structural embedding features output by the joint attention graph attention network of all layers; wherein the joint attention coefficient is used to represent the indicator of the interactive attention strength between two different modal information; The entity's relational embedding vector is used as the relational attention vector coefficient. The relational-aware structural embedding of the entity is learned through a preset relation-aware graph attention network to obtain the entity's relation-aware structural embedding feature. The relation-aware structural embedding features output by the relation-aware graph attention network at all layers are then connected to obtain the entity's relation-aware structural embedding output feature. The joint structure embedding output feature of the entity and the relationship-aware structure embedding output feature of the entity are concatenated to obtain an entity structure embedding vector as the entity feature.

6. The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph according to claim 5 is characterized in that: The joint attention coefficient is obtained as follows: For each modal information of the entity, an entity-level weight is generated through a preset self-attention learning mechanism; Fusing the entity-level weights of each modal information with the embedding vectors of the entities corresponding to the modal information to obtain a joint embedding vector of the entity; wherein the embedding vector of the entity is used to represent the structural embedding information of the entity; The joint embedding vectors of two adjacent entities are merged and normalized to obtain the joint attention coefficient.

7. The noise-resistant entity alignment method based on the multimodal industrial chain and supply chain knowledge graph according to claim 1 is characterized in that: After taking the pseudo labels to be divided corresponding to the entity pairs whose weights are not less than the second preset threshold as clean labels and taking the pseudo labels to be divided corresponding to the entity pairs whose weights are less than the second preset threshold as noise labels, the method further includes: Determining an adjusted alignment loss function based on a preset weight adjustment function and a preset initial alignment loss function; Adding the adjusted alignment loss function and the preset cross entropy loss function to obtain a target loss function; Based on the objective loss function, parameters of the entity alignment model are updated.

8. A noise-resistant entity alignment device based on a multimodal industrial chain and supply chain knowledge graph, characterized in that: include: An acquisition module is configured to acquire a plurality of pseudo labels, wherein the pseudo labels are used to represent predicted aligned entity pairs obtained by learning unlabeled entities through a preset entity alignment model, wherein the entity pairs include entities in the industry chain knowledge graph and entities in the supply chain knowledge graph, and the entity information includes at least two modal information; A first determination module is configured to determine the similarity of entity pairs based on entity features of each pseudo-labeled entity pair using a preset distance measurement method, and if the similarity of the entity pairs is greater than a first preset threshold, the pseudo-label corresponding to the entity pair is used as a clean label; if the similarity of the entity pairs is not greater than the first preset threshold, the pseudo-label corresponding to the entity pair is used as a pseudo-label to be divided; The second determination module is used to determine the weight of the entity pair of the pseudo-label to be divided based on a preset weight adjustment function for each of the pseudo-labels to be divided, and to use the pseudo-label to be divided corresponding to the entity pair whose weight is not less than the second preset threshold as a clean label, and to use the pseudo-label to be divided corresponding to the entity pair whose weight is less than the second preset threshold as a noise label.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images