Link prediction method based on multimodal embedding interaction for important data completion in public networks

Abstract

A multimodal embedding interaction-based link prediction method for completing important public network data belongs to the field of network data security. It addresses the problem that existing interactive methods randomly generate embedding vectors, thus limiting the model's ability to mine more valuable information. It achieves better integration of multi-source heterogeneous data and completion of important public network data. The method steps are as follows: obtaining a knowledge graph dataset of important public network data and textual descriptions of these important data; training a text embedding model; training a structural embedding model; alternating concatenation; feature dimensionality reduction; feature rearrangement; circular convolution; and predicting tail entity vectors. Building on the embedding interaction method, this method utilizes the multimodal information of important data to address the difficulty in integrating and completing multi-source heterogeneous public network data, improving the accuracy of predicting important data.

Description

Technical Field

[0001] The present invention relates to a link prediction method based on multimodal embedding interaction for completing important data on public networks. Background Art

[0002] Network data security management is of great significance to standardizing network data processing activities, protecting the legitimate rights and interests of individuals and organizations in cyberspace, and maintaining national security and public interests. The analysis of important data on public networks faces challenges such as multi-source heterogeneity, huge data volume, and lack of integrity. The study is based on knowledge graphs and deep learning technologies to achieve scanning and completion of Internet public data resources. The essence of a knowledge graph (KG) is a semantic network, in which the nodes in the network represent entities or concepts, and the edges in the network represent various semantic relationships between entities or concepts. Knowledge graphs can effectively solve the problems of multi-source heterogeneity and lack of integrity of important data on public networks by using entities and relationships. A typical knowledge graph is usually a multi-relational dataset that contains a large number of facts, such as Figure 1 As shown in the figure, the triple (William Shakespeare, birthplace, England) represents the fact that Shakespeare was born in England. However, existing knowledge graphs are mostly incomplete. This problem becomes more serious with the increase in available information on the internet. The performance of knowledge graph-related applications such as recommendation and search is heavily dependent on the supplementation of knowledge graphs. Therefore, the need to improve knowledge graphs has become more urgent. Link prediction (also known as knowledge graph completion) can predict unknown facts based on known facts in the knowledge graph, which can effectively improve the prediction and reasoning performance of knowledge graphs. Knowledge graph embedding (KGE) for important public network data is an effective link prediction method. It embeds entities and relations in the knowledge graph into a continuous vector space, thereby preserving the structural information in the knowledge graph while facilitating computation. Specifically, KGE aims to learn representational embeddings for entities and relations through a discriminator that automatically assigns high scores to valid triples and low scores to invalid triples.

[0003] Using knowledge graphs and deep learning technologies, we rapidly scan and fuse multi-source heterogeneous data in public networks. We developed a link prediction method based on multimodal embedding interactions to alleviate the problem of missing data and lay a data foundation for discovering important data in public networks. Knowledge graph embedding for public important data can be categorized into three types: translation distance-based, text-based, and neural network-based. Translation distance-based KGE methods, such as TransE, TransH, TransR, TransD, and TransA, leverage the internal structural information in knowledge graphs for representation learning. However, these models process each triple independently and therefore fail to fully capture the rich semantic information of triples. Text-based knowledge graph embedding can enhance the semantic representation of entity and relationship vectors, improving the semantic accuracy of the model. Some researchers have cited the integration of text and structural information to make entity representations more semantically robust. However, these models rely on single-layer mining to train the scoring function, which has limitations in terms of embedding vector representation. The advantage of neural networks is that they use multi-layer network structures to improve the representation of parameters under the same parameters. Many existing models use different neural network architectures to learn KGE. For example, ConvE and InteractE use convolutional neural networks to mine features of entity and relationship embeddings. These models can enhance their representational capabilities by increasing the possible interactions between entity and relationship embedding vectors, thereby improving link prediction performance. However, in the embedding interaction process of knowledge graph embedding methods based on convolutional neural networks, the entity and relationship embedding vectors are randomly generated and lack any semantic information. This limits the convolutional neural network's ability to explore different aspects between entities and relationships, making it difficult to effectively mine more valuable information from the entity and relationship embedding vectors, thereby limiting the model's ability to improve link prediction accuracy. Summary of the Invention

[0004] This paper addresses the difficulty in fusing and completing multi-source heterogeneous data on the public web. Specifically, it proposes a link prediction method based on multimodal embedding interaction for completing important public web data. This method enhances the quality of capturing the interaction between entity and relationship embedding vector features by combining multi-source heterogeneous data within important public web data. This improves the link prediction accuracy of public web knowledge graphs and further enhances the ability to fuse and complete multi-source heterogeneous data on the public web.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is a multimodal embedded interactive link prediction method (MEILP). In the MEILP model, the input is the text description information of important data in the public network and the public network knowledge graph dataset. The method flow chart is as follows: Figure 2As shown in Figure 1, the implementation steps of this method are as follows: Step (1) Obtain a knowledge graph dataset of important public network data; collect information from different public network dataset websites, and then summarize the relevant information of the dataset into an Excel table. The relevant information includes: dataset field, dataset name, download address, data type, dataset introduction, data size, whether it is labeled, whether it can be downloaded, etc. By summarizing these relevant information, a comprehensive understanding of the basic information of the dataset is obtained. A total of 144 datasets were collected. Based on whether the dataset has a label, 132 datasets were screened out, totaling 11 public network domain data, including security, address transportation, e-commerce, computer science, Internet, finance, automobile, tobacco, medical health, humanities and natural sciences, etc., with about 2787.6G data (the data size can be determined according to the actual situation), and the data types include text, image, audio and video. In this study, considering that the data type of the humanities field dataset (Freebaseddataset) is unified and the semantic information is rich, it is suitable for related research such as knowledge fusion and knowledge graph completion. Therefore, this study is based on the data in Freebase. The Freebase dataset is in the form of triples (head entity, relation, tail entity), where the head entity is represented by , the relation is represented by r, and the tail entity is represented by t. Triples are divided into positive samples and negative samples. For example, there are two triplets (h1, r1, t1) and (h2, r2, t2). Given a head entity h1 and a relation r1 to predict the tail entity, a correct prediction is a positive sample group (h1, r1, t1), and an incorrect prediction is a negative sample group (h1, r1, t2).

[0006] Step (2) Text embedding: The purpose of embedding text information of important data is to process text type data through text embedding method to obtain the text embedding vector of the entity; the input is the text description information related to the important data in the Freebase dataset collected on the public network, and the text embedding process adopts the BERT (Bidirectional Encoder Representation from Transformers) method. BERT has been proven to be a powerful natural language processing model that can fully retain the important data information in the text description. To this end, the text description is processed using BERT-based query encoding to obtain a high-quality entity text embedding vector S e In the original BERT, a "sentence" can be any range of continuous text or word sequence, rather than an actual language sentence. In contrast, this method packages the text description sentences of important data into a single input token sequence. Each input token sequence contains multiple sentences, using a unique token [SEP] to separate sentences. The input representation is constructed by summing the corresponding token, segment, and position embeddings. Token Embeddings tGenerated by WordPiece embedding. Different elements are distinguished by [SEP], so they contain different segment embeddings e s Position embedding p It is a mapping of position size to a low-dimensional dense vector. Each input token has a positional embedding representation, and different tokens have the same positional embedding at the same position. The first token in each input sequence is a unique classification token [CLS]. The final hidden state corresponding to this token is the entity text embedding vector.

[0007] Step (3) Structural embedding: The purpose of embedding the structural information of the important data knowledge graph in the Freebase dataset is to obtain the structural embedding vector of the entity; in the structural embedding, the TransD structural embedding is used to train the triples to obtain the entity structural embedding vector S that satisfies the structure h+r=t e and relational structure embedding vector S r TransD constructs a low-dimensional continuous vector to describe the semantic information of the KG. The generated embedding captures the semantic similarity between entities in the embedding space. In the structural embedding module, the standardized TransD model is used to learn the structural embedding of link prediction, so that the entity structure embedding vector S e and relational structure embedding vector S r Preserve the predicted structural information. Since TransD considers the diversity of relations and entities at the same time, it can well maintain the feature diversity of the embedding vector. Using the vectors trained by TransD, the various features of entity relationships can be mined in the subsequent interaction process. TransD uses two vectors to represent each entity and relationship. The first vector represents the meaning of the entity or relationship, and the other projection vector will be used to construct the mapping matrix. Given a triple (e h , e r , e t ), where e h represents the randomly initialized head entity embedding vector, e r represents the randomly initialized relation embedding vector, e t Represents the randomly initialized tail entity embedding vector; TransD maps the head and tail entities to the relational space, and obtains the mapping matrix of the entity respectively The mapping matrix of the relationship

[0008]

[0009]

[0010] in, Represents the head entity embedding vector e h Perform the transpose operation, I m×nrepresents the identity matrix, Represents the tail entity embedding vector e t Perform a transpose operation. The mapping matrix is ​​related to both entities and relationships;

[0011]

[0012]

[0013] are the mapping vectors of the head entity and the tail entity respectively.

[0014]

[0015] Score function f(t r ) represents the mapping vector of the head entity Add the relationship vector e r Mapping vector to tail entity Therefore, the objective function S is minimized by calculating the mapping vectors of the head and tail entities EP :

[0016]

[0017] f r ={ <e h , e r , e t >| <e h , e r , e t >∈E};

[0018] f′ r ={ <e′ h , e r , e t >|e′ h ∈E}∪{ <e h , e r , e′ t >|e′ t ∈E}

[0019] Where is the boundary parameter, which is used to limit the relative position between entities and relations in the relation embedding space. Usually, the boundary parameter can be randomly initialized to a random number between [-0.001, 0.001]. During the training process, the model learns the embedded representation between entities and relations by optimizing the loss function and limits the relative position relationship between entities and relations by updating the boundary parameter. E is the set of all entities in the dataset. f r is the set of positive samples in the training dataset; f r ′ is the damaged triplet negative sample set, which is obtained by replacing the positive sample set fr The head entity e of the valid triple h or tail entity e t Randomly replace with any head entity e′ in the entity set h or tail entity e′ t and produced;

[0020] Step (4) Alternating splicing module; the knowledge graph embedding of the text information and structural information of important data helps to enhance feature interaction, thereby mining different features to achieve the purpose of integrating multi-source heterogeneous data in the public network and completing the knowledge graph of important data. In order to better preserve the text and structural information of the embedding vector, an alternating splicing model is designed to combine the entity text embedding vector and the entity structure embedding vector. Since the training spaces of the text and structure embedding vectors are different, a simple summation operation or an attention weighted summation operation is used to add the text embedding vector to the structure embedding vector. However, the fact is that each dimension represents a different feature, and the sum operation does not provide good information about the text and structure. In order to better preserve the text and structural information, the alternating connection model uses an alternating form to embed the entity text vector T e With the structure embedding vector S e Stitching is performed so that the text and structural information of important data are well preserved.

[0021] Step (5) Feature dimensionality reduction module; entity embedding vector [T e , S e ] is mapped to 200 dimensions using the feature dimensionality reduction method, and the entity embedding vector J is obtained. e Finally, to J e and S r Cross-arrange so that no two adjacent units are embedded in the same vector. This method combines the entity embedding vector J e and relation embedding vector S r Perform joint interactive learning. Entity embedding vector [T e , S e ] is mapped to 200 dimensions using the feature dimensionality reduction method, and the entity embedding vector J is obtained. e ; Feature dimensionality reduction first performs a centralization operation on the matrix C.

[0022]

[0023] in, is the centralized data matrix, X is the total number of dimensions of the text embedding vector and the structure embedding vector, C i is the i-th element in matrix C. Next, calculate the centralized data matrix The covariance matrix of :

[0024]

[0025] Represents the covariance matrix. Then, the covariance matrix can be obtained by eigenvalue decomposition The eigenvalues ​​and eigenvectors of are:

[0026]

[0027] Finally, the original matrix C is projected onto the matrix consisting of the first 200-dimensional eigenvectors v1, v2, ..., v 200 In the space formed by e .

[0028] Step (6) Feature rearrangement module: The method embeds the entity vector J5 and the relationship vector S r Rearrange the features into different matrices. The first step in feature rearrangement is to randomly sort the elements of the entity and relationship embedding vectors to generate different vector representations. This aims to alternately concatenate the interaction information generated by the entity and relationship embedding vectors from different perspectives. The second step is to alternately concatenate the two embedding vectors element-wise. The third step is to rearrange the concatenated vectors into different matrices as input to the circular convolution.

[0029] Step (7) Circular convolution module; the reconstructed permutation vector is trained using a circular convolutional neural network. The circular convolution module provides a unified embedding space for text and structure embedding vectors. The text embedding vector is made to fully interact with the structural information to capture more features. Circular convolution can improve feature interaction and has been successfully applied to tasks such as image recognition. Circular convolution stacks each permutation into a separate channel. Circular convolution is applied to the circular padding of the convolution layer in a deep manner. It expands the size of the matrix by filling the top and bottom of the matrix with vectors of half the size of the convolution kernel, respectively. The normalization function and activation function used simultaneously are used to prevent gradient explosion. Then, the hidden layer is passed to prevent overfitting, and finally, the activation function is used for backpropagation.

[0030] Step (8) maps to the vector space module; maps to the vector space module; P k The entity text is embedded into the vector J using alternating connections and feature dimensionality reduction e With the structure embedding vector S r Next, the feature reordering operation φ() is used to reorder the entity and relationship concatenation vector P k Arrange; Θ represents the deep circular convolution operation, using the entity relationship connected by the filter w as the input of the convolution layer; vec() vectorizes the obtained feature map tensor; e oRepresents the target entity embedding matrix. Functions f() and g() are ReLU activation function and sigmoid activation function respectively. The model maps g() to the vector space through the linear transfer of matrix parameters W; and performs matrix multiplication with the target entity embedding matrix e o Multiply to get the final prediction score ψ;

[0031] ψ(o)=g(vec(f(φ(P k )Θw))W)e o

[0032] This method, based on interactive knowledge graph embedding, leverages the multi-source heterogeneous information of important public network data to improve the learning performance of the original method and solve the problem of difficult fusion and completion of multi-source heterogeneous data for the public network. The performance of the recurrent convolutional model based on multimodal embedding interaction is significantly improved compared to the original interactive method. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the knowledge graph

[0034] Figure 2 Flowchart of this method DETAILED DESCRIPTION

[0035] The purpose of the present invention is to propose a link prediction method based on multimodal embedding interaction for important data in public networks. On the basis of embedding interaction, the multi-source heterogeneous information of important data is used to produce more accurate prediction results, thereby integrating multi-source heterogeneous data in the public network and supplementing the important data information of the public network.

[0036] In order to achieve the above objectives, the technical solution adopted by the present invention is a link prediction method based on multimodal embedding interaction, such as Figure 2 The implementation steps of this method are as follows:

[0037] Step (1) Obtain the knowledge graph dataset of important data on the public network and the text description information of important data:

[0038] Public web knowledge graph datasets are represented as triples (head entity, relation, entity), where the head entity is represented by , the relation is represented by r, and the tail entity is represented by t. For example, (Shakespeare, author, Romeo and Juliet) indicates that Shakespeare is the author of Romeo and Juliet. In this study, the proposed interaction method was validated using the knowledge graph datasets FB15k-237 and WN18RR, demonstrating that the proposed model can effectively integrate multi-source heterogeneous information and complete important data in the public web. FB15k-237 is primarily derived from the Freebase dataset, a large portion of which describes knowledge facts about movies, actors, awards, and sports. WN18RR is the English vocabulary database WordNet. FB15k-237 has 14,496 entities, 237 relations, 272,115 triples in the training set, 17,535 triples in the validation set, and 20,466 triples in the test set. The WN18RR dataset has 40,943 entities and 11 relations. The training set has 86,835 triples, the validation set has 3,034 triples, and the test set has 3,134 triples. The text descriptions of important data are collected from Wikipedia and the public internet.

[0039] Step (2) divide the triples into positive and negative sample sets according to the data set;

[0040] Since the knowledge graph dataset of important data only records the triples that are consistent with the facts, but lacks the triples that are inconsistent with the facts, it is necessary to divide each entity into a set of positive and negative sample triples. The positive sample triple set of the entity is represented by f t Indicates that the negative sample set uses f t 'express.

[0041] Step (3) Text embedding module for important data: The input is the text description information of the important data, and the text embedding process adopts the BERT method. BERT has been proven to be a powerful natural language processing model that can fully retain the important data information in the text description. To this end, the text description is processed using BERT-based query encoding to obtain high-quality text embedding vectors of important data;

[0042] Step (4) Structural embedding module for important data: Use TransD structural embedding to train the triples and obtain the entity and relation structural embedding vector S that satisfies the structure h+r=t e and S r TransD constructs a low-dimensional continuous vector to describe the semantic information of the KG. The generated embedding captures the semantic similarity between entities in the embedding space. In the structural embedding module, the standardized TransD model is used to learn the structural embedding of link prediction, so that the entity structure embedding vector Se and relational structure embedding vector S r Save the predicted structural information.

[0043] Step (5) Alternating splicing module; knowledge graph embedding that combines text information and structural information of important data helps to enhance feature interaction, thereby mining different features. In order to better preserve the text and structural information of the embedding vector, an alternating splicing model is designed to combine the entity text embedding vector and the entity structure embedding vector. Since the training spaces of the text and structure embedding vectors are different, a simple summation operation or an attention weighted summation operation is used to add the text embedding vector to the structure embedding vector. However, the fact is that each dimension represents a different feature, and the sum operation does not provide good information about the text and structure. In order to better preserve the text and structural information, the alternating splicing model uses an alternating form to embed the entity text vector T e With the structure embedding vector S e Splicing is performed, which can well preserve the text and structural information of important data.

[0044] Step (6) Feature dimensionality reduction module; entity embedding vector [T e , S e ] is mapped to 200 dimensions by using the feature dimensionality reduction method to obtain the entity embedding vector J after dimensionality reduction e .

[0045] Step (7) Feature rearrangement module: embed the entity vector J e and relation embedding vector S r Cross-arrange so that no two adjacent units are embedded in the same vector, which is helpful for the prediction results. Therefore, this method combines the entity embedding vector J e and relation embedding vector S r Arranged into different matrix forms through different arrangement methods.

[0046] Step (8) Circular convolution module; the reconstructed permutation vector is trained using a circular convolutional neural network. Circular convolution embedding provides a unified embedding space for text and structure embedding vectors. The text embedding vector is made to fully interact with the structural information to capture more features. Circular convolution can improve feature interaction and has been successfully applied to tasks such as image recognition. Circular convolution stacks each permutation into a separate channel. Circular convolution is applied to the circular padding of the convolution layer in a deep manner. It expands the size of the matrix by filling the top and bottom of the matrix with vectors of half the size of the convolution kernel, respectively. The normalization function and activation function used simultaneously are used to prevent gradient explosion. Then, the hidden layer is passed to prevent overfitting, and finally, the activation function is used for backpropagation.

[0047] Step (9) maps to the vector space module; maps to the vector space module; P k The entity text is embedded into the vector J using alternating connections and feature dimensionality reduction e With the structure embedding vector S r Next, the feature reordering operation φ() is used to reorder the entity and relationship concatenation vector P k Arrange; Θ represents the deep circular convolution operation, using the entity relationship connected by the filter w as the input of the convolution layer; vec() vectorizes the obtained feature map tensor; e o Represents the target entity embedding matrix. Functions f() and g() are the ReLU activation function and the sigmoid activation function, respectively. The model maps g() to the vector space through a linear transfer of the matrix parameter W; the final prediction score ψ is obtained by multiplying it with the target entity embedding matrix e0 through matrix multiplication.

[0048] ψ=g(vec(f(φ(P k )Θw))W)e o

[0049] Step (10) determines the evaluation metrics; a filtering setting is used, i.e., when evaluating the test triples, all valid triples are filtered out from the candidate set, which is generated by damaging the head or tail entity of the triple. For the test triple (h, r, t), the goal of the KGE task is to predict the missing link, for example, given the head entity h and the relation r, predict the tail entity t, or given the tail entity t and the relation r, predict the head entity h. Performance is reported on standard evaluation metrics: mean reciprocal rank (MRR), mean rank (MR), and Hits@1 (the accuracy of the top 1 prediction) and Hits@10 (the accuracy of the top 10 predictions). Lower MR and higher MRR, higher Hits@10 and Hits@1 indicate better performance. The model is evaluated following the "filtering" setting protocol, i.e., all entities are ranked, excluding other real entity sets that appear in the training, validation, and test sets.

[0050] Step (11) Experimental parameter setting: In the MEILP model, model parameters were selected by grid search based on the MRR value on the validation set. The dropout rate was selected from the range of (0.1, 0.5), the learning rate was selected from the range of (0.0001, 0.001), the batch size was selected from {32, 64, 128, 256, 512}, and the smoothing parameter λ was selected from the range of (1, 10). The specific experimental settings are as follows: the number of negative samples per training round is 1000, the batch size is set to 256, the number of iterations is 500, and the embedding dimension is 200 dimensions. Finally, based on the MRR indicator and annealing test, the learning rate of the FB15k-237 dataset is set to 0.0001. In the convolution module, the number of filters for each convolution operation is set to 96, the convolution kernel size is 9, the number of convolution channels is 1, the input layer dropout rate is 0.2, the hidden layer dropout rate is 0.5, and the feature dropout rate is 0.5. The learning rate of the WN18RR dataset is set to 0.001. In the convolution module, the number of filters for each convolution operation is set to 96, the convolution kernel size is 11, the number of convolution channels μ is 4, the dropout rate of the input layer is 0.2, the dropout rate of the hidden layer is 0.3, and the dropout rate of the feature is 0.2.

[0051] The multimodal embedding interaction-based link prediction method for completing important data in public networks can produce better prediction results than existing methods. Experimental results show that the tail entity can be accurately predicted by the head entity and relationship in the important data, thereby completing the information of important data in public networks. Table 1 plots the comparison of the multimodal embedding interaction-based link prediction model (MEILP) for completing important data in public networks with existing methods. On the FB15k-237 dataset, MEILP significantly outperforms the baseline in MRR, MR and Hits@1, and is second only to TECRL in Hits@10, which shows the importance of enhanced interaction. On the WN18RR dataset, the MEILP model achieves state-of-the-art performance in MRR, MR and Hits@10, which is more than 70% higher than the MR of the latest baseline model. (2) On the FB15K-237 and WN18RR datasets, the TECRL, MEILP and DKRL models that integrate the textual information of important data outperform other algorithms in Hits@10 and MR. This shows that adding textual description information of important data can lead to higher ranking of correct entities. (3) The SACN, TECRL, and MEILP models combine translation models with convolutional network architectures, achieving better results than the convolutional network models RSN, RotatE, ConvE, ConvR, and InteractE on the FB15K-237 and WN18RR datasets on Hits@10. The improvements demonstrate the advantages of learning entity structure features in knowledge graphs. This verifies the hypothesis that multimodal information of important data can effectively enhance the interaction between entities and relations and improve the accuracy of link prediction. In summary, this paper proves the effectiveness of MEILP and can well complement important data information.

[0052] Table 1: Experimental results

[0053]

Claims

1. A link prediction method based on multimodal embedding interaction for completing important data on public networks, characterized by: Here are the steps: Step (1) obtain a knowledge graph dataset of important public network data; collect information from different public network dataset websites, and then summarize the dataset related information into an Excel table, the related information includes: dataset field, dataset name, download address, data type, dataset introduction, data size, whether it has a label, whether it can be downloaded; the dataset is represented in the form of a triple (head entity, relationship, entity), where the head entity is represented by h, the relationship is represented by r, and the tail entity is represented by t; the triples are divided into positive samples and negative samples, there are two triplets (h1, r1, t1) and (h2, r2, t2), given a head entity h1 and a relationship r1 to predict the tail entity, the correct prediction is the positive sample group (h1, r1, t1), and the wrong prediction is the negative sample (h1, r1, t2); Step (2) Text embedding: The purpose of embedding text information of important data is to process text type data through text embedding method to obtain the text embedding vector of the entity; the input is the text description information related to the important data in the Freebase dataset collected in the public network, and the text description is processed using BERT-based query encoding to obtain the entity text embedding vector S e ; Text description sentences of important data are packaged into a single input token sequence; Each input token sequence contains multiple sentences, using a unique token [SEP] to separate sentences; The input representation is constructed by summing the corresponding, token, segment and position embeddings; Token embedding e w Generated by WordPiece embeddings; different elements are distinguished by [SEP], so they contain different segment embeddings e s ; Position embedding e p It is a mapping from position size to a low-dimensional dense vector; each input token has a position embedding representation, and different tokens contain the same position embedding at the same position; the first token in each input sequence is a unique classification token [CLS]; the final hidden state corresponding to this token is the entity text embedding vector; Step (3) Structural embedding: In structural embedding, the triples are trained using TransD structural embedding to obtain the entity structural embedding vector S that satisfies the structure h+r=t e and relational structure embedding vector S r ; TransD uses two vectors to represent each entity and relation; the first vector represents the meaning of the entity or relation, and the other projection vector will be used to construct the mapping matrix; given a triple (e h , e r , e t ), where e h represents the randomly initialized head entity embedding vector, e r represents the randomly initialized relation embedding vector, e t Represents the randomly initialized tail entity embedding vector; TransD maps the head and tail entities to the relational space, and obtains the mapping matrix of the entity respectively The mapping matrix of the relationship in, Represents the head entity embedding vector e h Perform the transpose operation, I m×n represents the identity matrix, Represents the tail entity embedding vector e t Perform a transpose operation; the mapping matrix is ​​related to both entities and relationships; are the mapping vectors of the head entity and the tail entity respectively; Score function f(t r ) represents the mapping vector of the head entity Add the relationship vector e r Mapping vector to tail entity Therefore, the objective function S is minimized by calculating the mapping vectors of the head and tail entities EP : f r ={ <e h ,And r ,And t >| <e h ,And r ,And t >∈E}; f' r ={ <e' h ,And r ,And t >|e′ h ∈E}∪{ <e h ,And r ,And' t >|it's t ∈E} Where γ is a boundary parameter used to limit the relative position between entities and relations in the relation embedding space. Usually, the boundary parameter can be randomly initialized to a random number between [-0.001, 0.001]. During the training process, the model learns the embedded representation between entities and relations by optimizing the loss function and limits the relative position relationship between entities and relations by updating the boundary parameter. E is the set of all entities in the dataset. f r is the set of positive samples in the training data set; f' r is a set of damaged triplet negative samples, which is obtained by replacing the positive sample set f r The head entity e of the valid triple h or tail entity e t Randomly replace with any head entity e' in the entity set h or tail entity e' t and produced; Step (4) Alternating splicing module: The alternating splicing model embeds the entity text into the vector T by alternating the elements in the vector e With the structure embedding vector S e Perform splicing to obtain matrix C; Step (5) Feature dimensionality reduction module; entity embedding vector [T e ,S e ] is mapped to 200 dimensions using the feature dimensionality reduction method, and the entity embedding vector J is obtained. e ; Feature dimensionality reduction first performs a centralization operation on the matrix C; in, is the centralized data matrix, X is the total number of dimensions of the text embedding vector and the structure embedding vector, C i is the i-th element in matrix C; next, calculate the centralized data matrix The covariance matrix of : Represents the covariance matrix; then, the covariance matrix can be obtained by eigenvalue decomposition The eigenvalues ​​and eigenvectors of are: λ represents the eigenvalue, and v represents the corresponding eigenvector; finally, the original matrix C is projected onto the first 200-dimensional eigenvector v1, v2, ..., v 200 In the space formed by e : Step (6) Feature rearrangement module: Method embeds the entity into vector J e and relation embedding vector S r Rearrange into different matrices; the first step of feature rearrangement is to randomly sort the elements of the entity and relationship embedding vectors to generate vector representations with different μ, aiming to alternately splice the interaction information generated by the entity and relationship embedding vectors from different angles; the second step is to alternately splice the two embedding vectors element by element; the third step is to rearrange the spliced ​​vectors into different matrices as the input of the circular convolution; Step (7) Circular Convolution Module; The reconstructed permutation vector is trained using a circular convolutional neural network; The circular convolution module provides a unified embedding space for text and structure embedding vectors; Circular convolution stacks each permutation into a separate channel; Circular convolution is applied to the circular padding of the convolution layer in a deep manner; It expands the size of the matrix by filling the top and bottom of the matrix with vectors of half the size of the convolution kernel respectively; The normalization function and activation function used simultaneously are used to prevent gradient explosion; Then, it is passed through the hidden layer to prevent overfitting, and finally, the activation function is used for backpropagation; Step (8) is mapped to the vector space module; P k The entity text is embedded into the vector J using alternating connections and feature dimensionality reduction e With the structure embedding vector S r Next, the feature reordering operation φ() is used to reorder the entity and relationship concatenation vector P k Arrange; Θ represents the deep circular convolution operation, using the entity relationship connected by the filter w as the input of the convolution layer; vec() vectorizes the obtained feature map tensor; e o Represents the target entity embedding matrix; functions f() and g() are ReLU activation functions and sigmoid activation functions respectively; the model maps g() to the vector space through the linear transfer of matrix parameters W; and is embedded in the target entity matrix e by matrix multiplication. o Multiply to get the final prediction score ψ; ψ=g(vec(f(φ(P k )Θw))W)e o 。 2. The link prediction method based on multimodal embedding interaction for completing important public network data according to claim 1 is characterized by: Steps (2-4) specifically use the alternating splicing algorithm to fully integrate multi-source heterogeneous information in important data on the public network, and obtain the optimal embedding vector representation of important data entities and relationships, so as to better integrate multi-source heterogeneous data and complete important data information in the public network.

Images