A graph embedding expert entity alignment method enhanced by reinforcement learning

Abstract

The present invention discloses a graph embedding entity alignment method based on reinforcement learning enhancement. The present invention adopts the method of constructing heterogeneous subgraphs, and only performs message aggregation on the n-hop neighbors of the entity pairs to be aligned, which directly reduces the computing resource requirements. A graph embedding learning algorithm based on feature linear modulation is used, and the idea of ​​super network is introduced to complete the message transmission mechanism and node update mechanism with high computational complexity with a small number of parameters, so as to better utilize the interactive information between nodes. In addition, the present invention proposes a node selector enhanced by reinforcement learning, proposes and applies a reliability measurement method based on self-supervisory signals in the node selector, samples a certain number of reliable edges, filters problem edges while limiting the size of heterogeneous subgraphs, and ensures the reliability of edges participating in node updates. The present invention also implements a node sampling quantity update strategy based on reinforcement learning, dynamically optimizes the number of sampled nodes, and enhances the node selector.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology in natural language processing, and in particular to a graph embedding entity alignment method based on reinforcement learning enhancement. Background Art

[0002] "Competition in the 21st century is a competition for talent", and talent factors occupy a core position in industrial factors. Talent knowledge bases can serve as upstream data knowledge support for downstream decision-making tasks such as factor scheduling, talent recommendation, and project precision investment. They are the basis for local governments to improve and optimize industrial transformation, reasonably coordinate and mobilize major industrial factors. At present, a large amount of talent-related data is widely distributed on Internet platforms such as academic institution websites and academic search engines, with serious island phenomena, diverse data types, and uneven quality. In order to build a large-scale, high-quality talent knowledge base, integrating the knowledge of other talent knowledge bases is one of the necessary means. As a hub linking different knowledge bases, expert entities are very important for integrating various talent knowledge bases. The process of identifying expert entities that express the same individual in the real world in different talent knowledge bases is called expert entity alignment.

[0003] Entity alignment usually compares some features of the entity pairs to be aligned, such as entity names, entity attributes and attribute values, and uses some machine learning methods or representation learning-based methods to calculate and score similarity. However, for the talent knowledge base, its existing data characteristics and real-world application requirements put forward some requirements for the existing entity alignment methods: First, the available information is reduced. The relationships and attributes in the talent knowledge base are enumerable and do not need to be aligned. This feature makes it impossible for some existing entity alignment methods to use the alignment information of relationship predicates and attribute predicates, resulting in a decrease in model performance. Second, the computing resources are limited and the running results are unstable. The scale of entities in the talent knowledge base is very large, and a large number of paper entities or expert entities will be added every day. In addition, the number of results published by different experts also varies greatly, which may cause a certain degree of computing instability in actual application scenarios. Third, the problem edges in the knowledge base. In various existing knowledge bases, it is common to have erroneous triple data. The existence of these erroneous problem edges will undoubtedly have a negative impact on the model's judgment of whether the entity pair is the same real entity. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a graph embedding expert entity alignment method based on reinforcement learning enhancement. The technical solution of the present invention is as follows:

[0005] The present invention provides a graph embedding expert entity alignment method based on reinforcement learning enhancement, which comprises the following steps:

[0006] Step 1: Obtain data from two talent knowledge bases G1 = (E1, T1, R) and G2 = (E2, T2, R), where E represents an entity set, and entity types include expert entities and paper entities; R represents a relationship set; and T represents a triple set, which is a subset of E×R×E.

[0007] Step 2: For each expert entity e in a talent knowledge base, generate a candidate expert set C from another talent knowledge base based on the name of the expert through the candidate entity pair generation module and the regular matching template generator method;

[0008] Step 3: For each candidate expert c∈C, construct<e,c> The 2-hop heterogeneous subgraph HG = (V, H, T, R), where H1 and H2 represent the entity initial vector sets of the two talent knowledge bases respectively; the initial node vector represents h (0) :

[0009]

[0010]

[0011] h struct =LINE(G)

[0012] in Represents vector concatenation operation, h attr is the average vector of the word vectors obtained by the skip-gram model for each attribute feature of the entity, h struct It is a structural vector obtained by encoding the structural information of each entity in the knowledge base through the LINE model;

[0013] Step 4: In the node vector update module, in each graph embedding layer, the reliability of each edge is calculated using the reliability measurement method based on the self-supervisory signal;

[0014] Step 5: In each graph embedding layer, for the heterogeneous subgraph HG, use the top-p sampling strategy to sample p of each relationship from large to small according to the reliability calculated in step 4. r The node sampling quantity update strategy based on reinforcement learning is used to update the node sampling quantity.

[0015] Step 6: In each graph embedding layer, after obtaining the sampled heterogeneous subgraph, the node vector is updated using the graph embedding learning algorithm based on feature linear modulation;

[0016] Step 7: After L layers of graph embedding, take out the updated entity pairs to be aligned<e,c> The node vector and Calculate the matching probability through multi-layer perceptron

[0017] Step 8: Based on the matching probabilities of all candidate entities, select the candidate expert with the highest probability as the matching expert.

[0018] Compared with the existing methods, the advantages of the method of the present invention are:

[0019] (1) To address the problem of reduced available information, a graph embedding learning algorithm based on feature linear modulation is used. The idea of ​​super network is introduced to dynamically generate the relationship weight matrix according to the target node vector, and the message passing mechanism and node update mechanism with high computational complexity are completed with a small number of parameters, so as to better utilize the interactive information between nodes.

[0020] (2) For the problem edges in the knowledge base, the method of the present invention proposes a node selector enhanced by reinforcement learning, which applies a reliability measurement method based on self-supervisory signals and constructs self-supervisory signals based on existing edges, so that the relationship edges can be sampled according to the reliability, thereby filtering the problem edges and ensuring the reliability of the edges participating in the node update. Since the size of the heterogeneous subgraph after sampling is controlled, the problem of limited computing resources and unstable operation results is solved at the same time.

[0021] (3) To avoid a large amount of manual adjustment of the number of node sampling, the node selector also implements a node sampling number update strategy based on reinforcement learning to dynamically optimize the number of sampling nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the overall framework diagram of the graph embedding expert entity alignment method based on reinforcement learning enhancement.

[0023] Figure 2 This is the overall flow chart of the candidate entity regular matching template generator.

[0024] Figure 3 This is the overall flow chart of the node vector update module, where the dashed box part is only implemented during the training process. DETAILED DESCRIPTION

[0025] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0026] Talent knowledge bases usually contain information on multiple entity types and relationship types. Given two talent knowledge bases, G1 = (E1, T1, R) and G2 = (E2, T2, R), E represents an entity set, and entity types include expert entities and paper entities; R represents a relationship set; and T represents a triple set, which is a subset of E×R×E. Seed entity pair set Represents the set of aligned entity pairs used for training. The expert entity alignment task aims to use the known entity pair information to train a model and use it to predict potential expert alignment results. The equal sign indicates that the two expert entities refer to the same individual in the real world.

[0027] Given an expert entity in a talent knowledge base, the process of finding its corresponding expert entity in another talent knowledge base can be regarded as a matching problem, that is, in a certain feature space, the matching probability of the given expert entity and all candidate expert entities in another talent knowledge base is calculated, and the entity with the highest matching score is regarded as the alignment result.

[0028] like Figure 1 As shown, the present invention designs an overall framework diagram of a graph embedding expert entity alignment method enhanced by reinforcement learning: for two talent knowledge bases, firstly, candidate entity pairs are generated according to names; then, for each pair of entity pairs, a 2-hop heterogeneous subgraph is constructed; then, in the node vector update module, in each graph embedding layer, a node selector based on a self-supervisory signal is first used to perform reliable edge sampling of the heterogeneous subgraph, and at the same time, a reinforcement learning module is used to dynamically update the number of reliable edge samplings, and the heterogeneous subgraph after filtering the problem edges completes the node vector update of the current layer through graph embedding based on feature linear modulation, with a total of L graph embedding layers; finally, the final node vector representation of the expert entity pair to be aligned is taken out, and after vector dot multiplication, a multilayer perceptron is used to complete the calculation of the matching probability.

[0029] The following is a detailed introduction to each module (step) in the present invention.

[0030] S1 Candidate entity pair generation: In order to cope with the abbreviations, polyphones, and double surnames in Chinese and English names, a method based on regular matching template generator is implemented. Figure 2 The process of how the regular matching template generator generates regular matching templates is described in detail.

[0031] First, process Chinese and English names separately, and use the name parser to generate a parsing dictionary. For English names, first parse the surname and first name, determine whether the order of the surname and first name can be determined according to certain rules, and then generate a parsing dictionary. For Chinese names, convert the Chinese names into pinyin form, that is, convert them into English names, while considering polyphones and compound surnames, and generate intermediate parsing results and process them the same as English names. Table 1 shows the parsing rules for converting Chinese names to English names, Table 2 shows the rules for determining the order of surnames and first names for English names / converted Chinese pinyin names, and Table 3 shows the parsing dictionary generation method.

[0032] Table 1 Analysis rules for converting Chinese names into English

[0033]

[0034] Table 2 Rules for determining the order of surnames and given names

[0035]

[0036] Table 3 Parsing dictionary generation method

[0037]

[0038] The parsing dictionary generated by the name parser is used as the input of the template generator to output the corresponding regular template. First, the order of the surname and the first name is determined, then whether the name is an abbreviation is determined, and finally 2 or 4 groups of regular templates are generated according to the order of "surname first" and "surname last". Table 4 shows the abbreviation judgment rules, and Table 5 shows the rules for generating regular templates based on the parsing dictionary.

[0039] Table 4 Abbreviation judgment rules

[0040]

[0041] Table 5 Rules for generating regular templates based on parsing dictionaries

[0042]

[0043]

[0044] S2 builds n-hop heterogeneous subgraphs: Taking n = 2 as an example, for the expert entity pairs to be aligned<a1,a′1> , obtain the first-order neighbor nodes from G1 and G2, including published paper nodes, conference / journal nodes, expert nodes of co-author relationships, and paper nodes and conference / journal nodes of first-order neighbor expert nodes, so as to construct a subgraph with a1 or a′1 as the core. When the results of partial entity alignment are known (for example, whether the conference / journal names are the same, whether the paper titles are the same, and aligned expert entities), the two originally separated subgraphs can be merged to obtain a graph of expert entity pairs to be aligned.<a1,a′1> A heterogeneous subgraph with .

[0045] S3 node vector update module: This step completes the node vector representation update in the heterogeneous subgraph through the L-layer graph embedding layer. The initial node vector representation h needs to be obtained. (0) The specific process of updating the node vector of each graph embedding layer is as follows: Figure 3 As shown in Figure 2, the reliability of each edge is first calculated using a reliability measurement method based on self-supervisory signals, and then a top-p sampling strategy is used to sample a certain number p of each relationship from large to small according to the calculated reliability. r After obtaining the sampled heterogeneous subgraph, the node vector is updated using the graph embedding learning algorithm based on feature linear modulation. Finally, the final node vector representation of the entity pair to be aligned is taken out, and the matching probability is calculated by the multi-layer perceptron. In the training stage, in order to obtain a better reliability measurement method, the existing relationship edge SPT is sampled, and the nodes that do not have the relationship with the target node are sampled at a ratio of 1:1 to construct the non-existent relationship edge SPF, construct the self-supervisory signal, and calculate the average edge sampling loss, so as to train the weight matrix in the reliability measurement method. In addition, in order to dynamically update the number of node samples p for each relationship r , calculate the average unreliability of the sampled edges, and update the sampling quantity using the node sampling quantity update strategy based on reinforcement learning.

[0046] The initial node vector represents h (0) :

[0047]

[0048]

[0049] h struct =LINE(G)

[0050] in Represents vector concatenation operation, h attr is the average vector of the word vectors obtained by the skip-gram model for each attribute feature of the entity (including paper title, abstract, conference / journal name, etc.), h structThe structural information of each entity in the knowledge base is encoded through the LINE model.

[0051] Calculate the reliability of each edge: for a triple<u,r,v> (That is, there is a relationship r between node u and node v, and the direction is u pointing to v), and the calculation process of its reliability S(u, v) is as follows:

[0052] First, calculate the feature representation η of the node on each relationship:

[0053] η u , η v =σ1(W τ(u) h u ),σ1(W τ(v) h v ))

[0054] Where W τ(u) and W τ(v) Represents the weight matrix related to the corresponding node type, and its dimension is R |R|×d , h u and h v It is the node vector representation of nodes u and v in this graph embedding layer, with dimension d.

[0055] Then calculate the node pair<u,v> The probability α that a relation r exists u，v :

[0056] α u，v =σ2(W TF ·(η u ⊙η v ))

[0057] Where W TF The dimension is R 2×|R| , ⊙ is the vector dot product operation.

[0058] Finally, the Manhattan distance is used to calculate the unreliability D(u, v) and the reliability S(u, v) is obtained:

[0059] D(u,v)=||α u，v ||1

[0060] S(u,v)=1-D(u,v)

[0061] Top-p sampling strategy: For each node in the heterogeneous subgraph, for each relationship r, sample p from large to small according to the calculated reliability. r Strip edge.

[0062] The graph embedding learning algorithm based on feature linear modulation updates the node vector: At the lth layer, for node v, its node vector is Updated to h v ∈R d :

[0063]

[0064] Where W r (l) ∈R d×d is the message conversion function weight, σ is the ReLU function, They are the message weights of each neighbor node u calculated by node v in the lth graph embedding layer:

[0065]

[0066] Where g(h v θ g，l，r ) is a hypernetwork implemented using a single linear layer, θ g，l，r ∈R 2d×d are the parameters of this hypernetwork.

[0067] S4 calculates the matching probability: after L layers of graph embedding, the updated entity pairs to be aligned are taken out.<e,c> The node vector and Calculate the matching probability through multi-layer perceptron The specific calculation process is:

[0068]

[0069] Among them, FC consists of 3 linear layers, and each layer uses a ReLU nonlinear function.

[0070] In the S5 training phase, the number of node samplings is updated and the loss function is calculated: (1) The average unreliability of the sampled edges is calculated, and the number of node samplings is dynamically optimized using the reinforcement learning strategy; (2) A self-supervisory signal is constructed, and the average edge sampling loss is calculated, which is then summed with the alignment loss to obtain the final loss.

[0071] Construct self-supervisory signal: For each relation r, 60% of the existing partial relation edges SPT in the sampling graph are<u,r,v> |<u,r,v> ∈T}, as a positive example, and sample nodes that do not have the relationship with the target node at a ratio of 1:1 to build a non-existent relationship edge As negative examples, construct self-supervisory signals based on existing relationship edges.

[0072] Calculate the average edge sampling loss: Use cross entropy loss as the loss function to minimize the average edge sampling loss of all graph embedding layers Thus, the weight coefficients used in training and calculating reliability are:

[0073]

[0074] Where ψ(u, r, v) = 1 represents the edge<u,r,v> exists, otherwise, ψ(u, r, v) = 0 means that the edge does not exist.

[0075] Calculate the average unreliability of the sampled edges: In each epoch e, for relation r, calculate the average unreliability of the sampled edges E r，sampled The average unreliability

[0076] Update the number of node sampling: Update the number of node sampling p of relation r according to the changing trend of the average unreliability between two adjacent epochs r , the Action, Reward, and Termination in the reinforcement learning strategy used are:

[0077] Action = {+∈, -∈}

[0078]

[0079]

[0080] Where ∈ is a small fixed integer.

[0081] If the average unreliability of the current epoch is less than that of the previous epoch, the reward function is positive, and the instant reward is used to greedily increase p r , otherwise, reduce p r .

[0082] During the training process, the loss of the expert entity alignment task includes the alignment loss of all entity pairs to be aligned in addition to the average edge sampling loss based on the reliability metric method of the self-supervisory signal. The goal of this loss is to make the vector representation of the aligned expert entity pairs <z e , z c >After passing through the multi-layer perceptron, the probability is higher and closer to 1, while the score of the non-aligned vector representation is close to 0:

[0083]

[0084] The final loss function of the model is the weighted sum of alignment loss and average edge sampling loss:

[0085]

[0086] Where ||Θ||2 is the L2 regularization penalty term, λ3 is the corresponding penalty term coefficient, λ1 and λ2 are the weight coefficients of the alignment loss and the average edge sampling loss, respectively.

[0087] In the experiment, this implementation example uses the dataset OAG, which involves two major academic networks, Microsoft Academic Network and AMiner. The network sizes are shown in Table 6, and the basic statistical information of the dataset is shown in Table 7.

[0088] Table 6 Network data size in the OAG dataset

[0089] network Number of conferences / journals Number of papers Number of experts Relationship Type AMiner 69，397 172，209，563 113，171，945 co-author,write,publish_on MAG 52，678 208，915，369 253，144，301 co-author,write,publish_on

[0090] Table 7 Basic statistics of the OAG dataset

[0091]

[0092]

[0093] Evaluation indicators: Precision, Recall and F1-score are used as evaluation indicators. Precision is the proportion of positive samples predicted correctly by the evaluation classifier. The larger its value, the more accurate the model's prediction of positive samples is. Recall is the proportion of positive samples predicted correctly by the evaluation classifier to all positive samples. The larger its value, the higher the probability that the model predicts the correct positive samples. F1-score comprehensively evaluates Precision and Recall. The higher its value, the better the overall prediction performance of the model.

[0094] Hyperparameter settings: In the experiment, epoch=100, the size of each batch is 64, the dimension of the vector representation of each node is d=300, the optimizer is Adam, the learning rate of the graph embedding learning algorithm based on feature linear modulation is 0.005, and the learning rate of the reliability measurement method based on self-supervisory signals is 0.001; the initial sampling number p of each relationship r are all 30, each action update ∈ in the reinforcement learning update strategy is 2; the number of graph embedding layers in the node vector update module is L=3, and the loss coefficients λ1, λ2, and λ3 in the loss function are 2, 1, and 0.001 respectively. The weight coefficient dimensions of the three linear layers in the multilayer perceptron are R 3d×d , R dX3d , R d×2 .

[0095] The method of this implementation example is compared with five other methods: (1) Exact Name Match, if the names of the entities to be aligned are completely matched, the two entities to be aligned are considered to be matched; (2) SVM, using the character-level 4-gram similarity of the attributes such as expert name, affiliated institution, frequently occurring conference / journal name, paper title keywords and co-author name; (3) COSNET, taking the attributes of the two entities as local factors and the relationship between the entity pairs as correlation factors, considering that the two aligned entities have the same label, and using a factor graph model to propagate this label information; (4) MEgo2Vec, mining potential matching entity pairs as nodes of the matching ego network, using a multi-view node embedding method to model the literal semantic features of different attributes, and using the attention mechanism to distinguish the influence of different neighbor nodes, and adding the topological structure of the graph to obtain the regularized structural embedding; (5) LinKG, also using the information of potential matching entity pairs, but the network nodes it constructs are still entities, not entity pairs. It uses a multi-head attention mechanism based on node type to calculate the attention weights and complete the node vector update based on the relationship graph attention network. At the same time, two ablation experiments were designed: (1) This method - without reliability, only using the graph embedding learning algorithm based on feature linear modulation; (2) This method - without reinforcement learning, removing the reinforcement learning dynamic update strategy based on this method; respectively verifying the contribution of the graph embedding learning algorithm based on feature linear modulation, the reliability measurement method based on self-supervisory signals, and the node sampling quantity update strategy based on reinforcement learning to the model. Table 8 shows the performance comparison of this implementation example method and other comparison methods on the OAG dataset.

[0096] Table 8 Comparative experimental results of various methods on the OAG dataset

[0097] Model Precision Recall F1-score Exact Name Match 44.48 80.63 57.33 SVM 84.70 92.22 88.30 COSNET 91.73 85.33 88.42 MEgo2Vec 91.03 90.82 90.92 LinKG 95.37 93.48 94.42 This method - no reliability 96.74 96.86 96.80 This method - no reinforcement learning 97.18 96.82 97.00 This method 96.58 97.84 97.20

[0098] The experimental results show that this method is significantly better than the five comparison methods in all three evaluation indicators, and the F1 value is improved by at least 2.78%. Through the experimental result of this method-no reliability, the F1 value is improved by at least 2.38% compared with the five comparison methods, proving the effectiveness of the graph embedding learning algorithm based on feature linear modulation. By comparing the experimental results of this method-no reinforcement learning with this method-no reliability, the former has an improvement of 0.20% in F1 value, verifying the effectiveness of the reliability measurement method based on self-supervisory signals. By comparing the experimental results of this method with this method-no reinforcement learning, the former has an improvement of 0.20% in F1 value, verifying the effectiveness of the node sampling quantity update strategy based on reinforcement learning.

[0099] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A graph embedding expert entity alignment method based on reinforcement learning, characterized in that: The steps include: Step 1: Obtain data from two talent knowledge bases G1 = (E1, T1, R) and G2 = (E2, T2, R), where E represents an entity set, and entity types include expert entities and paper entities; R represents a relationship set; T represents the set of triples, which is a subset of E×R×E; Step 2: For each expert entity e in a talent knowledge base, generate a candidate expert set C from another talent knowledge base based on the name of the expert through the candidate entity pair generation module and the regular matching template generator method; Step 3: For each candidate expert c∈C, construct<e,c> The 2-hop heterogeneous subgraph HG=(V,H,T,R), where H1 and h2 represent the entity initial vector sets of the two talent knowledge bases respectively; the initial node vector represents h (0) : h struct =LINE(G) in Represents vector concatenation operation, h attr is the average vector of the word vectors obtained by the skip-gram model for each attribute feature of the entity, h struct It is a structural vector obtained by encoding the structural information of each entity in the knowledge base through the LINE model; Step 4: In the node vector update module, in each graph embedding layer, the reliability of each edge is calculated using the reliability measurement method based on the self-supervisory signal; Step 5: In each graph embedding layer, for the heterogeneous subgraph HG, use the top-p sampling strategy to sample p of each relationship from large to small according to the reliability calculated in step 4. r The node sampling quantity update strategy based on reinforcement learning is used to update the node sampling quantity. Step 6: In each graph embedding layer, after obtaining the sampled heterogeneous subgraph, the node vector is updated using the graph embedding learning algorithm based on feature linear modulation; Step 7: After L layers of graph embedding, take out the updated entity pairs to be aligned<e,c> The node vector and Calculate the matching probability through multi-layer perceptron Step 8: Based on the matching probabilities of all candidate entities, select the candidate expert with the highest probability as the matching expert.

2. The graph embedding expert entity alignment method based on reinforcement learning enhancement according to claim 1 is characterized in that: The method is first trained with a training set before being applied, and then applied to expert entity alignment after the training is completed; During the training process, the loss of the expert entity alignment task includes the alignment loss of all entity pairs to be aligned in addition to the average edge sampling loss based on the reliability metric method of the self-supervisory signal. The goal of this loss is to make the vector representation of the aligned expert entity pairs <z e ,z c >The higher the probability obtained after the multi-layer perceptron, the closer it is to 1, while the score of the non-aligned vector representation is close to 0.

3. The graph embedding expert entity alignment method based on reinforcement learning enhancement according to claim 1 is characterized in that: The step 2 specifically includes the following steps: Step 2-1: Process Chinese and English names separately and use the name parser to generate a parsing dictionary; Step 2-2: Use the parsing dictionary generated by the name parser as the input of the template generator and output the corresponding regular template.

4. The graph embedding expert entity alignment method based on reinforcement learning enhancement according to claim 3 is characterized in that: In the step 2-1, for English names, the surname and the given name are first parsed to determine whether the order of the surname and the given name is unique, and then a parsing dictionary is generated. For Chinese names, the Chinese names are converted into pinyin form, that is, converted into English names, while considering polyphones and compound surnames. After the intermediate parsing results are generated, they are processed the same as the English names.

5. The graph embedding expert entity alignment method based on reinforcement learning enhancement according to claim 3 is characterized in that: In the step 2-2, first determine whether the order of the surname and the given name can be determined, then determine whether the given name is an abbreviation, and finally generate 2 or 4 groups of regular templates based on the two orders of "surname first" and "surname last".

6. The method for graph embedding expert entity alignment based on reinforcement learning enhancement according to claim 1, characterized in that: In step 4, for the triple<u,r,v> , that is, there is a relationship r between node u and node v, and the direction is u pointing to v. The calculation process of its reliability S(u,v) is as follows: Step 4-1: First, calculate the feature representation η of the node on each relationship: or u ,or v =σ1(W τ(u) h u ),σ1(W τ(v) h v )) Where W τ(u) and W τ(v) Represents the weight matrix related to the corresponding node type, and its dimension is R |R|×d ,h u and h v is the node vector representation of nodes u and v in this graph embedding layer, with dimension d, and σ1(·) represents an activation function; Step 4-2: Calculate the node pair<u,v> The probability α that a relation r exists u,v : a u,v =σ2(W TF ·(or v ⊙h v )) Where W TF The dimension is R 2×|R| , ⊙ is a vector dot multiplication operation, σ2(·) represents an activation function; Step 4-3: Use Manhattan distance to calculate the unreliability D(u,v) and get the reliability S(u,v): D(u,v)=||α u,v ||1 S(u,v)=1-D(u,v) Step 4-4: In the training phase, in order to make the proposed reliability measurement method more effective in calculating the reliability of edges, the reliability measurement method based on the self-supervisory signal samples some existing relationship edges SPT={<u,r,v> |<u,r,v> ∈T}, as a positive example, and sample nodes that do not have the relationship with the target node at a ratio of 1:1 to build a non-existent relationship edge As negative examples, we construct self-supervisory signals based on existing relationship edges and use cross entropy loss as the loss function to minimize the average edge sampling loss of all graph embedding layers. Thus, the weight coefficients used in training and calculating reliability are: Where ψ(u,r,v)=1 represents the edge<u,r,v> exists, otherwise, ψ(u,r,v)=0 means that the edge does not exist.

7. The graph embedding expert entity alignment method based on reinforcement learning enhancement according to claim 1 is characterized in that: In the step 5), the node sampling quantity update strategy based on reinforcement learning specifically includes the following steps: Step 5-1: In each epoche, for relation r, calculate the sampled edges E r,sampled The average unreliability where |E| r,sampled Refers to the number of edges with relationship r among the sampled edges; Step 5-2: Update the node sampling number p of relation r according to the change trend of the average unreliability between two adjacent epochs r , the Action, Reward, and Termination in the reinforcement learning strategy used are: Action={+∈,-∈} Where ∈ is a small fixed integer; if the average unreliability of the current epoch is less than that of the previous epoch, the reward function is positive, and the instant reward method is used to greedily increase p r , otherwise, reduce p r .

8. The graph embedding expert entity alignment method based on reinforcement learning enhancement according to claim 1 is characterized in that: The step 6 is specifically as follows: At the lth layer, for node v, its node vector is Updated to and in is the message conversion function weight, σ is the ReLU function, They are the message weights of each neighbor node u calculated by node v in the lth graph embedding layer: Where g(h v θ g,l,r ) is a hypernetwork implemented using a single linear layer, θ g,l,r ∈R 2d×d are the parameters of this hypernetwork.

Images