A Differentiable Method for Mining Constant Rules

Abstract

The present invention discloses a method for differentiable mining constant rules. On the basis of defining constant operators and path operators according to knowledge graphs, a fusion attention mechanism is adopted, and the relationship through which the rule passes is evaluated by using the attention value. At the same time, the surrounding attributes and corresponding attribute values ​​of the tail node of each hop are aggregated and the attention is calculated respectively, and the selection of the relationship in the rule is enhanced by the attention of the attribute, so as to realize link prediction; according to the generated model parameters, the parameters are parsed and high-quality symbolized rules are output through statistics. This method is particularly suitable for application scenarios with complex reasoning requirements that require high prediction accuracy, provide explanations for predictions, and need to precipitate reasoning rules.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graph reasoning, and in particular to a method for differentiable mining constant rules. Background Art

[0002] Traditional knowledge graph embedding representation learning methods mostly input triples containing head entities, relations, and tail entities, and calculate triple scores and optimize model embedding vectors through certain assumptions such as h+r=t. The calculation process is one-step, and the connection prediction results only give the tail entity without steps. Therefore, most of them are black box models with poor interpretability. For example, the Chinese patent document with publication number CN113377968A discloses a knowledge graph link prediction method using fused entity context.

[0003] Traditional knowledge graph rule mining models are mostly based on search and matching on graphs. Rule mining is a search and statistical process rather than a computational process. The search space on large knowledge graphs increases dramatically, and so does the time and hardware resources consumed. At the same time, they only consider the association between relationships, and do not notice that the attributes connected by the intermediate entities will also affect the rules themselves. For example, the Chinese patent document with publication number CN111126828A discloses a multi-layer abnormal fund flow monitoring method based on knowledge graphs.

[0004] At the same time, the traditional knowledge graph embedding representation learning model can only complete the prediction task, and the reasoning process is a black box. It cannot provide an explanation for the prediction results, nor can it parse the prediction rules. The traditional rule mining model aims to produce symbolic rules, but it cannot constrain the corresponding attributes of the nodes. For example, the relationship between a person's parents, if the gender is male, it corresponds to the father relationship, and if the gender is female, it corresponds to the mother relationship. Considering the attributes, the relationship itself can be fine-grained.

[0005] In the current demand, knowledge graph reasoning not only needs high accuracy, but also requires algorithms to provide explanations for prediction results and generate symbolic rules. This can not only help experts judge the rationality of prediction results and discover currently undiscovered relationships, but also avoid recalculation and multiple development due to the addition of new entities, thereby broadening the scope of application of the model. Summary of the invention

[0006] In view of the above, the present invention provides a differentiable method for mining constant rules, which is suitable for prediction application scenarios that need to consider constants to improve accuracy. The method also provides an explanation of symbolic rules to avoid recalculation due to new entities, thereby reducing computational overhead.

[0007] The technical solution provided by the embodiment is:

[0008] A method for differentiable mining of constant rules, comprising the following steps:

[0009] Step 1: Obtain the knowledge graph, build a path operator for each relationship based on the knowledge graph, and represent the connection of each relationship in the entity. Build a constant operator for each combination of relationship and entity based on the knowledge graph, and represent all possible constants connected to the entity through the relationship.

[0010] Step 2: After initializing the target relationship of the target triple as a relationship vector, the relationship vector is repeatedly input into the bidirectional long short-term memory model 2T times according to the expected rule length T to obtain a vector sequence of 2T fixed lengths;

[0011] Step 3: extract T fixed-length vector sequences for first attention mechanism processing to obtain the first attention parameter of the path operator during T-step rule extraction, extract the remaining T fixed-length vector sequences for second attention mechanism processing to obtain the second attention parameter of the constant operator during T-step rule extraction;

[0012] Step 4: For each step of rule extraction, first expand the rule path according to the path operator, that is, predict the node entity according to the path operator and the corresponding first attention parameter, and then judge whether the rule needs a constant according to the constant operator, that is, take the predicted node entity as the benchmark, use the constant operator and the corresponding second attention parameter to predict the node entity considering the constant;

[0013] Step 5, generating multiple rules for each target relationship based on the first attention parameter and the second attention parameter and the predicted intermediate parameters in the rule extraction process of step 4;

[0014] Step 6: Screen all rules of multiple target relations to determine the final extracted rules and output them.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The method for mining constant rules in a differentiable manner provided in the embodiment is currently the only technical solution that can simultaneously have a high prediction accuracy, provide the ability to interpret constraint constants, and be parsed into symbolic rules. The method adopts a fusion attention mechanism, uses attention values ​​to evaluate the relationship through which the rule passes, and simultaneously calculates attention for the surrounding attributes and corresponding attribute values ​​of the tail node of each step, and uses the attention of the attributes to enhance the selection of relationships in the rule. It is particularly suitable for application scenarios that need to constrain constants and can be parsed into symbolic rules, require high prediction accuracy, and have complex reasoning requirements. For example, for product attributes in e-commerce scenarios, if rules with constants can be parsed according to the attributes, the recommendation scenario of each product can be directly inferred based on the rules, eliminating the step of manual marking. At the same time, for new users with no shopping or search records, specific products can be directly recommended based on the attributes in their user portraits to avoid cold start problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] In other words, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 is a flow chart of a method for differentiable mining constant rules provided by an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of a process of judging whether a rule requires a constant according to a constant operator provided by an embodiment of the present invention;

[0021] Figure 3 It is a statistical analysis of the number of rules for parsing different data sets provided by the embodiments of the present invention.

[0022] Figure 4 This is a specific example of rule parsing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0024] Figure 1FIG. 1 is a flow chart of a method for differentiable mining constant rules provided by an embodiment of the present invention. Figure 1 As shown, the method for differentiable mining constant rules provided by the embodiment includes the following steps:

[0025] Step 1: Obtain the knowledge graph and construct path operators and constant operators based on the knowledge graph.

[0026] In the embodiment, the triple information (h, r, t) of the acquired knowledge graph is used to form a relationship set R and an entity set E. According to the relationship set R and the entity set E, for each relationship r i To facilitate the construction of path operators, the knowledge graph is constructed into a dictionary with relations as keys and tail nodes corresponding to the head nodes as keys. i The constructed path operator represents the adjacency of each relationship in the entity, that is, the adjacency matrix, which is used to expand the overall length of the rule. The path operator is expressed as The element value in is 1 or 0, where 1 indicates that there is a relationship between two entities. i , 0 means there is no relationship between the two entities i ,It should be noted that a special path operator with 1 on the diagonal is also defined to represent the relationship from an entity to itself, which is generally not directly and explicitly described in the knowledge graph. Therefore, there are a total of |R|+1 path operators.

[0027] In the embodiment, a constant operator is constructed for each combination of a relationship and an entity based on the knowledge graph to represent other possible constant situations connected to the entity through the relationship. The constant operator is used to define the constant constraints required for the intermediate steps of the rule. In the embodiment, the constant is the entity. In order to distinguish and facilitate description, it is called a constant. i and the kth entity e k The constant operator constructed by the combination of Each element value is 1 or 0, 1 represents entity e k Through the relationship i Connected with other possible constants e, 0 represents entity e k Through the relationship i No other possible constant e connected to it can be found. Since there are |E|*|R| such combinations in the entire knowledge graph, there are a total of |E|*|R| constant operators.

[0028] Step 2: Use the bidirectional long short-term memory model to extract the vector sequence of the relationship in the target triple.

[0029] In the embodiment, an expected rule length T is preset, and the relationship vector of the target triple is randomly initialized. The relationship vector corresponds to all relationships one by one, that is, the dimension of the relationship vector is equal to the number of relationships in the relationship set R.

[0030] According to the expected rule length T, the relation vector is repeatedly input 2T times into the BiLSTM model to obtain a vector sequence of 2T fixed lengths. The specific process is as follows:

[0031] h0,h′ 2T =BiLSTM(r)

[0032] h t ,h′ 2T-t-1 =BiLSTM(r,h t-1 ,h′ 2T-t ),t>1

[0033] Among them, t is the step index, r is the relationship vector, h represents the vector sequence obtained by forward calculation, and h′ represents the vector sequence obtained by reverse calculation.

[0034] Step 3: Perform attention mechanism processing based on the vector sequence extracted from the bidirectional long short-term memory model to obtain the attention parameters corresponding to the path operator and the constant operator.

[0035] In the embodiment, T vector sequences of fixed length are extracted for first attention mechanism processing to obtain first attention parameters of path operators during T-step rule extraction, and the remaining T vector sequences of fixed length are extracted for second attention mechanism processing to obtain second attention parameters of constant operators during T-step rule extraction.

[0036] In one possible implementation, a fully connected layer is used to perform an attention mechanism on the vector sequence. The specific process is as follows:

[0037] α t =Softmax(W a (h t +h′ t )+b a ),t=1,3,...,2T-1

[0038] β t =Softmax(W b (h t +h′ t )+b b ),t=2,4,...,2T

[0039] Among them, t is the index of the step number, α t represents the first attention parameter of the path operator, β trepresents the second attention parameter of the constant operator, h t , h′ t They represent the two vector sequences output by the bidirectional long short-term memory model at the tth step, W a 、b a They represent the weight parameter and bias parameter in the first attention processing, W b 、b b They represent the weight parameter and bias parameter in the second attention processing, and Softmax(·) is the Softmax function. a 、b a and W b 、b b It needs to be optimized. The above formula shows that the first attention parameter α of the path operator is calculated by extracting the vector sequence of odd steps t , extract the vector sequence of even steps to calculate the second attention parameter β of the constant operator t .

[0040] Step 4: Perform rule inference based on the first attention parameter, the second attention parameter, the path operator, and the constant operator.

[0041] In the embodiment, for the rules with a maximum length of T, each step of rule extraction needs to perform two operations using a path operator and a rule operator, including: first, expanding the rule path according to the path operator, that is, predicting the node entity according to the path operator and the corresponding first attention parameter, and then judging whether the rule requires a constant according to the constant operator, that is, taking the predicted node entity as a benchmark, using the constant operator and the corresponding second attention parameter to perform node entity prediction considering the constant.

[0042] In a possible implementation, the process of expanding the rule path according to the path operator includes:

[0043] Predict the node entity based on the path operator and the corresponding first attention parameter:

[0044]

[0045] Among them, z t-1 represents the node entity considering constant prediction at step t-1, Represents the i-th relation r i The path operator, Indicates that at step t, the relationship r i The first attention parameter of the path operator, z t ′ represents the expansion result of the rule path, that is, the predicted node entity, For the prediction relationship, when t = 1, z t-1is the target head entity vector in the target triplet.

[0046] The expansion process of the rule path is essentially through the path operator Find the predicted result z t-1 Neighbor entity of , because the path operator The first attention parameter different, The weights of the neighbor entities formed will also be different. The first attention parameter The weights of neighbor entities are constructed by weighted summation, which fully considers all the relationships in the knowledge graph, realizes the expansion of the rule path, and can obtain multiple paths for rule extraction.

[0047] In one possible implementation, Figure 2 As shown, the process of judging whether a rule requires a constant according to the constant operator includes:

[0048] First, for the i-th relation r i The corresponding constant operator u ik , and compare it with the node entity prediction result z of step t t ′ multiplied by the third attention parameter to obtain a constant Right now

[0049] The third attention parameter, as the attention of the constant, focuses on the degree of association between the node entity and the constant at each step, and is used to select the constant and implement the restriction on the constant.

[0050] Then, using the third attention parameter For the i-th relation r i All corresponding constant operators u ik Perform aggregate weighted calculations to obtain aggregate constant operators Right now:

[0051]

[0052] Here, k is the index of the constant. The purpose of Scale(·) is to control each value of the weighted vector between (0, 1). The constant aggregation process is used to screen relatively strong constant relationships.

[0053] Finally, based on the node entity prediction result z t ′, the second attention parameter according to the constant operator and aggregate constant operators Make node entity predictions taking into account constants, namely:

[0054]

[0055] Among them, z t represents the node entity considering the constant, ⊙ represents the Haddam product, that is, the bitwise multiplication of two vectors, and when i=0, Set to a vector of all 1s, indicating that no constants need to be considered in this step.

[0056] Because the maximum rule length is T, the above steps 3-4 need to be repeated T times to get the constant prediction result z from the initial entity vector h T , this constant predicts the result z T It is the inferred probability distribution of the entity, and the value of each position represents the possibility of the entity at the corresponding position.

[0057] It should be noted that the bidirectional long short-term memory model in step 2 and the attention mechanism in step 3 need to be optimized before mining constant rules. The specific parameter optimization process is as follows:

[0058] For the sample triple (h, r, t), after initializing the vectors of entities and relations in the sample triple, the constant prediction result is obtained by using the process of steps 2 to 4, and then the prediction result z is used. T The evaluation score score(t|h,r) of the inference result is calculated with the actual tail entity vector t in the sample triple, that is:

[0059] score(t|h,r)=t·log[z T ,τ] +

[0060] Among them, τ represents the set filtering threshold, z T The values ​​less than τ are filtered out, the sum of the evaluation scores of all sample triplets in a training batch is used as the loss function, and the parameters are optimized with the maximum loss function as the goal.

[0061] Step 5, parse and generate multiple rules for each target relationship based on the first attention parameter, the second attention parameter, and the third attention parameter.

[0062] After completing the rule extraction task, the first attention parameter, the second attention parameter, and the third attention parameter need to be parsed to generate symbolic rules. As mentioned above, α is used to select the path, β is used to select the constant relationship, and b is used to select the specific constant. Based on this, the process of parsing and generating multiple rules for each target relationship is as follows:

[0063] For each step of rule extraction, the relations corresponding to values ​​greater than the first threshold are screened from the first attention parameter according to the preset first threshold as the relations determined in the current step; the relations corresponding to values ​​greater than the second threshold are screened from the second attention parameter according to the preset second threshold as the constant relations determined in the current step; the constants corresponding to values ​​greater than the third threshold are screened from the third attention parameter according to the preset third threshold as the constants determined in the current step; in this way, all possible specific paths (formed by relations and constant relations in step order) and constraint constants in the target triplet are obtained. Note that there may be multiple rules that can deduce results for one path. Therefore, all the relations, constant relations and constants determined in the steps form multiple rules in step order.

[0064] In the embodiment, for the target triple, the path interpretation of the formed length T is expressed as:

[0065] r 1 ∧r 2 …∧r T →r

[0066] Each r i Represented as r i ,attribute j (value jz ) Among them, attribute j (·) represents the constant relationship between the node entity and the constant, j is the index of the constant relationship, and value jz Indicates that the constant relationship attribute j A constant is determined, z is the index of the constant, that is, the previous hop is connected to the next hop node entity through the relationship r, and the node entity is required to have the attribute attribute j , and the attribute has the attribute value value jz , where value is a constant, and it needs to be a definite entity in the rule.

[0067] Step 6: Screen multiple rules of multiple target relations to determine the final extraction rules and output them.

[0068] For each target triple, multiple rules are generated using the process of step 2 to step 5. Finally, all the rules parsed as a whole are counted, and the rules that are used more times in the reasoning process of all triples are selected as the final extraction rules.

[0069] In the embodiment, the process of screening multiple rules of multiple target relationships to determine the final extraction rules and outputting them is as follows:

[0070] For each rule of all target relationships, the confidence is calculated according to the first attention parameter, the second attention parameter, and the third attention parameter of each step, and the rules with high confidence are selected as the final extraction rules. Preferably, the rules with confidence higher than the set threshold can be selected as the rules with higher occurrence times and output.

[0071] The confidence calculation formula for each rule is:

[0072]

[0073] Among them, i is the step index, T is the preset rule length, that is, the number of steps, α ir is the first attention parameter of relation r in the path operator at step i, β ir is the second attention parameter of the constant relation a in the constant operator at step i, u ia It is the average value of the third attention parameters of all constants exceeding the hyperparameter threshold under the constant relationship a at the i-th step.

[0074] The above-mentioned differentiable method for mining constant rules can firstly use neural networks to infer triples, and secondly can parse symbolic rules from model parameters. Finally, the overall framework is carried out in a differentiable way, providing a basis for reasoning using neural networks.

[0075] Different from the previous rule learning reasoning method that performs differentiable reasoning on simpler chain rules, the method provided by the embodiment of the present invention considers a more complex rule form, introduces constants into the rule as a constraint, uses attention values ​​to simulate the overall rule path and generate explanations, and can generate at least one rule for each triple prediction. At the same time, the framework of this method is also backward compatible with rule forms that do not contain constants. This method is particularly suitable for application scenarios with complex reasoning requirements that require higher prediction accuracy, provide explanations for predictions, and require precipitation of reasoning rules.

[0076] The embodiment provides a differentiable method for mining constant rules, which is suitable for prediction application scenarios that need to consider constants to improve accuracy. When applied in e-commerce scenarios, the knowledge graph corresponds to a product knowledge graph suitable for e-commerce scenarios, wherein entities represent products and attributes, and relationships represent relationships between products and attributes. The goal of rule mining is to find out which scenarios or groups of people products with certain attributes are suitable for. Product attributes in e-commerce scenarios have a strong effect on rule reasoning, and the attribute values ​​in this scenario can be regarded as constants. For example, it can be considered that a bag with genuine leather attributes is more suitable for business scenarios. Moreover, there are few types of relationships between different products in e-commerce scenarios, but there are many attributes that can be regarded as constants. As a result, in this scenario, the application value of the rules with constants defined by the present invention is far greater than the rule form defined by the traditional method.

[0077] In order to verify the technical effect of the method for mining constant rules that can be differentiable provided by the above embodiment, the following specific experiments were conducted:

[0078] During the training process, the Adam optimizer was used for optimization, and 4 training iterations were performed. The batch size of each iteration was set to 14, the dimensions of BiLSTM and the fully connected function were set to 256, and the learning rate was set to 0.0001. The performance of the model was evaluated using MRR, Hit@10, Hit@3, and Hit@1, and entities that were correct other than the predicted entity were excluded from the scoring range.

[0079] Among them, MRR represents the reciprocal mean of the correct entity ranking in the prediction result probability. Hit@10, Hit@3, and Hit@1 represent the probability of the correct entity ranking in the top ten, top three, and first in the prediction result probability.

[0080] The connection prediction results targeting the FB15K-237 dataset are shown in Table 1 below:

[0081] Table 1

[0082]

[0083]

[0084] Among them, the ConvE model is from "Convolutional 2D Knowledge Graph Embeddings", the ComplEx model is from "Complex Embeddings for Simple Link Prediction", the DistMult model is from "Embedding Entities and Relations for Learning and Inference in KnowledgeBases", the RotatE model is from "RotatE: Knowledge Graph Embedding by Relational Rotationin Complex Space", the Neural-LP model is from "Differentiable Learning of Logical Rulesfor Knowledge Base Reasoning", and the DRUM model is from "DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs". By analyzing Table 1, it can be obtained that in all the models, the method provided in the embodiment of the present invention has higher values ​​of the four parameters of MRR, Hit@10, Hit@3 and Hit@1 than other models, indicating that the method provided in the embodiment of the present invention has a higher prediction accuracy.

[0085] In order to verify that the differentiable method for mining constant rules provided by the embodiment of the present invention can provide explanation capabilities, the quality and quantity of the explanations provided by the model are evaluated. The rules mined on four datasets, Constant, Family-gender, UMLS and FB15K-237, are evaluated as follows: Figure 3 As shown, Figure 3 In the figure, the horizontal axis represents the confidence interval of the mined rules, while the vertical axis represents the number of rules within the confidence interval. Figure 3 It can be concluded that the method provided in the embodiment of the present invention mines a large number of rules in a high confidence interval on each data set, indicating that an appropriate number of rules can be mined.

[0086] In order to evaluate the quality of the rules, the embodiment uses standard confidence to evaluate the quality. The standard confidence represents the probability that the head relationship does exist when the rule body is satisfied. The standard confidence improvement represents the confidence without considering the constant in the rule and the standard confidence improvement when considering the constant. Taking the UMLS dataset related to medical and biological concepts as an example, Figure 4 The following are examples of rule results on the UMLS dataset. Figure 4 In the example, circles with C, C1, and C2 represent constant entities predicted by constant operators, and the remaining circles represent node entities predicted by path operators. Solid arrows represent the relationship between two entities, and hollow arrows and Uses represent rules obtained through multiple relationships and multiple entities. Figure 4 It can be seen that the standard confidence SC is improved. For example, for the first rule, by predicting the constant entity, the standard confidence SC of the rule is improved from 0.818 to 1.00, which fully shows that the method provided in the embodiment of the present invention can improve the quality of the rule.

[0087] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A differentiable method for mining constant rules, characterized in that: The application in the e-commerce scenario includes the following steps: Step 1: Obtain a knowledge graph. The knowledge graph corresponds to a commodity knowledge graph applicable to e-commerce scenarios, where entities represent commodities and attributes, and relationships represent the relationships between commodities and attributes. The goal of rule mining is to find out which scenarios or groups commodities with certain attributes are suitable for. Based on the knowledge graph, a path operator is constructed for each relationship to indicate the connection of each relationship in the entity. Based on the knowledge graph, a constant operator is constructed for each combination of a relationship and an entity to indicate all possible constants connected to the entity through the relationship. Among them, the triple information of the acquired knowledge graph forms a relationship set R and an entity set E. According to the relationship set R and the entity set E, for each relationship r i The constructed path operator is represented as The element value in is 1 or 0, where 1 indicates that there is a relationship between two entities. i , 0 means there is no relationship between the two entities i , we also define a special path operator with 1 on the diagonal, which represents the relationship from an entity to itself. There are a total of |R|+1 path operators. For a relationship r i and the kth entity e k The constant operator constructed by the combination of Each element value is 1 or 0, 1 represents entity e k Through the relationship i Connected with other possible constants e, 0 represents entity e k Through the relationship i No other possible constant e can be found connected to it, there are a total of |E|*|R| constant operators; Step 2: After initializing the target relationship of the target triple as a relationship vector, the relationship vector is repeatedly input into the bidirectional long short-term memory model 2T times according to the expected rule length T to obtain a vector sequence of 2T fixed lengths; Step 3: extract T fixed-length vector sequences for the first attention mechanism processing, obtain the first attention parameter of the path operator when extracting T-step rules, extract the remaining T fixed-length vector sequences for the second attention mechanism processing, and obtain the second attention parameter of the constant operator when extracting T-step rules; specifically, the fully connected layer is used to process the vector sequence with the attention mechanism, and the specific process is as follows: α t =Softmax(W a (h t +h ′ t )+b a ),t=1,3,...,2T-1 β t =Softmax(W b (h t +h ′ t )+b b ),t=2,4,...,2T Among them, t is the index of the step number, α t represents the first attention parameter of the path operator, β t represents the second attention parameter of the constant operator, h t 、h ′ t They represent the two vector sequences output by the bidirectional long short-term memory model at the tth step, W a 、b a They represent the weight parameter and bias parameter in the first attention processing, W b 、b b They represent the weight parameter and bias parameter in the second attention processing, respectively. Softmax(·) is the Softmax function. Step 4: For each step of rule extraction, first expand the rule path according to the path operator, that is, predict the node entity according to the path operator and the corresponding first attention parameter: Among them, z t-1 represents the node entity considering constant prediction at step t-1, Represents the i-th relation r i The path operator, Indicates that at step t, the relationship r i The first attention parameter of the path operator, z t ′ Represents the expansion result of the rule path, that is, the predicted node entity, For the prediction relationship, when t = 1, z t-1 is the target head entity vector in the target triplet; According to the constant operator, it is judged whether the rule needs a constant, that is, based on the predicted node entity, the constant operator and the corresponding second attention parameter are used to predict the node entity considering the constant, specifically: First, for the i-th relation r i The corresponding constant operator u ik , and compare it with the node entity prediction result z of step t t ′ Multiply them together to get the third attention parameter of the constant Right now Then, using the third attention parameter For the i-th relation r i All corresponding constant operators u ik Perform aggregate weighted calculations to obtain aggregate constant operators Right now: Where k is the index of the constant, and the purpose of Scale(·) is to control each value of the weighted vector between (0,1); Finally, based on the node entity prediction result z t ′ , according to the second attention parameter of the constant operator and aggregate constant operators Perform node entity prediction considering constants, namely: Among them, z t represents the node entity considering the constant, ⊙ represents the Haddam product, that is, the bitwise multiplication of two vectors, and when i=0, Set it to a vector of all 1s, indicating that no constants need to be considered in this step; Step 5: Analyze and generate multiple rules for each target relationship based on the first attention parameter and the second attention parameter and the predicted intermediate parameters in the rule extraction process of step 4, specifically: For each step of rule extraction, a relationship whose value is greater than the first threshold is selected from the first attention parameter according to a preset first threshold as the relationship determined in the current step, a relationship whose value is greater than the second threshold is selected from the second attention parameter according to a preset second threshold as the constant relationship determined in the current step, and a constant whose value is greater than the third threshold is selected from the third attention parameter according to a preset third threshold as the constant determined in the current step; all the relationships, constant relationships and constants determined in the steps form multiple rules in the order of the steps; Step 6: Screen all rules of multiple target relations to determine the final extracted rules and output them.

2. The method for mining constant rules according to claim 1, characterized in that: The bidirectional long short-term memory model in step 2 and the attention mechanism in step 3 need to be optimized before mining constant rules. The specific parameter optimization process is as follows: For the sample triple (h, r, t), after initializing the vectors of entities and relations in the sample triple, the constant prediction result is obtained by using the process of steps 2 to 4, and then the prediction result z is used. T The evaluation score score(t|h,r) of the inference result is calculated with the actual tail entity vector t in the sample triple, that is: score(t|h,r)=t·log[z T ,τ] + Among them, τ represents the set filtering threshold, z T The values ​​less than τ are filtered out, the sum of the evaluation scores of all sample triplets is used as the loss function, and the parameters are optimized with the maximum loss function as the goal.

3. The method for mining constant rules according to claim 1, characterized in that: In step 6, the process of screening multiple rules of multiple target relations to determine the final extracted rules and output is as follows: For each rule of all target relationships, the confidence is calculated based on the first attention parameter, the second attention parameter, and the third attention parameter of each step, and the rules with high confidence are selected as the final extraction rules.

4. The method for mining differentiable constant rules according to claim 3, characterized in that: The confidence calculation formula for each rule is: Among them, i is the step index, T is the preset rule length, that is, the number of steps, α ir is the first attention parameter of relation r in the path operator at step i, β ir is the second attention parameter of the constant relation a in the constant operator at step i, u ia It is the average value of the third attention parameters of all constants exceeding the hyperparameter threshold under the constant relationship a at the i-th step.

Images