A relationship prediction model training method, an application method, an apparatus, and a device
By constructing and optimizing sample triplet information and negative sample information, and using a convolutional scoring function to train a relationship prediction model, the problem of high computational resource efficiency in knowledge graph embedding models is solved, and efficient many-to-many relationship prediction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2023-06-28
- Publication Date
- 2026-05-19
AI Technical Summary
Existing knowledge graph embedding models have high computational resource requirements and low computational efficiency in relation prediction, which cannot meet the needs of large-scale applications.
By constructing sample triplet information and negative sample information, and using the convolutional scoring function to iteratively optimize the training vectors, the model complexity is controlled, and a relationship prediction model is formed.
It reduces the demand for computing resources, improves computing efficiency, and can handle many-to-many relationship prediction, thus broadening the application scope of knowledge graph embedding models.
Smart Images
Figure CN116757277B_ABST
Abstract
Description
Technical Field
[0001] The embodiments in this specification relate to the field of data processing technology and are applicable to the financial field, such as banking applications. In particular, they relate to a method for training a relationship prediction model, an application method, an apparatus, and a device. Background Technology
[0002] In the mesh topology of a knowledge graph, each node represents an entity with its own text description. It can refer to real people, things, or events, such as "teacher" or "school," or it can refer to abstract concepts such as "language" or "science." Each edge represents a relationship, including symmetrical relationships such as "is a classmate of..." and antisymmetric relationships such as "younger than...".
[0003] The topological structure of a knowledge graph can be represented by a triple (head entity, relation, tail entity), abbreviated as (h, r, t). This triple represents a relationship between a head entity and a tail entity, and is the smallest unit used to represent relationships between multiple entities, also known as a fact. A complete knowledge graph consists of a large number of facts, and from a set perspective, it can also be viewed as a set of all triples. Each node in a knowledge graph contains its own descriptive information, but this self-descriptive information is not part of the topological structure.
[0004] However, knowledge graphs only serve the functions of knowledge structure modeling and knowledge storage, and cannot be applied to large-scale practical applications that require automated computer recognition or prediction. Current knowledge graph embedding models are broadly classified into two categories: translational distance models and semantic matching models. Translational distance models utilize vector space norm-based models, measuring the probability of a fact's validity by defining the distance between entities. These models are highly intuitive, with relatively simple scoring functions and high computational efficiency. Semantic matching models, based on the concept of similarity, measure the probability of a triplet fact's existence by matching the latent semantics of entities and relationships in the embedding space. These models are further divided into bilinear models and neural network models. For the same model, more parameters result in greater computational cost and more relationships that can be simulated, but they also place higher demands on computational resources and have lower computational efficiency.
[0005] There is an urgent need for a training and application method for relationship prediction models to solve the problems of high computational resource requirements and low computational efficiency of existing relationship prediction methods based on knowledge graph embedding models. Summary of the Invention
[0006] To address the problems in existing technologies, embodiments of this specification provide a method, application method, apparatus, and device for training a relation prediction model. This method controls model complexity by manipulating the dimension of the recurrent convolution, thereby reducing computational resource requirements and improving computational efficiency. Furthermore, it can map many-to-many relationships in knowledge graphs, broadening the application scope of knowledge graph embedding models.
[0007] To solve the above-mentioned technical problems, the specific technical solution in this specification is as follows:
[0008] On the one hand, the embodiments of this specification provide a method for training a relationship prediction model, including:
[0009] Based on entities in the sample knowledge graph and the connection relationships between those entities, multiple sample triplet information is constructed.
[0010] For the multiple sample triplet information, multiple negative sample information and a weight corresponding to each negative sample information are created;
[0011] The convolutional scoring function is used to process the information of the multiple sample triples and the multiple negative sample information to obtain the corresponding scoring function value;
[0012] Based on the scoring function value and the weight, the training vectors corresponding to the entity and the connection relationship are iteratively optimized until the target vector that meets the preset conditions is obtained.
[0013] The convolutional scoring function is used to process the target vector corresponding to each sample triplet and each negative sample to obtain threshold data. The convolutional scoring function and the target vector are then used to construct a trained relation prediction model based on the threshold data.
[0014] Furthermore, the step of creating multiple negative sample information and a weight corresponding to each negative sample information for the multiple sample triple information includes:
[0015] Select multiple target sample triplet information from the multiple sample triplet information;
[0016] For each of the target sample triplet information, create multiple corresponding negative sample information;
[0017] The weight corresponding to the negative sample information is determined based on the correlation between the negative sample information and the target sample triple information.
[0018] Furthermore, for each of the target sample triplet information, creating corresponding multiple negative sample information includes:
[0019] The target sample triplet information is determined to include two target entities and the target connection relationship between the two target entities;
[0020] For any one of the two target entities, determine a candidate entity from the entity that does not have the target connection relationship with the other target entity;
[0021] The candidate entity is used to replace the target entity, so that it, together with the other target entity of the two target entities and the target connection relationship, constitutes the negative sample information.
[0022] Furthermore, the scoring function value includes a first sub-scoring function value and a second sub-scoring function value;
[0023] The process of using a convolutional scoring function to process the information of the multiple sample triples and the multiple negative sample information to obtain the corresponding scoring function value includes:
[0024] For each entity and each connection relationship, a training vector is determined separately;
[0025] For each of the sample triplet information, the convolutional scoring function is used to process the training vector corresponding to the sample triplet information to obtain the first sub-scoring function value;
[0026] For each negative sample information, the training vector corresponding to the negative sample information is processed using a convolutional scoring function to obtain the second sub-scoring function value.
[0027] Further, the iterative optimization of the training vectors corresponding to the entity and the connection relationship, based on the scoring function value and the weights, until a target vector satisfying the preset conditions is obtained includes:
[0028] The loss function is used to process the first sub-scoring function value, the second sub-scoring function value, and the weight to obtain the loss function value;
[0029] Based on the optimization algorithm and the loss function value, the training vectors corresponding to the entities and the connection relationships are optimized to obtain optimized vectors.
[0030] The optimized vector is used to replace the training vector for iteration until the preset condition is met, thus obtaining the target vector.
[0031] Furthermore, the convolution scoring function includes:
[0032]
[0033] Among them, fr (h,t) represents the scoring function value corresponding to (h,r,t), where h and t represent the training vector or target vector corresponding to the two entities included in the sample triplet information, and r represents the training vector or target vector corresponding to the connection relationship included in the sample triplet information. F represents the circular convolution operation, and F represents the L1 norm or L2 norm.
[0034] Furthermore, the loss function includes:
[0035]
[0036] Where L represents the loss function, log represents the logarithmic loss function, σ represents the logistic function, γ represents the training strictness controlling the information of the sample triplets and the negative sample information, (h' i ,r,t' i f represents the training vector corresponding to the i-th negative sample information corresponding to (h,r,t), and (h,r,t) represents the training vector corresponding to the sample triplet information. r (h,t) represents the value of the first sub-scoring function corresponding to (h,r,t), f r (h ’ i ,t ’ i ) characterization and the (h) ’ i ,r,t i ’ The second sub-scoring function value corresponding to ) is p(h ’ i ,r,t ’ i ) characterization and the (h) ’ i ,r,t ’ i The weights corresponding to the sample triplet information are denoted by n, where n represents the total number of negative sample information corresponding to the sample triplet information.
[0037] Based on the same inventive concept, embodiments of this specification also provide a method for applying a relationship prediction model, including:
[0038] For the received triplet to be predicted, determine the processing vector from the target vector that corresponds to the entity in the triplet to be predicted and the connection relationship between the entity;
[0039] The target value is obtained by processing the vector to be processed using a convolutional scoring function;
[0040] Determine whether the target value is less than the threshold data;
[0041] If the target value is determined to be less than the threshold value, the connection relationship is determined to be valid.
[0042] The target vector and the threshold data are determined based on the relationship prediction model training method described above.
[0043] On the other hand, embodiments of this specification also provide a relationship prediction model training apparatus, including:
[0044] The sample triplet information construction unit is used to construct multiple sample triplet information based on entities in the sample knowledge graph and the connection relationships between the entities;
[0045] A creation unit is used to create multiple negative sample information and a weight corresponding to each of the multiple sample triplet information;
[0046] The convolution scoring calculation unit is used to process the multiple sample triplet information and the multiple negative sample information using the convolution scoring function to obtain the corresponding scoring function value;
[0047] An optimization unit is used to iteratively optimize the training vectors corresponding to the entity and the connection relationship based on the loss function value obtained from the scoring function value and the weight, until a target vector that meets the preset conditions is obtained.
[0048] The relation prediction model determination unit is used to process the target vector corresponding to each sample triplet information and each negative sample information using the convolutional scoring function to obtain threshold data, and to construct a trained relation prediction model based on the convolutional scoring function and the target vector using the threshold data.
[0049] Based on the same inventive concept, embodiments of this specification also provide a relation prediction model application device, including:
[0050] The vector determination unit is used to determine, for example, a target vector from a target vector corresponding to the entity in the triplet to be predicted and the connection relationship between the entity;
[0051] The convolution scoring calculation unit is used to process the vector to be processed using the convolution scoring function to obtain the target value;
[0052] The judgment unit is used to determine whether the target value is less than the threshold data;
[0053] The connection relationship determination unit is used to determine that the connection relationship is valid when the target value is less than the threshold value.
[0054] The target vector and the threshold data are determined based on the relationship prediction model training device described above.
[0055] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.
[0056] On the other hand, embodiments of this specification also provide a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-described method.
[0057] On the other hand, embodiments of this specification also provide a computer program product, including a computer program / instructions, and a method implemented when the computer program / instructions are executed by a processor.
[0058] Using the embodiments in this specification, sample triple information is first constructed based on entities and connections between entities in the sample knowledge graph. Then, corresponding negative sample information and weights are created. A convolutional scoring function is then used to process the sample triple information and negative sample information to obtain corresponding scoring function values. Based on the scoring function values and weights, the training vectors corresponding to entities and connections are iteratively optimized until the training vectors meet preset conditions and are used as target vectors. Finally, the convolutional scoring function is used to process the target vectors corresponding to each sample triple information and each negative sample information to obtain threshold data. The convolutional scoring function, target vectors, and threshold data constitute the trained relationship prediction model. Through the convolutional scoring function, prediction of one-to-many, many-to-one, and many-to-many relationships is achieved, improving prediction efficiency and solving the problems of high computational resource requirements and low computational efficiency in existing relationship prediction methods based on knowledge graph embedding models. Attached Figure Description
[0059] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0060] Figure 1 The figure shown is a schematic diagram of an implementation system for a relationship prediction model training method in an embodiment of this specification;
[0061] Figure 2 The diagram shown is a flowchart of a relationship prediction model training method in an embodiment of this specification.
[0062] Figure 3 The diagram illustrates the steps in this embodiment of the invention to create multiple negative sample information and the weights corresponding to each negative sample information for multiple sample triplet information.
[0063] Figure 4 The diagram illustrates the steps in an embodiment of this specification for creating multiple corresponding negative sample information for each target sample triplet information.
[0064] Figure 5 The figure shows the steps in this embodiment of the specification to process multiple sample triplet information and multiple negative sample information using a convolutional scoring function to obtain the corresponding scoring function value;
[0065] Figure 6 The diagram shows the steps for iterative optimization of the training vectors in the embodiments of this specification;
[0066] Figure 7 The diagram shown is a structural schematic of a relationship prediction model training device according to an embodiment of this specification.
[0067] Figure 8 The diagram shown is a flowchart of an application method of a relationship prediction model in an embodiment of this specification.
[0068] Figure 9 The diagram shown is a structural schematic of a relation prediction model application device according to an embodiment of this specification.
[0069] Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of this specification.
[0070] [Explanation of Labels in the Attached Image]
[0071] 101. Terminal;
[0072] 102. Server;
[0073] 701. Sample Triple Information Construction Unit;
[0074] 702. Create a cell;
[0075] 703. Convolution scoring calculation unit;
[0076] 704, Optimization Unit;
[0077] 705. Relationship prediction model determination unit;
[0078] 901. Unit for determining the vector to be processed;
[0079] 902. Convolution scoring calculation unit;
[0080] 903. Judgment Unit;
[0081] 904. Connection relationship determination unit;
[0082] 1002. Computer equipment;
[0083] 1004. Processing equipment;
[0084] 1006. Storage resources;
[0085] 1008. Drive mechanism;
[0086] 1010. Input / Output Module;
[0087] 1012. Input devices;
[0088] 1014. Output devices;
[0089] 1016. Presentation device;
[0090] 1018. Graphical User Interface;
[0091] 1020. Network interface;
[0092] 1022. Communication link;
[0093] 1024. Communication bus. Detailed Implementation
[0094] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0095] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0096] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0097] Figure 1 The diagram shows an implementation system of a relationship prediction model training method according to an embodiment of this specification, including a terminal 101 and a server 102. The terminal 101 and the server 102 can communicate via a network, which may include a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, and is connected to a website, user equipment (e.g., computing devices), and a backend system.
[0098] Staff can send data to server 102 through terminal 101. Server 102 can construct a knowledge graph based on the data input from terminal 101, and then train a relationship prediction model and store it based on the entities and the connection relationships between entities in the knowledge graph.
[0099] After the relationship prediction model is trained, staff can send the data of the relationship to be predicted to the server 102 through the terminal 101. The server 102 uses the stored relationship prediction model to calculate the data of the relationship to be predicted input by the terminal 101, obtain the connection relationship between the data of the relationship to be predicted, and send the connection relationship to the terminal 101 so that the terminal 101 can display the connection relationship to the staff.
[0100] Server 102 can be configured with code for training a relationship prediction model and applying the relationship prediction model. The code can be executed to train the relationship prediction model or to calculate the connection relationships of the data to be predicted.
[0101] In addition, it should be noted that, Figure 1 The example shown is merely one application environment provided by this disclosure. In practical applications, other application environments may also be included, and this specification does not impose any limitations.
[0102] Specifically, this specification provides a method for training a relationship prediction model, which controls model complexity by adjusting the dimension of the recurrent convolution, thereby reducing computational resource requirements and improving computational efficiency. Furthermore, it can map many-to-many relationships in knowledge graphs, broadening the application scope of knowledge graph embedding models. Figure 2 The diagram shown is a flowchart illustrating a relationship prediction model training method according to an embodiment of this specification. The process of quantitatively training the relationship prediction model is depicted in this diagram. The order of steps listed in the embodiment is merely one possible execution order among many steps and does not represent the only possible execution order. In actual system or device products, the method can be executed sequentially or in parallel according to the embodiment or the accompanying drawings. Specifically, as shown... Figure 2 As shown, the method can be executed by server 102 and may include:
[0103] Step 201: Based on the entities in the sample knowledge graph and the connection relationships between the entities, construct multiple sample triplet information;
[0104] Step 202: For the multiple sample triplet information, create multiple negative sample information and a weight corresponding to each negative sample information;
[0105] Step 203: Process the information of the multiple sample triples and the multiple negative sample information using the convolutional scoring function to obtain the corresponding scoring function value;
[0106] Step 204: Based on the scoring function value and the weight, iteratively optimize the training vectors corresponding to the entity and the connection relationship respectively until a target vector that meets the preset conditions is obtained;
[0107] Step 205: Use the convolutional scoring function to process the target vector corresponding to each sample triplet information and each negative sample information to obtain threshold data. Use the convolutional scoring function and the target vector to construct a trained relation prediction model based on the threshold data.
[0108] Using the embodiments in this specification, sample triple information is first constructed based on entities and connections between entities in the sample knowledge graph. Then, corresponding negative sample information and weights are created. A convolutional scoring function is then used to process the sample triple information and negative sample information to obtain corresponding scoring function values. Based on the scoring function values and weights, the training vectors corresponding to entities and connections are iteratively optimized until the training vectors meet preset conditions and are used as target vectors. Finally, the convolutional scoring function is used to process the target vectors corresponding to each sample triple information and each negative sample information to obtain threshold data. The convolutional scoring function, target vectors, and threshold data constitute the trained relationship prediction model. Through the convolutional scoring function, prediction of one-to-many, many-to-one, and many-to-many relationships is achieved, improving prediction efficiency and solving the problems of high computational resource requirements and low computational efficiency in existing relationship prediction methods based on knowledge graph embedding models.
[0109] In the embodiments of this specification, the objects of the relation prediction model training are entities and relations in the knowledge graph. The training result is the algebraic representation (i.e., multidimensional vector) mapped to the entities and relations. The trained relation prediction model is the set of key-value pair mappings between all entities and relations and their corresponding algebraic representations in the knowledge graph.
[0110] In the embodiments of this specification, the data in the knowledge graph can be customer data of a bank. A full scan of the bank's relationship graph can be performed to extract all the triples (h, r, t) contained in the relationship graph, for example, h = "customer 1", r = "card number", t = "card number 1". This yields an entity set E and a relationship set R, ultimately resulting in a large dataset S = {(h, r, t) | h, t belong to E, r belongs to R}.
[0111] Then, the entities and relationships in datasets E, R, and S are standardized according to the specific needs of the bank, retaining only important features. This simplifies and reduces potentially redundant information in the original knowledge graph, and also improves computational efficiency. For example, the entities "father" and "dad" are unified into the single word "father," thereby reducing the amount of redundant data in the three datasets.
[0112] The sample triplet information in the embodiments of this specification is not numerical but symbolic. Therefore, the sample triplet information needs to be initialized to numerical before training. In the embodiments of this specification, the entities and relations in E and R are algebraically initialized. The initialization method can be either uniform distribution sampling or Gaussian distribution sampling, the purpose of which is to distribute the data as evenly as possible within the spatial range, increasing the possibility of iterating to a local optimum. If uniform distribution is used, the sample distribution of each dimension is generally as follows: Where X represents the sampled data, U represents the uniform distribution sampling algorithm, i is a custom integer parameter that flexibly controls the range of values for random initialization variables, ε represents the expected value, and k represents the dimension of the embedding space. The expected value ε reflects the rigor of training; generally, this value needs to be obtained through multiple training iterations to avoid overfitting or underfitting to some extent. Selecting the dimension k of the embedding space maps entities and relationships to k-dimensional vectors, reflecting the amount of information carried by the entities or relationships. The amount of information increases exponentially with increasing k. Simultaneously, the larger k is, the greater the complexity of the circular convolution, and the larger the circular matrix generated by the discrete circular convolution will be; therefore, the value of k cannot be too large.
[0113] Then, multiple negative sample information and the corresponding weights for each negative sample information are created.
[0114] Specifically, such as Figure 3 As shown, the step of creating multiple negative sample information and weights corresponding to each negative sample information for the multiple sample triplet information includes:
[0115] Step 301: Select multiple target sample triplet information from the multiple sample triplet information;
[0116] Step 302: For each target sample triplet information, create multiple corresponding negative sample information;
[0117] Step 303: Determine the weight corresponding to the negative sample information based on the correlation between the negative sample information and the target sample triplet information.
[0118] In the embodiments of this specification, multiple target sample triplet information is first selected, and then negative sample information corresponding to these target sample triplet information is created. In the embodiments of this specification, positive samples represent actual existing triplets, and negative samples represent non-existent triplets. Negative samples can be obtained by replacing the head entity and / or tail entity of the positive sample triplet. The replacement entity must also be a real entity (usually the head entity or tail entity is replaced with a random value). Typically, a positive sample triplet can replace a large number of negative sample triplets, which is also consistent with practical significance. Using the self-adversarial strategy, the generated negative samples are multiplied by a weight, which represents the reciprocal of the correlation between the positive sample and the corresponding generated negative sample; the smaller the correlation, the smaller the weight is assigned to the negative sample, thereby weakening the negative impact of its strong correlation with the positive sample.
[0119] To improve the accuracy of the calculation, according to one embodiment of this specification, such as Figure 4 As shown, for each target sample triplet, multiple corresponding negative sample information are created, including:
[0120] Step 401: Determine that the target sample triplet information includes two target entities and the target connection relationship between the two target entities;
[0121] Step 402: For any one of the two target entities, determine from the entities a candidate entity that does not have the target connection relationship with the other target entity;
[0122] Step 403: Replace the target entity with the candidate entity to form the negative sample information with the other target entity among the two target entities and the target connection relationship.
[0123] In the embodiments of this specification, the target sample triple information is first determined to include two target entities A and B, and the target connection relationship x between the two target entities A and B. This target connection relationship x is a positive sample. Then, for any one of the two target entities A and B (for example, target entity A, i.e., this embodiment of the specification does not limit the replacement of the head entity or the tail entity), an entity C is determined from all entities in the sample knowledge graph that does not have a target connection relationship x with the other target entity B of the two target entities A and B. This entity is selected as a candidate entity, and target entity A is replaced by candidate entity C. Finally, candidate entity C, target entity B, and target connection relationship x are used as negative samples.
[0124] Then, the convolutional scoring function is used to process the information of the multiple sample triples and the multiple negative sample information to obtain the corresponding scoring function value.
[0125] According to one embodiment of this specification, the scoring function value includes a first sub-scoring function value and a second sub-scoring function value;
[0126] like Figure 5 As shown, the process of using a convolutional scoring function to process the information of the multiple sample triples and the multiple negative sample information to obtain the corresponding scoring function values includes:
[0127] Step 501: For each entity and each connection relationship, determine the vector to be trained;
[0128] Step 502: For each of the sample triplet information, the convolutional scoring function is used to process the training vector corresponding to the sample triplet information to obtain the first sub-scoring function value;
[0129] Step 503: For each negative sample information, the convolutional scoring function is used to process the training vector corresponding to the negative sample information to obtain the second sub-scoring function value.
[0130] In the embodiments of this specification, the scoring function is used to score whether a triple (h,r,t) is a true sample. When calculating the scoring function value, firstly, for each entity and each connection, a training vector is randomly initialized. Then, the convolutional scoring function is used to process the training vectors corresponding to the head entity, tail entity, and connection in each sample triple information to obtain the first sub-scoring function value. Then, the same convolutional scoring function is used to calculate the training vectors corresponding to the head entity, tail entity, and connection in the negative sample information to obtain the second sub-scoring function value.
[0131] Specifically, the convolution scoring function is:
[0132] Among them, f r (h,t) represents the scoring function value corresponding to (h,r,t), h and t respectively represent the training vectors corresponding to the two entities included in the sample triplet information or negative sample information, and r represents the training vector corresponding to the connection relationship included in the sample triplet information or negative sample information. F represents the circular convolution operation, and F represents the L1 norm or L2 norm.
[0133] According to one embodiment of this specification, such as Figure 6 As shown, based on the scoring function value and the weights, the training vectors corresponding to the entity and the connection relationship are iteratively optimized until a target vector satisfying the preset conditions is obtained, including:
[0134] Step 601: Process the first sub-scoring function value, the second sub-scoring function value, and the weight using a loss function to obtain the loss function value;
[0135] Step 602: Based on the optimization algorithm and the loss function value, optimize the training vectors corresponding to the entity and the connection relationship respectively to obtain the optimized vectors;
[0136] Step 603: Iterate using the optimized vector to replace the vector to be trained until the preset condition is met to obtain the target vector.
[0137] In the embodiments of this specification, parameters such as an adjustable learning rate and a maximum number of iterations are set to optimize the training vector. The adjustable learning rate is set initially, and then gradually decreased according to the training progress. Decreasing the learning rate when the loss function value oscillates further can approximate a local optimum. The maximum number of iterations is generally set to the number of iterations at which the training effect no longer changes significantly, effectively stopping the training.
[0138] Furthermore, the loss function includes:
[0139]
[0140] Where L represents the loss function, log represents the logarithmic loss function, σ represents the logistic function, γ represents the training strictness controlling the information of the sample triplets and the negative sample information, (h' i ,r,t' i f represents the training vector corresponding to the i-th negative sample information corresponding to (h,r,t), and (h,r,t) represents the training vector corresponding to the sample triplet information. r (h,t) represents the value of the first sub-scoring function corresponding to (h,r,t), f r (h' i ,t' i ) characterization and the (h' i ,r,t' i The second sub-scoring function value corresponding to ) is p(h' i ,r,t' i ) characterization and the (h' i ,r,t' i The weights corresponding to the sample triplet information are denoted by n, where n represents the total number of negative sample information corresponding to the sample triplet information.
[0141] Then, the convolutional scoring function is used to process the target vector corresponding to each sample triplet and each negative sample, respectively, to obtain threshold data. The threshold data is used to compare with the target value calculated by the convolutional scoring function to determine whether the predicted relationship is valid. The convolutional scoring function used to calculate the threshold data is the same as the convolutional scoring function used to calculate the scoring function value; that is, the data types of the training vector and the target vector are the same, and the data type of the threshold data is the same as the data type of the scoring function value. Additionally, performance metrics such as MR, MRR, and HITS@n can be used to calculate the threshold data. MR (Mean Rank) represents the average of the ranking of the scoring functions calculated for each positive sample triplet in its generated negative sample triplets, arranged from smallest to largest. The smaller this metric, the better the relationship prediction model. MRR (Mean Reciprocal Ranking) is calculated similarly to MR, except that MR takes the average of the rankings, while MRR takes the average of the reciprocals of the rankings. The larger this metric, the better the relationship prediction model. HITS@n represents the proportion of positive samples in triples with an index less than n. The larger the value, the better the relationship prediction model performs.
[0142] Finally, the convolutional scoring function and the target vector are used to construct a trained relationship prediction model based on the threshold data.
[0143] Based on the same inventive concept, embodiments of this specification also provide a relation prediction model training device, such as... Figure 7 As shown, it includes:
[0144] The sample triplet information construction unit 701 is used to construct multiple sample triplet information based on entities in the sample knowledge graph and the connection relationships between the entities;
[0145] The creation unit 702 is used to create multiple negative sample information and weights corresponding to each negative sample information for the multiple sample triplet information;
[0146] The convolution scoring calculation unit 703 is used to process the multiple sample triplet information and the multiple negative sample information using the convolution scoring function to obtain the corresponding scoring function value;
[0147] The optimization unit 704 is used to iteratively optimize the training vectors corresponding to the entity and the connection relationship based on the loss function value obtained from the scoring function value and the weight, until a target vector that meets the preset conditions is obtained.
[0148] The relation prediction model determination unit 705 is used to process the target vector corresponding to each sample triplet information and each negative sample information using the convolution scoring function to obtain threshold data, and to construct a trained relation prediction model based on the convolution scoring function and the target vector using the threshold data.
[0149] Since the principle of the above-mentioned device in solving the problem is similar to that of the above-mentioned method, the implementation of the above-mentioned device can refer to the implementation of the above-mentioned method, and the repeated parts will not be described again.
[0150] The relationship prediction model described in this specification can be used in scenarios such as completing the relationship graph of a bank and mining potential customer relationships. Therefore, based on the same inventive concept, the embodiments of this specification also provide a method for applying the relationship prediction model, such as... Figure 8 As shown, it includes:
[0151] Step 801: For the received triplet to be predicted, determine the processing vector from the target vector that corresponds to the entity in the triplet to be predicted and the connection relationship between the entity;
[0152] Step 802: Process the vector to be processed using a convolutional scoring function to obtain the target value;
[0153] Step 803: Determine whether the target value is less than the threshold data;
[0154] Step 804: If the target value is less than the threshold value, the connection relationship is determined to be valid.
[0155] The target vector and the threshold data are determined based on the relationship prediction model training method described above.
[0156] In the embodiments of this specification, the triple to be predicted includes two entities and a specified connection relationship. This can be used to predict whether the specified connection relationship exists between the two entities. The entities and connection relationships in the triple to be predicted are also data from the sample knowledge graph. A processing vector corresponding to the entities and connection relationships in the triple to be predicted is determined from the target vector obtained through training. Then, a convolutional scoring function is used to process the processing vector to obtain a target value. This convolutional scoring function is the same as the convolutional scoring function used when training the relationship prediction model; that is, the data type of the processing vector is the same as the data type of the target vector. The data type of the obtained target value is the same as the data type of the threshold data. Therefore, it can be determined whether the target value is less than the threshold data. When it is determined that the target value is less than the threshold data, the triple to be predicted is determined to be true, meaning that the specified connection relationship exists between the head and tail entities of the triple to be predicted.
[0157] Taking the mining of potential customer relationships as an example, suppose we want to determine whether a relationship x exists between customer A and customer B, i.e., predict the triple (A, x, B). First, from the trained relationship prediction model, we extract all (A, x, ${tail_entity}) and (${head_entity}, x, B), where the variables ${tail_entity} and ${head_entity} represent the head and tail entities that actually exist in the sample's knowledge graph, respectively. We then use a convolutional scoring function to calculate the target vector and threshold data for all extracted triples. Finally, we use the convolutional scoring function to calculate the target value for the triple (A, x, B). If the target value is less than the threshold data, then we determine that a relationship x does indeed exist between customer A and customer B.
[0158] Based on the same inventive concept, embodiments of this specification also provide an application device for a relationship prediction model, such as... Figure 9 As shown, it includes:
[0159] The vector determination unit 901 is used to determine, for the received triplet to be predicted, a vector to be processed from the target vector that corresponds to the entity in the triplet to be predicted and the connection relationship between the entity;
[0160] The convolution scoring calculation unit 902 is used to process the vector to be processed using the convolution scoring function to obtain the target value;
[0161] Judgment unit 903 is used to determine whether the target value is less than the threshold data;
[0162] The connection relationship determination unit 904 is used to determine that the connection relationship is established when the target value is less than the threshold value.
[0163] Wherein, the target vector and the threshold data are based on, for example Figure 7 The relationship prediction model training device shown is determined.
[0164] Since the principle of the above-mentioned device in solving the problem is similar to that of the above-mentioned method, the implementation of the above-mentioned device can refer to the implementation of the above-mentioned method, and the repeated parts will not be described again.
[0165] like Figure 10 The diagram illustrates the structure of a computer device according to an embodiment of this document. The apparatus described herein can be the computer device in this embodiment, executing the methods described above. The computer device 1002 may include one or more processing devices 1004, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. The computer device 1002 may also include any storage resource 1006 for storing information of any kind, such as code, settings, data, etc. Without limitation, for example, the storage resource 1006 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any storage resource can use any technology to store information. Furthermore, any storage resource can provide volatile or non-volatile retention of information. Further, any storage resource may represent a fixed or removable component of the computer device 1002. In one case, when the processing device 1004 executes associated instructions stored in any storage resource or combination of storage resources, the computer device 1002 can perform any operation of the associated instructions. The computer device 1002 also includes one or more drive mechanisms 1008 for interacting with any storage resource, such as hard disk drive mechanism, optical disk drive mechanism, etc.
[0166] Computer device 1002 may also include an input / output module 1010 (I / O) for receiving various inputs (via input device 1012) and providing various outputs (via output device 1014). A specific output mechanism may include a presentation device 1016 and an associated graphical user interface (GUI) 1018. In other embodiments, the input / output module 1010 (I / O), input device 1012, and output device 1014 may be omitted, and the device may function solely as a computer device within a network. Computer device 1002 may also include one or more network interfaces 1020 for exchanging data with other devices via one or more communication links 1022. One or more communication buses 1024 couple the components described above together.
[0167] The communication link 1022 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. The communication link 1022 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.
[0168] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0169] This specification also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method.
[0170] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0171] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0174] The above specific embodiments further illustrate the purpose, technical solutions, and beneficial effects of this specification. It should be understood that the above are merely specific embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.
Claims
1. A method for training a relationship prediction model, characterized in that, include: Based on entities in the sample knowledge graph and the connection relationships between those entities, multiple sample triplet information is constructed. For the multiple sample triplet information, multiple negative sample information and a weight corresponding to each negative sample information are created; The convolutional scoring function is used to process the information of the multiple sample triples and the multiple negative sample information to obtain the corresponding scoring function value; Based on the scoring function value and the weight, the training vectors corresponding to the entity and the connection relationship are iteratively optimized until the target vector that meets the preset conditions is obtained. The convolutional scoring function is used to process the target vector corresponding to each sample triplet and each negative sample to obtain threshold data. The convolutional scoring function and the target vector are then used to construct a trained relation prediction model based on the threshold data. The convolution scoring function includes: Among them, f r (h,t) represents the scoring function value corresponding to (h,r,t), where h and t represent the training vector or target vector corresponding to the two entities included in the sample triplet information, and r represents the training vector or target vector corresponding to the connection relationship included in the sample triplet information. F represents the circular convolution operation, and F represents the L1 norm or L2 norm.
2. The method according to claim 1, characterized in that, The step of creating multiple negative sample information and a weight corresponding to each negative sample information for the multiple sample triplet information includes: Select multiple target sample triplet information from the multiple sample triplet information; For each of the target sample triplet information, create multiple corresponding negative sample information; The weight corresponding to the negative sample information is determined based on the correlation between the negative sample information and the target sample triple information.
3. The method according to claim 2, characterized in that, For each target sample triplet, multiple corresponding negative sample information are created, including: The target sample triplet information is determined to include two target entities and the target connection relationship between the two target entities; For any one of the two target entities, determine a candidate entity from the entity that does not have the target connection relationship with the other target entity; The candidate entity is used to replace the target entity, so that it, together with the other target entity of the two target entities and the target connection relationship, constitutes the negative sample information.
4. The method according to claim 1, characterized in that, The scoring function value includes a first sub-scoring function value and a second sub-scoring function value; The process of using a convolutional scoring function to process the information of the multiple sample triples and the multiple negative sample information to obtain the corresponding scoring function value includes: For each entity and each connection relationship, a training vector is determined separately; For each of the sample triplet information, the convolutional scoring function is used to process the training vector corresponding to the sample triplet information to obtain the first sub-scoring function value; For each negative sample information, the training vector corresponding to the negative sample information is processed using a convolutional scoring function to obtain the second sub-scoring function value.
5. The method according to claim 4, characterized in that, The step of iteratively optimizing the training vectors corresponding to the entity and the connection relationship based on the scoring function value and the weight, until a target vector satisfying the preset conditions is obtained, includes: The loss function is used to process the first sub-scoring function value, the second sub-scoring function value, and the weight to obtain the loss function value; Based on the optimization algorithm and the loss function value, the training vectors corresponding to the entities and the connection relationships are optimized to obtain optimized vectors. The optimized vector is used to replace the training vector for iteration until the preset condition is met, thus obtaining the target vector.
6. The method according to claim 5, characterized in that, The loss function includes: Where L represents the loss function, log represents the logarithmic loss function, σ represents the logistic function, γ represents the training strictness controlling the sample triplet information and the negative sample information, (h ’ i ,r,t ’ i f represents the training vector corresponding to the i-th negative sample information corresponding to (h,r,t), and (h,r,t) represents the training vector corresponding to the sample triplet information. r (h ’ ,t ’ ) represents the value of the first sub-scoring function corresponding to (h,r,t), f r (h ’ i ,t ’ i ) characterization and the (h) ’ i ,r,t ’ i The second sub-scoring function value corresponding to ) is p(h ’ i ,r,t ’ i ) characterization and the (h) ’ i ,r,t ’ i The weights corresponding to the sample triplet information are denoted by n, where n represents the total number of negative sample information corresponding to the sample triplet information.
7. A method for applying a relationship prediction model, characterized in that, include: For the received triplet to be predicted, determine the processing vector from the target vector that corresponds to the entity in the triplet to be predicted and the connection relationship between the entity; The target value is obtained by processing the vector to be processed using a convolutional scoring function; Determine whether the target value is less than the threshold data; If the target value is determined to be less than the threshold data, the connection relationship is determined to be valid. The target vector and the threshold data are determined based on the method described in any one of claims 1-6.
8. A relationship prediction model training device, characterized in that, include: The sample triplet information construction unit is used to construct multiple sample triplet information based on entities in the sample knowledge graph and the connection relationships between the entities; A creation unit is used to create multiple negative sample information and a weight corresponding to each of the multiple sample triplet information; The convolution scoring calculation unit is used to process the multiple sample triplet information and the multiple negative sample information using the convolution scoring function to obtain the corresponding scoring function value; An optimization unit is used to iteratively optimize the training vectors corresponding to the entity and the connection relationship based on the loss function value obtained from the scoring function value and the weight, until a target vector that meets the preset conditions is obtained. The relation prediction model determination unit is used to process the target vector corresponding to each sample triplet information and each negative sample information using the convolutional scoring function to obtain threshold data, and to construct a trained relation prediction model based on the convolutional scoring function and the target vector using the threshold data. The convolution scoring function includes: Among them, f r (h,t) represents the scoring function value corresponding to (h,r,t), where h and t represent the training vector or target vector corresponding to the two entities included in the sample triplet information, and r represents the training vector or target vector corresponding to the connection relationship included in the sample triplet information. F represents the circular convolution operation, and F represents the L1 norm or L2 norm.
9. A device for applying a relationship prediction model, characterized in that, include: The vector determination unit is used to determine, for example, a target vector from a target vector corresponding to the entity in the triplet to be predicted and the connection relationship between the entity; The convolution scoring calculation unit is used to process the vector to be processed using the convolution scoring function to obtain the target value; The judgment unit is used to determine whether the target value is less than the threshold data; A connection relationship determination unit is used to determine that the connection relationship is established when the target value is less than the threshold data; The target vector and the threshold data are determined based on the apparatus as described in claim 8.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1-7.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the method of any one of claims 1-7.
12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method according to any one of claims 1-7.