Graph model training method and apparatus

Abstract

Embodiments of the present specification provide a graph model training method and device. A to-be-trained graph model is suitable for a first business scenario; the method comprises: obtaining a teacher network graph model; wherein the teacher network graph model is suitable for a second business scenario and is a trained graph model; obtaining a training sample from the first business scenario; learning the training sample by using the teacher network graph model and the to-be-trained graph model respectively; performing similarity constraint on a learning result of the training sample according to the teacher network graph model and a learning result of the training sample according to the to-be-trained graph model, to obtain a difference loss; obtaining a business loss by using the learning result of the training sample according to the to-be-trained graph model; and adjusting model parameters of the to-be-trained graph model according to the difference loss and the business loss. Embodiments of the present specification can reduce the requirement for the number of training samples in the first business scenario.

Description

Technical Field

[0001] This specification relates to artificial intelligence technology in one or more embodiments, and more particularly to graph model training methods and apparatus. Background Technology

[0002] A graph is an abstract data structure used to represent relationships between objects. It is described using vertices and edges, where nodes represent objects and edges represent relationships between them. Each node in a graph has its own set of characteristics, and each edge also has its own set of characteristics. For example, see... Figure 1 As shown in the diagram, the nodes can represent user account information, and the edges can represent transaction behaviors between users. Each node can include all features related to the account, such as account ID, demographics, gender, age, education, account information, asset information, historical transaction habits, and other information. Each edge can include all features related to a transaction, such as transaction ID, transaction time, transaction location, amount, payment channel, and the nature of the transaction, such as whether it is a violation.

[0003] Currently, many business scenarios, such as financial transaction risk control systems and product recommendation systems, require analysis and processing based on the relationships between objects. Graph models using neural networks, as a method specifically designed for processing this type of unstructured data, have been widely used in recent years. However, as machine models, graph models require a huge number of samples during the training phase; only through massive amounts of data can a high-performance graph model be trained.

[0004] However, in many business scenarios, such as emerging fields, the number of samples is limited, making it impossible to obtain a sufficient number of samples to meet training requirements. Furthermore, if a massive number of training samples are used, then graph model construction and feature cleaning often occupy most of the time in the entire graph model deployment process, consuming the vast majority of the engineering and technical personnel's energy.

[0005] Therefore, a new graphical model training method is needed that no longer relies on massive amounts of training samples. Summary of the Invention

[0006] This specification describes one or more embodiments of a method and apparatus for training graph models that can reduce the number of training samples required.

[0007] According to the first aspect, a graph model training method is provided, wherein the graph model to be trained is applicable to a first business scenario; the method includes:

[0008] A teacher network graph model is obtained; wherein the teacher network graph model is applicable to the second business scenario and is a trained graph model;

[0009] Training samples are obtained from the first business scenario;

[0010] Training samples are learned using the teacher network graph model and the graph model to be trained, respectively;

[0011] To obtain the difference loss, a similarity constraint is applied to the learning results of the teacher network graph model on the training samples and the learning results of the graph model to be trained on the training samples.

[0012] The learning results of the graph model to be trained on the training samples are used to obtain the business loss.

[0013] The model parameters of the graph model to be trained are adjusted based on the difference loss and the business loss.

[0014] The step of learning training samples using the teacher network graph model and the graph model to be trained includes:

[0015] The graph structure is extracted from the training samples using a teacher network graph model to obtain the first graph structure representation;

[0016] The graph structure is extracted from the training samples using the graph model to be trained, so as to obtain the second graph structure representation;

[0017] The teacher network graph model is used to extract graph features from the training samples to obtain the first graph feature representation;

[0018] The graph model to be trained is used to extract graph features from the training samples to obtain the second graph feature representation.

[0019] Correspondingly, the step of applying a similarity constraint between the learning results of the teacher's network graph model on the training samples and the learning results of the graph model to be trained on the training samples to obtain the difference loss includes:

[0020] Similarity constraints are applied to the structural representations of the first and second graphs to obtain the first difference loss;

[0021] A similarity constraint is applied to the feature representations of the first and second images to obtain the second difference loss.

[0022] The method further includes:

[0023] A neighbor relationship matrix is ​​obtained based on the training samples; wherein, the neighbor relationship matrix is ​​an N*N matrix, and N is the number of nodes obtained based on the training samples; for any two nodes, if the two nodes are directly connected and form a first-order neighbor relationship, then the value of the matrix element corresponding to the two nodes in the neighbor relationship matrix is ​​1, otherwise it is 0;

[0024] A feature matrix is ​​obtained based on the training samples; wherein, the feature matrix is ​​an N*M matrix, where M is the number of features included in each node obtained from the training samples; each row in the feature matrix corresponds to a node, and different matrix elements in the row represent different features of the node.

[0025] The method further includes:

[0026] Set all matrix elements in the feature matrix to 0 to obtain the first feature matrix;

[0027] The extraction of graph structure from training samples using a teacher network graph model includes:

[0028] The neighbor relationship matrix and the first feature matrix are input into the teacher network graph model to obtain the first graph structure representation output by the teacher network graph model.

[0029] The step of extracting graph structure from training samples using the graph model to be trained includes:

[0030] The neighbor relationship matrix and the first feature matrix are input into the graph model to be trained to obtain the second graph structure representation output by the graph model to be trained.

[0031] The method further includes:

[0032] For each element in the feature matrix, the value of the element is set to the value of the corresponding feature of the node to which the element belongs, so as to obtain the second feature matrix.

[0033] The extraction of graph features from training samples using a teacher network graph model includes:

[0034] The neighbor relationship matrix and the second feature matrix are input into the teacher network graph model to obtain the first graph feature representation output by the teacher network graph model.

[0035] The step of extracting graph features from training samples using the graph model to be trained includes:

[0036] The neighbor relationship matrix and the second feature matrix are input into the graph model to be trained to obtain the second graph feature representation output by the graph model to be trained.

[0037] The first difference loss obtained by applying similarity constraints to the first graph structural representation and the second graph structural representation includes:

[0038] The first difference loss is obtained by calculating the mean square error and comparing the structural representations of the first and second graphs.

[0039] The second difference loss is obtained by applying similarity constraints to the feature representations of the first and second images, including:

[0040] The similarity between the feature representations of the first and second images is calculated using the mean square error method to obtain the second difference loss.

[0041] The step of obtaining the business loss by utilizing the learning results of the graph model to be trained on the training samples includes:

[0042] The second graph feature representation is inferred using a multilayer perceptron, thereby converting the second graph feature representation into a predicted value between 0 and 1;

[0043] The business loss is obtained by calculating the binary classification loss using the predicted values ​​and the label information included in the training samples.

[0044] The step of adjusting the parameters of the graph model to be trained based on the difference loss and the business loss includes:

[0045] The constraint function of the graphical model to be trained is set as: L = loss0 + α·loss1 + β·loss2;

[0046] Where L represents the constraint function; loss0 represents the business loss; loss1 represents the first difference loss; loss2 represents the second difference loss; α and β are pre-set hyperparameter constraints, and the larger the values ​​of α and β, the greater the weight ratio of the knowledge borrowed from the teacher's network graph model.

[0047] According to the second aspect, a graph model training apparatus is provided, wherein the graph model to be trained is applicable to a first business scenario; the apparatus includes:

[0048] The teacher network acquisition module is configured to obtain a teacher network graph model; wherein the teacher network graph model is applicable to the second business scenario and is a trained graph model;

[0049] The training sample acquisition module is configured to obtain training samples from the first business scenario;

[0050] The learning module is configured to learn training samples using the teacher network graph model and the graph model to be trained, respectively.

[0051] The difference loss module is configured to apply similarity constraints to the learning results of the training samples based on the teacher's network graph model and the learning results of the training samples based on the graph model to be trained, so as to obtain the difference loss.

[0052] The business loss acquisition module is configured to obtain the business loss by utilizing the learning results of the graph model to be trained on the training samples.

[0053] The model parameter adjustment module is configured to adjust the model parameters of the graph model to be trained based on the difference loss and the business loss.

[0054] According to a third aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described in any embodiment of this specification.

[0055] The graph model training method and apparatus provided in the embodiments of this specification, when training a graph model applicable to a first business scenario, uses a graph model already trained in other business scenarios (referred to as the teacher network graph model) to learn from the training samples in the first business scenario. Because a similarity constraint is applied between the learning results of the teacher network graph model on the training samples and the learning results of the graph model to be trained on the training samples, resulting in a difference loss, it is possible to distill the knowledge learned by the teacher network graph model based on the training samples into the graph model to be trained applicable to the first business scenario. That is, performing knowledge distillation on the graph model of the new scenario not only reduces the required number of training samples in the first business scenario (i.e., reduces the scale of the graph data training samples needed), but also utilizes related knowledge to improve the performance of the graph model to be trained. Therefore, the embodiments of this specification actually propose a knowledge distillation scheme for graph models, utilizing existing historical graph models to reduce the scale of training samples and improve the performance of the graph model to be trained. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of a graph network.

[0058] Figure 2 This is a flowchart of a graph model training method in one embodiment of this specification.

[0059] Figure 3 This is a flowchart of a method in one embodiment of this specification for learning training samples and applying similarity constraints using a teacher network graph model and a graph model to be trained, respectively.

[0060] Figure 4 This is a schematic diagram of a neighbor relationship matrix in one embodiment of this specification.

[0061] Figure 5 This is a schematic diagram of a feature matrix in one embodiment of this specification.

[0062] Figure 6 This is a schematic diagram of a graph model training method in one embodiment of this specification.

[0063] Figure 7 This is a schematic diagram of the structure of a training device for a graphical model in one embodiment of this specification.

[0064] Figure 8 This is a schematic diagram of the structure of the training device for the graphical model in another embodiment of this specification. Detailed Implementation

[0065] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0066] First, it should be noted that the terminology used in the embodiments of this invention is for the purpose of describing specific embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0067] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0068] Figure 2 This is a flowchart of a graph model training method in one embodiment of this specification. The main body executing this method is the graph model training device. It is understood that this method can also be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. See also... Figure 2 This requires training a graph model suitable for the first business scenario; the method includes:

[0069] Step 201: Obtain the teacher network graph model; wherein, the teacher network graph model is applicable to the second business scenario and is a trained graph model.

[0070] Step 203: Obtain training samples from the first business scenario.

[0071] Step 205: Use the teacher network graph model and the graph model to be trained to learn training samples respectively.

[0072] Step 207: Apply similarity constraints to the learning results of the teacher's network graph model on the training samples and the learning results of the graph model to be trained on the training samples to obtain the difference loss.

[0073] Step 209: Use the learning results of the graph model to be trained on the training samples to obtain the business loss.

[0074] Step 211: Adjust the model parameters of the graph model to be trained based on the difference loss and the business loss.

[0075] According to the above Figure 2 As shown in the flowchart, in this embodiment of the specification, when it is necessary to train a graph model suitable for the first business scenario, a graph model already trained in other business scenarios (referred to as the teacher network graph model) is used to learn from the training samples in the first business scenario. Because a similarity constraint is applied between the learning results of the teacher network graph model on the training samples and the learning results of the graph model to be trained on the training samples, resulting in a difference loss, it is possible to distill the knowledge learned by the teacher network graph model based on the training samples into the graph model to be trained suitable for the first business scenario. That is, performing knowledge distillation on the graph model of the new scenario not only reduces the required number of training samples in the first business scenario (i.e., reduces the scale of the graph data training samples needed), but also utilizes related knowledge to improve the performance of the graph model to be trained. Therefore, this embodiment of the specification actually proposes a knowledge distillation scheme for graph models, utilizing existing historical graph models to reduce the scale of training samples and improve the performance of the graph model to be trained.

[0076] The above will be explained below with reference to the accompanying drawings and specific examples. Figure 2 The process shown is explained below.

[0077] First, for step 201: obtain the teacher network graph model; wherein, the teacher network graph model is applicable to the second business scenario and is a trained graph model.

[0078] In the embodiments of this specification, when it is necessary to train a graph model in a business scenario, denoted as the first business scenario, problems such as insufficient training samples in the first business scenario may occur. For example, for an emerging business scenario, due to insufficient historical business data, the number of training samples that can be used to train the graph model is also very limited. Therefore, it is impossible to train a graph model with excellent performance suitable for the emerging business scenario.

[0079] For other business scenarios, such as any second business scenario, because its historical business data is sufficient, a graph model suitable for that second business scenario has already been trained using enough training samples. Although the graph model suitable for the second business scenario cannot be directly applied to the first business scenario, since both are graph models, the learning and training methods are the same. Both need to learn the relationships formed by the various objects in the graph data (this relationship can be called the graph structure) and the features of each object. Therefore, in the embodiments of this specification, a knowledge distillation method is considered to be used to distill the features of the training samples in the first business scenario learned by the teacher network graph model suitable for the second business scenario into the graph model to be trained in the first business scenario.

[0080] As mentioned earlier, the teacher network graph model is a pre-trained graph model applicable to the second business scenario. In the embodiments of this specification, the teacher network graph model is not limited to any specific network structure and can be a graph aggregation network applicable to any business scenario. Of course, to further improve the knowledge distillation effect, the second business scenario applicable to the teacher network graph model can be similar to the first business scenario applicable to the graph model to be trained, i.e., similar business domains. For example, risk control is a relatively mature business, and a well-trained, high-performance graph model already exists in the risk control business scenario. However, for scenarios such as e-commerce or other emerging business scenarios, the training samples are fewer. Therefore, when it is necessary to train a graph model applicable to the e-commerce business scenario or other emerging business scenarios, the graph model in the risk control business scenario can be used as the teacher network graph model.

[0081] Next, for step 203: obtain training samples from the first business scenario.

[0082] The first business scenario is the business scenario in which the graph model to be trained is applicable, such as the e-commerce business scenario mentioned above or other emerging business scenarios. The training samples can be obtained based on historical business data from the first business scenario. In essence, the training samples are labeled graph data from which nodes, edges, features, and the connections between nodes formed by edges can be obtained.

[0083] Next, for step 205: use the teacher network graph model and the graph model to be trained to learn the training samples respectively.

[0084] Step 205 involves having both the teacher's network graph model and the graph model to be trained learn from the training samples, i.e., learn the same target information, so that the graph model to be trained can subsequently learn from the learning results of the teacher's network graph model.

[0085] For a graph network, its core content includes the graph structure, that is, the connection relationships between the nodes. For example, which node is directly connected to node 1, and whether node 1 can be connected to node 5 through a certain node. Therefore, when learning from training samples, it is necessary to learn a graph structure, i.e., network topology, that reflects the characteristics of the first business scenario based on the training samples obtained in the first business scenario.

[0086] Furthermore, the core content of a graph network also includes graph features, that is, the features of each object in the graph network. For example, when there is an edge connecting node 1 and node 2, meaning that node 1 and node 2 are first-order neighbors, what features will node 1 have, and what features will node 2 have? Therefore, when learning from training samples, it is necessary to learn graph features that can reflect the characteristics of the first business scenario based on the training samples obtained in the first business scenario.

[0087] As can be seen, in step 205, it is necessary to simultaneously extract the representation of the graph structure and the representation of the graph features from the training samples. Accordingly, in one embodiment of this specification, see... Figure 3 , Figure 6 The specific implementation process of step 205 includes steps 2051 to 2057:

[0088] Step 2051: Extract the graph structure from the training samples using the teacher network graph model to obtain the first graph structure representation.

[0089] Step 2053: Extract the graph structure from the training samples using the graph model to be trained to obtain the second graph structure representation.

[0090] Step 2055: Extract graph features from the training samples using the teacher network graph model to obtain the first graph feature representation.

[0091] Step 2057: Extract graph features from the training samples using the graph model to be trained to obtain the second graph feature representation.

[0092] In the embodiments of this specification, the processes of steps 2051 to 2057 described above can be implemented using the neighbor relationship matrix A and the feature matrix B. The following explains how to implement these two matrices.

[0093] As mentioned earlier, it is necessary to learn a graph structure, i.e., network topology, that reflects the characteristics of the first business scenario based on the training samples obtained in the first business scenario. In one embodiment of this specification, a neighbor relationship matrix, denoted as A, can be used to represent the neighbor relationships between the nodes in the training samples, thereby representing the graph structure, i.e., the network topology. It should be noted that in this embodiment of the specification, a neighbor relationship refers to a first-order neighbor relationship between two nodes that are directly connected by an edge. Based on the training samples obtained in the first business scenario, in the neighbor relationship matrix A, each matrix element in the first row indicates whether node 1 in the training samples has a neighbor relationship with any other node in the training samples; each matrix element in the second row indicates whether node 2 in the training samples has a neighbor relationship with any other node in the training samples, and so on, with each matrix element in the Nth row indicating whether node N in the training samples has a neighbor relationship with any other node in the training samples.

[0094] Therefore, in one embodiment of this specification, the method further includes: obtaining a neighbor relationship matrix based on training samples; wherein the neighbor relationship matrix is ​​an N*N matrix, and N is the number of nodes included in the training samples; for any two nodes, if the two nodes are directly connected and form a first-order neighbor relationship, then the value of the matrix element corresponding to the two nodes in the neighbor relationship matrix is ​​1, otherwise it is 0. For example, see Figure 4 A 11 This represents the neighbor relationship between node 1 and node 2. For example, in a transaction scenario, if node 1 is a user account, and this user account has sent itself a red envelope or transferred funds, then node 1 and node 2 have a neighbor relationship. 11 The value is 1; A 12 This represents the neighbor relationship between node 1 and node 2. For example, if node 1 and node 2 do not have a neighbor relationship, then A... 12 The value of A is 0; 13 This represents the neighbor relationship between node 1 and node 3. For example, if node 1 and node 3 do not have a neighbor relationship, then A... 12 The value is 0, and so on.

[0095] As mentioned earlier, it is necessary to learn graph features that reflect the characteristics of the first business scenario based on the training samples obtained in the first business scenario. In one embodiment of this specification, the feature matrix, denoted as B, can be used to represent the various features of each node in the training samples. Therefore, in one embodiment of this specification, the method further includes: obtaining a feature matrix based on the training samples; wherein the feature matrix is ​​an N*M matrix, M is the number of features included in each node in the training samples; each row of the feature matrix corresponds to a node, and different matrix elements in that row represent different features of that node. In specific implementation, the number / dimensionality of features of all nodes is made the same. If a node has no feature in a certain dimension, the feature value of that node in that dimension can be set to empty, thereby ensuring that each node has M / dimensional feature values. For example, see Figure 5 Based on the training samples obtained in the first business scenario, in the feature matrix B, B 11 For the first feature of node 1 (e.g., node 1 can be an account), B (the first feature is the feature of the first dimension, such as the amount) 12 The second feature corresponding to node 1 (the second feature is a feature of the second dimension, such as transaction time), and so on, B 1M The corresponding Mth feature of node 1 (the Mth feature is a feature of the Mth dimension, such as the username); similarly, B 21 For the first feature (i.e., the first dimension feature, such as the amount) of node 2 (for example, node 2 could be another account), B 22 The second feature of node 2 (i.e., the feature of the second dimension, such as transaction time) and so on.

[0096] In steps 2051 and 2053, it is necessary to extract the representation of the graph structure. Because the graph structure is only related to the neighbor relationship matrix A and not to the feature matrix B, meaning the feature values ​​of nodes in the training samples cannot affect the representation of the graph structure, therefore, see... Figure 6 Set all matrix elements in feature matrix B to 0. The resulting feature matrix is ​​denoted as the first feature matrix.

[0097] Accordingly, see Figure 6 In step 2051, the process of extracting graph structure from training samples using the teacher network graph model includes: inputting the formed neighbor relationship matrix and the first feature matrix into the teacher network graph model to obtain the first graph structure representation output by the teacher network graph model.

[0098] Accordingly, see Figure 6In step 2053, the process of extracting graph structure from training samples using the graph model to be trained includes: inputting the formed neighbor relationship matrix and the first feature matrix into the graph model to be trained to obtain the second graph structure representation output by the graph model to be trained.

[0099] It can be seen that the first graph structure representation and the second graph structure representation do not depend on any features of the training samples, that is, they do not depend on the features of the nodes and the features of the edges, but only on the neighbor information of the graph data used as training samples.

[0100] In steps 2055 and 2057, it is necessary to extract the representation of graph features. Because graph features are related to both the neighbor relationship matrix A (requiring knowledge of the neighbor relationships between nodes) and the feature matrix B (requiring knowledge of the feature values ​​of each node), therefore, see... Figure 6 For each element in feature matrix B, the value of that element is set to the eigenvalue of the corresponding feature of the node to which that element belongs. The resulting feature matrix is ​​denoted as the second feature matrix. For example, as mentioned earlier, matrix element B... 11 The value is the feature value of the first feature (the first dimension feature, such as the amount) of node 1, for example, a normalized feature value of 0.5; matrix element B 12 The value is the feature value of the second feature of node 1 (the feature of the second dimension, such as transaction time), for example, a normalized feature value of 0.7; matrix element B NM The value is the feature value of the Mth feature (the feature of the Mth dimension, such as the username) of node N, for example, the normalized feature value of 0.3.

[0101] Accordingly, see Figure 6 In step 2055, the process of extracting graph features from training samples using the teacher network graph model may include: inputting the formed neighbor relationship matrix and the second feature matrix into the teacher network graph model to obtain the first graph feature representation output by the teacher network graph model.

[0102] Accordingly, see Figure 6 In step 2057, the process of extracting graph structure from training samples using the graph model to be trained includes: inputting the formed neighbor relationship matrix and second feature matrix into the graph model to be trained to obtain the second graph feature representation output by the graph model to be trained.

[0103] Next, for step 207: apply similarity constraints to the learning results of the teacher's network graph model on the training samples and the learning results of the graph model to be trained on the training samples to obtain the difference loss.

[0104] The purpose of step 207 is to constrain the representations extracted by the two graph models, namely the teacher network graph model and the graph model to be trained, to be as similar as possible, so as to distill the knowledge learned by the teacher network graph model into the graph model to be trained.

[0105] Corresponding to the above Figure 3 For the implementation process of step 205 shown, please refer to [link / reference]. Figure 3 , Figure 6 The specific implementation process of step 207 includes steps 2071 and 2073:

[0106] Step 2071: Apply similarity constraints to the structural representations of the first and second graphs to obtain the first difference loss, denoted as loss1.

[0107] One implementation of step 2071 includes: using the method of calculating the mean square error to find the similarity between the structural representation of the first graph and the structural representation of the second graph, so as to obtain the first difference loss.

[0108] For example, in step 2071, the structural representation of the first graph is usually a one-dimensional array, and the structural representation of the second graph is also a one-dimensional array, such as 1*128 dimensional arrays. The mean squared error method is used to find the similarity between the two one-dimensional arrays. That is, in each dimension, the values ​​in the two arrays are first calculated as a difference, and then the difference is squared to obtain the square of the difference in that dimension. Finally, 128 squared differences are obtained. The sum of these 128 squared differences and then divided by 128 gives the first difference loss, denoted as loss1. The first difference loss is a scalar value.

[0109] Step 2073: Apply similarity constraints to the feature representations of the first and second images to obtain the second difference loss, denoted as loss2.

[0110] One implementation of step 2073 includes: using the method of calculating the mean square error to find the similarity between the feature representation of the first image and the feature representation of the second image, so as to obtain the second difference loss.

[0111] The implementation method of step 2073 can be found in the description of step 2071.

[0112] Next, for step 209: use the learning results of the graph model to be trained on the training samples to obtain the business loss.

[0113] One implementation process for step 209 includes:

[0114] Step 2091: Use a multilayer perceptron to infer the feature representation of the second graph, thereby converting the feature representation of the second graph into predicted values ​​between 0 and 1.

[0115] Step 2093: Calculate the binary classification loss using the predicted values ​​and the label information included in the training samples, thereby obtaining the business loss.

[0116] The second graph feature representation corresponds to the labels of the training samples. That is, the second graph feature representation is the business result predicted by the graph model to be trained for the training samples, while the label is the correct business result of the training samples obtained based on historical data. The second graph feature representation needs to be compared with the label to adjust the model parameters of the graph model to be trained. However, the graph feature representation is a one-dimensional array, such as a 1*128 dimension array, while the labels of the training samples are probability values ​​between 0 and 1. Therefore, step 2091 needs to be performed to infer the second graph feature representation through a multilayer perceptron, thereby converting the second graph feature representation into a predicted value between 0 and 1, which represents a probability value. Only in this way can both the second graph feature representation and the label be values ​​between 0 and 1, allowing step 2093 to be performed, i.e., calculating the binary classification loss to obtain the business loss, denoted as loss0.

[0117] Steps 205 and 207 above obtain a difference loss by analyzing the learning results between the teacher network graph model and the graph model to be trained (which can also be called the student network graph model), and then use this difference loss to adjust the model parameters of the graph model to be trained. Step 209, however, uses the learning results of the graph model to be trained itself, such as whether the learning results are correct, to obtain a business loss, and then uses this business loss to adjust the model parameters of the graph model to be trained.

[0118] In other words, the purpose of step 205 is to achieve knowledge distillation of graph structure representation. The purpose of step 207 is to achieve knowledge distillation of graph feature representation. The purpose of step 209 is to address the task constraints of the first business scenario. Through the combined implementation of steps 205, 207, and 209, a graph model suitable for the first business scenario is jointly trained.

[0119] Next, for step 211: adjust the model parameters of the graph model to be trained based on the difference loss and business loss.

[0120] As mentioned earlier, the difference loss includes: loss function loss1, which reflects the knowledge distillation of the teacher's network graph model on the graph structure representation of the training samples; and loss function loss2, which reflects the knowledge distillation of the teacher's network graph model on the graph feature representation of the training samples. The business loss is loss0.

[0121] Therefore, see Figure 6In this step 211, the constraint function of the graph model to be trained can be set as: L=loss0+α·loss1+β·loss2;

[0122] Where L represents the constraint function; loss0 represents the business loss; loss1 represents the first difference loss; loss2 represents the second difference loss; α and β are pre-set hyperparameter constraints. The larger the values ​​of α and β, the greater the weight of the knowledge borrowed from the teacher's network graph model. The values ​​of α and β can be adjusted according to actual business needs.

[0123] In step 211, the constraint function of the graph model to be trained is set as: L = loss0 + α·loss1 + β·loss2. Then, the model parameters of the graph model to be trained can be adjusted according to the constraint function, and then the next round of training is carried out. That is, steps 203 to 211 are executed repeatedly until the graph model to be trained converges.

[0124] In one embodiment of this specification, a training apparatus for a graph model is provided, see [link to relevant documentation]. Figure 7 The device includes:

[0125] The teacher network acquisition module 701 is configured to obtain a teacher network graph model; wherein the teacher network graph model is applicable to the second business scenario and is a trained graph model;

[0126] The training sample acquisition module 702 is configured to obtain training samples from the first business scenario;

[0127] Learning module 703 is configured to learn training samples using the teacher network graph model and the graph model to be trained, respectively.

[0128] The difference loss module 704 is configured to impose similarity constraints on the learning results of the training samples based on the teacher network graph model and the learning results of the training samples based on the graph model to be trained, so as to obtain the difference loss.

[0129] The business loss acquisition module 705 is configured to obtain the business loss by utilizing the learning results of the graph model to be trained on the training samples.

[0130] The model parameter adjustment module 706 is configured to adjust the model parameters of the graph model to be trained based on the difference loss and the business loss.

[0131] In one embodiment of the device described in this specification, the learning module 703 is configured to perform:

[0132] The graph structure is extracted from the training samples using a teacher network graph model to obtain the first graph structure representation;

[0133] The graph structure is extracted from the training samples using the graph model to be trained, so as to obtain the second graph structure representation;

[0134] The teacher network graph model is used to extract graph features from the training samples to obtain the first graph feature representation;

[0135] The graph model to be trained is used to extract graph features from the training samples to obtain the second graph feature representation.

[0136] Correspondingly, the difference loss obtaining module 704 is configured to perform:

[0137] Similarity constraints are applied to the structural representations of the first and second graphs to obtain the first difference loss;

[0138] A similarity constraint is applied to the feature representations of the first and second images to obtain the second difference loss.

[0139] In one embodiment of the device described in this specification, it further includes: a training sample processing module 801;

[0140] The training sample processing module 801 is configured to obtain a neighbor relationship matrix based on the training samples; wherein the neighbor relationship matrix is ​​an N*N matrix, and N is the number of nodes obtained from the training samples; for any two nodes, if the two nodes are directly connected and form a first-order neighbor relationship, then the value of the matrix element corresponding to the two nodes in the neighbor relationship matrix is ​​1, otherwise it is 0; and to obtain a feature matrix based on the training samples; wherein the feature matrix is ​​an N*M matrix, and M is the number of features included in each node obtained from the training samples; each row in the feature matrix corresponds to a node, and different matrix elements in the row represent different features of the node.

[0141] In one embodiment of the apparatus described in this specification, the training sample processing module 801 is configured to perform: setting the values ​​of all matrix elements in the feature matrix to 0 to obtain a first feature matrix;

[0142] Accordingly, learning module 703 is configured to execute:

[0143] The neighbor relationship matrix and the first feature matrix are input into the teacher network graph model to obtain the first graph structure representation output by the teacher network graph model.

[0144] The neighbor relationship matrix and the first feature matrix are input into the graph model to be trained to obtain the second graph structure representation output by the graph model to be trained.

[0145] In one embodiment of the device described in this specification, the training sample processing module 801 is configured to perform: for each matrix element in the feature matrix, setting the value of the matrix element to the value of the corresponding feature of the node corresponding to the matrix element, so as to obtain a second feature matrix;

[0146] Accordingly, learning module 703 is configured to execute:

[0147] The neighbor relationship matrix and the second feature matrix are input into the teacher network graph model to obtain the first graph feature representation output by the teacher network graph model.

[0148] The neighbor relationship matrix and the second feature matrix are input into the graph model to be trained to obtain the second graph feature representation output by the graph model to be trained.

[0149] In one embodiment of the apparatus described in this specification, the difference loss obtaining module 704 is configured to perform:

[0150] The first difference loss is obtained by calculating the mean square error and comparing the structural representations of the first and second graphs.

[0151] The similarity between the feature representations of the first and second images is calculated using the mean square error method to obtain the second difference loss.

[0152] In one embodiment of the apparatus described in this specification, the service loss determination module 705 is configured to perform:

[0153] The second graph feature representation is inferred using a multilayer perceptron, thereby converting the second graph feature representation into a predicted value between 0 and 1;

[0154] The business loss is obtained by calculating the binary classification loss using the predicted values ​​and the label information included in the training samples.

[0155] In one embodiment of the device described in this specification, the model parameter adjustment module 706 is configured to perform: setting the constraint function of the graph model to be trained as: L=loss0+α·loss1+β·loss2;

[0156] Where L represents the constraint function; loss0 represents the business loss; loss1 represents the first difference loss; loss2 represents the second difference loss; α and β are pre-set hyperparameter constraints, and the larger the values ​​of α and β, the greater the weight ratio of the knowledge borrowed from the teacher's network graph model.

[0157] It should be noted that the above-mentioned devices are typically implemented on the server side. They can be set up on independent servers, or some or all of the devices can be combined and installed on the same server. This server can be a single server or a server cluster consisting of multiple servers. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system. The above-mentioned devices can also be implemented on computer terminals with strong computing capabilities.

[0158] This specification provides, in one embodiment, a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods of any embodiment in the specification.

[0159] This specification provides a computing device according to one embodiment, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to perform the method of any embodiment of the specification.

[0160] It is understood that the structures illustrated in the embodiments of this specification do not constitute a specific limitation on the apparatus of the embodiments of this specification. In other embodiments of the specification, the above-described apparatus may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0161] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0162] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, widgets, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0163] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. Graphical model training methods, where, The graph model to be trained is suitable for the first business scenario; The method includes: A teacher network graph model is obtained; wherein the teacher network graph model is applicable to the second business scenario and is a trained graph model; Training samples are obtained from the first business scenario; wherein the training samples are labeled graph data, which includes nodes, edges, features, and the connection relationships between nodes formed by the edges; Training samples are learned using the teacher network graph model and the graph model to be trained, respectively; To obtain the difference loss, a similarity constraint is applied to the learning results of the teacher network graph model on the training samples and the learning results of the graph model to be trained on the training samples. The learning results of the graph model to be trained on the training samples are used to obtain the business loss. Based on the difference loss and the business loss, adjust the model parameters of the graph model to be trained; The step of learning training samples using the teacher network graph model and the graph model to be trained includes: The graph structure is extracted from the training samples using a teacher network graph model to obtain the first graph structure representation; The graph structure is extracted from the training samples using the graph model to be trained, so as to obtain the second graph structure representation; The teacher network graph model is used to extract graph features from the training samples to obtain the first graph feature representation; The graph model to be trained is used to extract graph features from the training samples to obtain the second graph feature representation. Correspondingly, the step of applying a similarity constraint between the learning results of the teacher's network graph model on the training samples and the learning results of the graph model to be trained on the training samples to obtain the difference loss includes: Similarity constraints are applied to the structural representations of the first and second graphs to obtain the first difference loss; A similarity constraint is applied to the feature representations of the first and second images to obtain the second difference loss; The method further includes: A neighbor relationship matrix is ​​obtained based on the training samples; wherein, the neighbor relationship matrix is ​​an N*N matrix, and N is the number of nodes obtained based on the training samples; for any two nodes, if the two nodes are directly connected and form a first-order neighbor relationship, then the value of the matrix element corresponding to the two nodes in the neighbor relationship matrix is ​​1, otherwise it is 0; A feature matrix is ​​obtained based on the training samples; wherein, the feature matrix is ​​an N*M matrix, where M is the number of features included in each node obtained from the training samples; each row of the feature matrix corresponds to a node, and different matrix elements in that row represent different features of that node; Set all matrix elements in the feature matrix to 0 to obtain the first feature matrix; The extraction of graph structure from training samples using a teacher network graph model includes: The neighbor relationship matrix and the first feature matrix are input into the teacher network graph model to obtain the first graph structure representation output by the teacher network graph model. The step of extracting graph structure from training samples using the graph model to be trained includes: The neighbor relationship matrix and the first feature matrix are input into the graph model to be trained to obtain the second graph structure representation output by the graph model to be trained.

2. The method according to claim 1, wherein, The method further includes: For each element in the feature matrix, the value of the element is set to the value of the corresponding feature of the node to which the element belongs, so as to obtain the second feature matrix. The extraction of graph features from training samples using a teacher network graph model includes: The neighbor relationship matrix and the second feature matrix are input into the teacher network graph model to obtain the first graph feature representation output by the teacher network graph model. The step of extracting graph features from training samples using the graph model to be trained includes: The neighbor relationship matrix and the second feature matrix are input into the graph model to be trained to obtain the second graph feature representation output by the graph model to be trained.

3. The method according to claim 1, wherein, The first difference loss is obtained by applying similarity constraints to the structural representations of the first and second graphs, including: The first difference loss is obtained by calculating the mean square error and comparing the structural representations of the first and second graphs. The second difference loss is obtained by applying similarity constraints to the feature representations of the first and second images, including: The similarity between the feature representations of the first and second images is calculated using the mean square error method to obtain the second difference loss.

4. The method according to claim 1, wherein, The process of obtaining the business loss by utilizing the learning results of the graphical model to be trained on the training samples includes: The second graph feature representation is inferred using a multilayer perceptron, thereby converting the second graph feature representation into a predicted value between 0 and 1; The business loss is obtained by calculating the binary classification loss using the predicted values ​​and the label information included in the training samples.

5. The method according to claim 1, wherein, The step of adjusting the parameters of the graph model to be trained based on the difference loss and the business loss includes: The constraint function of the graphical model to be trained is set as: L = loss0 + α·loss1 + β·loss2; Where L represents the constraint function; loss0 represents the business loss; loss1 represents the first difference loss; loss2 represents the second difference loss; α and β are pre-set hyperparameter constraints, and the larger the values ​​of α and β, the greater the weight ratio of the knowledge borrowed from the teacher's network graph model.

6. A graph model training device, wherein, The graph model to be trained is suitable for the first business scenario; The device includes: The teacher network acquisition module is configured to obtain a teacher network graph model; wherein the teacher network graph model is applicable to the second business scenario and is a trained graph model; The training sample acquisition module is configured to obtain training samples from the first business scenario; wherein, the training samples are labeled graph data, which includes nodes, edges, features, and the connection relationships between nodes formed by the edges; The learning module is configured to learn training samples using the teacher network graph model and the graph model to be trained, respectively. The difference loss module is configured to apply similarity constraints to the learning results of the training samples based on the teacher's network graph model and the learning results of the training samples based on the graph model to be trained, so as to obtain the difference loss. The business loss acquisition module is configured to obtain the business loss by utilizing the learning results of the graph model to be trained on the training samples. The model parameter adjustment module is configured to adjust the model parameters of the graph model to be trained based on the difference loss and the business loss. The learning module is configured to perform the following: extracting graph structures from training samples using a teacher network graph model to obtain a first graph structure representation; extracting graph structures from training samples using a graph model to be trained to obtain a second graph structure representation; extracting graph features from training samples using a teacher network graph model to obtain a first graph feature representation; and extracting graph features from training samples using a graph model to be trained to obtain a second graph feature representation. The difference loss obtaining module is configured to perform: applying similarity constraints to the structural representations of the first graph and the structural representations of the second graph to obtain a first difference loss; applying similarity constraints to the feature representations of the first graph and the feature representations of the second graph to obtain a second difference loss; Further includes: a training sample processing module; The training sample processing module is configured to obtain a neighbor relationship matrix based on the training samples; wherein the neighbor relationship matrix is ​​an N*N matrix, and N is the number of nodes obtained from the training samples; for any two nodes, if the two nodes are directly connected and form a first-order neighbor relationship, then the value of the matrix element corresponding to the two nodes in the neighbor relationship matrix is ​​1, otherwise it is 0; and to obtain a feature matrix based on the training samples; wherein the feature matrix is ​​an N*M matrix, and M is the number of features included in each node obtained from the training samples; each row in the feature matrix corresponds to a node, and different matrix elements in the row represent different features of the node; and all matrix elements in the feature matrix are set to 0 to obtain a first feature matrix; The learning module is also configured to execute: The neighbor relationship matrix and the first feature matrix are input into the teacher network graph model to obtain the first graph structure representation output by the teacher network graph model. The neighbor relationship matrix and the first feature matrix are input into the graph model to be trained to obtain the second graph structure representation output by the graph model to be trained.

7. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-5.

Images