Training method and device of attribute prediction model, attribute prediction method and device

Abstract

Embodiments of the present specification describe a training method and device of an attribute prediction model, and an attribute prediction method. According to the method of the embodiments, firstly, a graph network constructed by using network behavior data needs to be split into multiple sub-networks. Then, a loss function value of a first node in each sub-network is calculated, and a weight value when the loss function value meets a distribution drift constraint is calculated, and further, the obtained weight value can be used to train an attribute prediction model. Since the weight value is obtained under the condition that the loss function distribution drift constraint is met, that is, the inconsistency rule of the model training data and the prediction data over time is constrained. Therefore, based on the present scheme, the problem of model performance degradation caused by distribution drift can be effectively alleviated, thereby improving the accuracy of attribute prediction using the model.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of computer technology, and more particularly to training methods, attribute prediction methods and apparatus for attribute prediction models. Background Technology

[0002] Training models based on graph networks is a common technique in attribute prediction scenarios, such as customer recommendation and risk account prediction.

[0003] However, in many business scenarios, the graph structure changes over time, which means that the graph data used for model training and the graph data used for prediction may differ. For example, for a certain risky account, the transfer process before it is treated as a risky account usually differs from the transfer process after it is treated as a risky account.

[0004] Therefore, the accuracy of current graph network-based models for attribute prediction is relatively low. Summary of the Invention

[0005] This specification describes one or more embodiments of a training method, an attribute prediction method, and an apparatus for an attribute prediction model, which can improve the accuracy of the model when predicting attributes.

[0006] Based on the first aspect, a training method for the attribute prediction model is provided, including:

[0007] Construct a graph network based on network behavior data recorded in business scenarios;

[0008] At least two sub-networks are obtained from the graph network; each sub-network includes a first node and at least one second node connected to the first node, and the nodes in any two sub-networks are not exactly the same; the nodes are entities obtained based on the network behavior data.

[0009] For each subnetwork, calculate the loss function value of the first node in the current subnetwork relative to the label value;

[0010] Based on the values ​​of each loss function, the weight values ​​of the loss functions of each sub-network that satisfy the distribution drift constraint are calculated; wherein, the distribution drift constraint is used to characterize the constraint on the inconsistent patterns of the model training data and prediction data over time.

[0011] The attribute prediction model is trained using the weight values ​​of the obtained loss function.

[0012] In one possible implementation, obtaining at least two sub-networks from the graph network includes:

[0013] In the graph network, with the first node as the base point, it extends to other nodes with connected edges according to at least two paths; wherein, the path includes at least one of the direction of extension and the depth of extension;

[0014] The at least two sets of nodes obtained through the at least two paths are determined as the at least two sub-networks; wherein each path yields a set of nodes, and each set of nodes constitutes a sub-network.

[0015] In one possible implementation, calculating the loss function value of the first node in the current sub-network relative to the label value includes:

[0016] Calculate the fusion representation of the first node in the current sub-network; wherein the fusion representation is used to represent the contribution of all nodes in the current sub-network to the prediction of the attributes of the first node.

[0017] Based on the fusion representation of the first node and the pre-set label information, the loss function value of the first node in the current sub-network is calculated.

[0018] In one possible implementation, calculating the fusion representation of the first node in the current sub-network includes:

[0019] For each hidden layer during the training of the attribute prediction model, the following is performed:

[0020] For each second node in the current sub-network, the representation of the second node is fused into the representation of the first node to obtain the first fused representation corresponding to the second node.

[0021] The first fusion representations obtained from each second node are aggregated to obtain the second fusion representation;

[0022] The second fusion representation is merged with the fusion representation of the first node obtained in the previous hidden layer of the current sub-network to obtain the fusion representation of the first node in the current hidden layer of the current sub-network.

[0023] In one possible implementation, fusing the representation of the second node into the representation of the first node to obtain a first fused representation corresponding to the second node includes:

[0024] The first fusion representation is calculated using the following formula:

[0025] h′=δ(H u H v )

[0026] Where h′ is used to characterize the first fusion characterization, H uThe representation used to characterize the first node, H v The representation used to characterize the second node, δ(·) is the activation function.

[0027] In one possible implementation, the aggregation of the first fusion representations obtained for each second node to obtain the second fusion representation includes:

[0028] The second fusion representation is calculated using the following formula:

[0029] h″=AGG(h′1,h′2,…,h′ j ,…,h′ n )

[0030] Where h″ is used to characterize the second fusion characterization, h′ j This is used to characterize the first fusion representation obtained through the j-th second node in the current sub-network, where n is the number of second nodes in the current sub-network, and AGG is the aggregation function.

[0031] In one possible implementation, merging the second fused representation with the fused representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fused representation of the first node in the current sub-network at the current hidden layer includes:

[0032] The fusion representation is calculated using the following formula:

[0033] h (l+1) =COMBINE(h (l) ,h″)

[0034] Among them, h (l+1) h is used to characterize the fusion representation in the current (l+1)th hidden layer. (l) The fusion representation obtained from the l-th hidden layer is used to characterize the l-th hidden layer, and the l-th hidden layer is the previous hidden layer of the (l+1)-th hidden layer. h″ is used to characterize the second fusion representation, and COMBINE(·) is the merging function.

[0035] In one possible implementation, calculating the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on each of the loss function values ​​includes:

[0036] Under the condition that the distance between the probability distribution of the actual weight values ​​corresponding to each loss function value and the first probability distribution is less than a first preset value, the weight value when the loss function reaches its maximum value is calculated; wherein, the first probability distribution is used to characterize the probability distribution when the weight values ​​corresponding to each loss function value satisfy a uniform distribution.

[0037] In one possible implementation, calculating the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on each of the loss function values ​​includes:

[0038] The weights of each subnetwork are calculated using the following formula:

[0039]

[0040] Among them, Q * The set of weights used to characterize the various subnetworks, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks. Used to characterize Q and The divergence between them, Q, is used to characterize the probability distribution satisfied by the actual weights corresponding to each loss function value. The probability distribution of the weight values ​​corresponding to each loss function value when they satisfy a uniform distribution is used to characterize the first preset value, and ρ is used to characterize the weight value when the loss value is maximized.

[0041] In one possible implementation, training the attribute prediction model using the weight values ​​of the obtained loss function includes:

[0042] Calculate the weighted average of the weights of the loss functions corresponding to each subnetwork; and,

[0043] The attribute prediction model is trained using a stochastic gradient descent algorithm based on the weighted average of the weight values.

[0044] In one possible implementation, the calculation of the weighted average of the weight values ​​of the loss function corresponding to each sub-network includes:

[0045] Calculate the weighted average of the weights of each subnetwork using the following formula:

[0046]

[0047] Where q is used to characterize the weighted average of the weights of each sub-network, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks.

[0048] Based on the second aspect, attribute prediction methods are provided, including:

[0049] Based on network behavior data occurring in the business scenario, nodes for attribute prediction are obtained;

[0050] The information of the node to be attribute predicted is input into the attribute prediction model to obtain the attribute prediction result of the node to be attribute predicted output by the attribute prediction model; wherein, the attribute prediction model is trained using any of the methods described in the first aspect.

[0051] According to the third aspect, a training device for the attribute prediction model is provided, including: a graph network construction module, a sub-network splitting module, a loss calculation module, a weight calculation module, and a model training module;

[0052] The graph network construction module is configured to construct a graph network based on network behavior data recorded in business scenarios.

[0053] The sub-network splitting module is configured to obtain at least two sub-networks from the graph network obtained by the graph network construction module; wherein each sub-network includes a first node and at least one second node that has an edge relationship with the first node, and the nodes contained in any two sub-networks are not completely the same; the node is an entity obtained based on the network behavior data;

[0054] The loss calculation module is configured to calculate the loss function value of the first node in the current sub-network relative to the label value for each sub-network obtained by the sub-network splitting module.

[0055] The weight calculation module is configured to calculate the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on the loss function values ​​obtained by the loss calculation module; wherein, the distribution drift constraint is used to characterize the constraint on the inconsistent patterns of the model training data and prediction data over time.

[0056] The model training module is configured to train the attribute prediction model using the weight values ​​of the loss function obtained by the weight calculation module.

[0057] According to the fourth aspect, an attribute prediction module is provided, including: a module for obtaining nodes to be predicted and a module for outputting attribute prediction results;

[0058] The node acquisition module is configured to obtain the node to be predicted based on network behavior data occurring in the business scenario.

[0059] The attribute prediction result output module is configured to input the information of the node to be predicted, obtained by the node acquisition module, into the attribute prediction model to obtain the attribute prediction result of the node to be predicted output by the attribute prediction model; wherein, the attribute prediction model is trained using the training device of the attribute prediction model described in the third aspect.

[0060] According to a fifth aspect, a computing device is provided, comprising: a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of the first and second aspects described above.

[0061] According to the method and apparatus provided in the embodiments of this specification, when training an attribute prediction model, a graph network is first constructed based on network behavior data recorded in a business scenario, and at least two sub-networks are obtained from this graph network. Then, the loss function value of the first node in each sub-network is calculated. Further, based on the calculated loss function values, weight values ​​that satisfy the distribution drift constraint of the loss function are calculated. The attribute prediction model can then be trained using these weight values. Therefore, the weight values ​​in this scheme are obtained under the condition of satisfying the distribution drift constraint of the loss function, thus constraining the inconsistent patterns of the model training data and prediction data over time. Therefore, this scheme can effectively alleviate the problem of model performance degradation caused by distribution drift, thereby improving the accuracy of attribute prediction using this model. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating a training method for an attribute prediction model provided in one embodiment of this specification;

[0064] Figure 2 This is a flowchart of a graph network decomposition method provided in one embodiment of this specification;

[0065] Figure 3 This is a schematic diagram of graph network decomposition provided in one embodiment of this specification;

[0066] Figure 4 This is a flowchart illustrating a method for calculating a loss function value according to one embodiment of this specification;

[0067] Figure 5 This is a flowchart illustrating a method for calculating the fusion representation of a first node according to an embodiment of this specification;

[0068] Figure 6 This is a flowchart of an attribute prediction method provided in one embodiment of this specification;

[0069] Figure 7 This is a schematic diagram of a training device for an attribute prediction model provided in one embodiment of this specification;

[0070] Figure 8 This is a schematic diagram of an attribute prediction device provided in one embodiment of this specification. Detailed Implementation

[0071] As mentioned earlier, many business scenarios involve uncertainties in data distribution drift, meaning that the evolutionary patterns of graph networks may not change consistently over time. This inconsistency means that the time dependencies of the training and testing graph network data may differ. If graph networks are used to optimize the loss of the training data, generalization failure on the testing data may occur due to the inconsistency in time dependencies. For example, if account A is a risky account and is treated as such at time T, users of account A may use different transfer processes because they are being treated as risky accounts. Thus, for account A, the transfer processes before and after being treated as risky accounts are inconsistent. If a model trained before being treated as a risky account is used for risk account prediction, it may be unable to predict the risky account A.

[0072] Based on this, this scheme considers constraining the distribution drift. The resulting loss function can reduce the distribution drift problem caused by time changes, thereby improving the accuracy of the model in predicting attributes.

[0073] like Figure 1 As shown in the embodiments of this specification, a training method for an attribute prediction model is provided, which may include the following steps:

[0074] Step 101: Construct a graph network based on network behavior data recorded in the business scenario;

[0075] Step 103: Obtain at least two sub-networks from the graph network; each sub-network includes a first node and at least one second node connected to the first node, and the nodes contained in any two sub-networks are not completely identical; the nodes are entities based on network behavior data.

[0076] Step 105: For each sub-network, calculate the loss function value of the first node in the current sub-network relative to the label value;

[0077] Step 107: Based on the values ​​of each loss function, calculate the weight values ​​of the loss function of each sub-network that satisfy the distribution drift constraint; whereby the distribution drift constraint is used to characterize the inconsistent patterns of the model training data and prediction data over time.

[0078] Step 109: Use the weight values ​​of the obtained loss function to train the attribute prediction model.

[0079] In this embodiment, when training the attribute prediction model, a graph network is first constructed based on network behavior data recorded in the business scenario, and at least two sub-networks are obtained from this graph network. Then, the loss function value of the first node in each sub-network is calculated. Further, based on the calculated loss function values, weight values ​​that satisfy the distribution drift constraint of the loss function are calculated. The attribute prediction model can then be trained using these weight values. Therefore, the weight values ​​in this scheme are obtained under the condition of satisfying the distribution drift constraint of the loss function, thus constraining the inconsistent patterns of the model training data and prediction data over time. Therefore, this scheme can effectively alleviate the model performance degradation problem caused by distribution drift, thereby improving the accuracy of attribute prediction using this model.

[0080] The following describes the appendix in conjunction with specific embodiments. Figure 1 The steps in the process will be explained.

[0081] First, in step 101, a graph network is constructed based on the network behavior data recorded in the business scenario.

[0082] A large amount of network behavior data is recorded in various business scenarios. This includes transaction data between accounts, basic user information, and account login records. Graph networks can be constructed based on the network behavior data recorded in these business scenarios. For example, in risk control, when determining whether an account is a risky account, network behavior data can include transaction connections between that account and other accounts, device information associated with that account, and bank card information associated with that account.

[0083] When predicting risky accounts, the accounts included in the network behavior data can be used as nodes, and edges can be established based on the relationships between these accounts. For example, for four accounts A, B, C, and D, if account A has transaction records with C and D respectively, then A and C have an edge relationship, and A and D also have an edge relationship; if B and C have transaction records, then B and C also have an edge relationship. By using each account as a node and establishing edges based on the relationships between accounts in the network behavior data, a graph network can be obtained.

[0084] For example, in the business scenario of merchant evaluation, network behavior data can be the merchant's operational data; that is, a graph network can be a network constructed based on the merchant's operational data. For instance, when predicting whether a merchant is a high-quality merchant, the graph network can be constructed using the merchant's operational data and credit rating information. Nodes in the graph network can include the merchant to be predicted, other different merchants, official entities, etc. Edge relationships can be constructed based on whether there are connections between the merchant to be predicted, other merchants, and official entities in the operational data and credit rating information.

[0085] Then in step 103, at least two subnetworks are obtained from the graph network.

[0086] Because industrial graphs contain such large amounts of data, they can severely impact model training efficiency and quality. Therefore, this step considers dividing the graph network into multiple sub-networks for model training, thereby improving training efficiency and quality.

[0087] It is easy to understand that each subnetwork obtained from the graph network should include a first node to be monitored, and at least one second node connected to the first node to be monitored, and the nodes contained in any two subnetworks are not exactly the same. The first node to be monitored can be manually specified when constructing the graph network. For example, when predicting the attributes of the first node to predict risky accounts, an account that has historically exhibited risky behavior can be designated as the first node. Each node is an entity derived from network behavior data. As described in step 101, a node can be any account contained in transaction data, or any merchant or official account contained in operational data and credit rating information.

[0088] In one possible implementation, step 103, when obtaining at least two subnetworks from the graph network, such as... Figure 2 As shown, this can be achieved through the following steps:

[0089] Step 201: In the graph network, starting from the first node, extend to other nodes with connected edges according to at least two paths; wherein, the path includes at least one of the direction of extension and the depth of extension;

[0090] Step 203: Determine at least two sets of nodes obtained through at least two paths as at least two sub-networks; wherein each path yields a set of nodes, and each set of nodes constitutes a sub-network.

[0091] In this embodiment, when collecting subnetworks from a graph network, the process begins by using the first node of interest as a base point and extending it to other nodes with connected edges along at least two paths. These paths can extend in different directions or to different depths. This allows the set of nodes obtained through at least two paths to be identified as the subnetwork.

[0092] For example, such as Figure 3 In the diagram illustrating graph network decomposition, the node of interest, A, can be extended along paths B and C to form a subnetwork; it can also be extended along paths D, E, and F to form a subnetwork; or it can be taken as the base node A and extended along paths B and D to form a subnetwork. Each path yields a set of nodes, and each set of nodes constitutes a subnetwork, with each subnetwork including at least one second node. Thus, by using different paths—that is, extending along different directions and / or to different depths from the base node A—it is possible to determine which nodes are included in each subnetwork, the number of nodes, etc., thereby obtaining different subnetworks.

[0093] Of course, it should be noted that a graph network can include at least two nodes of interest, so that when dividing the network into subnetworks, the first node in each subnetwork is unique. When a subnetwork obtained through extension includes multiple nodes of interest, only the node that extends from the base point is the first node in that subnetwork, while the other nodes of interest in that subnetwork are the second nodes that have an edge connection with the first node.

[0094] Then in step 105, for each sub-network, the loss function value of the first node in the current sub-network relative to the label value is calculated.

[0095] In this step, when each sub-network is obtained in step 103, it is possible to input the sub-network into a graph neural network to obtain the representation of the first node, and then calculate the loss function value corresponding to each sub-network by combining the label information. For example, in one possible implementation, such as... Figure 4 As shown, step 105 can be achieved through the following steps:

[0096] Step 401: Calculate the fusion representation of the first node in the current sub-network; wherein, the fusion representation is used to represent the contribution of all nodes in the current sub-network to the prediction of the attributes of the first node;

[0097] Step 403: Calculate the loss function value of the first node in the current sub-network based on the fusion representation of the first node and the pre-set label information.

[0098] In this embodiment, after obtaining at least two subnetworks from the graph network, the fusion representation of the first node in each subnetwork is first calculated. Then, based on the fusion representation of the first node and pre-defined label information, the loss function value of the first node in the current subnetwork is calculated. The fusion representation of each subnetwork represents the contribution of all nodes in that subnetwork to the prediction of the attributes of the first node. Therefore, the model trained using this fusion representation fully considers the correlation between the subnetworks in each subnetwork, resulting in a model with better performance.

[0099] Step 401 will be explained below.

[0100] Step 401 fully considers the correlation between each node and the first node to be predicted for attribute prediction, integrating the contributions of each node to attribute prediction into the representation of the first node, thereby obtaining a more comprehensive fused representation. In one possible implementation, such as... Figure 5 As shown, in step 401, when calculating the fusion representation of the first node in the current sub-network, the following steps can be performed for each hidden layer during the training of the attribute prediction model:

[0101] Step 501: For each second node in the current sub-network, merge the representation of the second node into the representation of the first node to obtain the first fused representation corresponding to the second node.

[0102] Step 503: Aggregate the first fusion representations obtained for each second node to obtain the second fusion representation;

[0103] Step 505: Merge the second fusion representation with the fusion representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fusion representation of the first node in the current hidden layer in the current sub-network.

[0104] In this embodiment, when calculating the fused representation of the first node in the current sub-network, the representation of each second node in the current sub-network is first fused into the representation of the first node to obtain the first fused representation corresponding to that second node. Then, the first fused representations obtained for each second node are aggregated to obtain the second fused representation. Further, the second fused representation is merged with the fused representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fused representation of the first node in the current hidden layer of the current sub-network. Therefore, this scheme fully considers the contribution of each second node in the sub-network to the first node, resulting in a more comprehensive fused representation that reflects the attribute information of the first node, which helps improve the accuracy of attribute prediction for the first node.

[0105] In step 501, when fusing the representation of the second node in the current sub-network into the representation of the first node to obtain the first fused representation corresponding to the second node, in one possible implementation, the first fused representation can be calculated using the following formula:

[0106] h′=δ(H u H v )

[0107] Where h′ is used to characterize the first fusion characterization, H u The representation used to characterize the first node, H v The representation used to characterize the second node, δ(·) is the activation function.

[0108] Step 503, when aggregating the first fusion representations obtained from each second node to obtain the second fusion representation, can use the following formula to calculate the second fusion representation:

[0109] h″=AGG(h′1,h′2,…,h′ j ,…,h′ n )

[0110] Where h″ is used to characterize the second fusion representation, h′ j This is used to characterize the first fusion representation obtained through the j-th second node in the current sub-network, where n is the number of second nodes in the current sub-network, and AGG is the aggregation function.

[0111] In step 505, when merging the second fused representation with the fused representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fused representation of the first node in the current hidden layer in the current sub-network, the following formula can be used to calculate the fused representation:

[0112] h (l+1) =COMBINE(h (l) ,h″)

[0113] Among them, h (l+1) h is used to characterize the fusion representation in the current (l+1)th hidden layer. (l) The fusion representation obtained from the l-th hidden layer is used to characterize the l-th hidden layer, and the l-th hidden layer is the previous hidden layer of the (l+1)-th hidden layer. h″ is used to characterize the second fusion representation, and COMBINE(·) is the merging function.

[0114] The embodiments in this specification can consider pre-setting the number of hidden layers. That is, by fusing features across multiple layers, more data related to the attribute prediction of the first node can be fused. For example, through the first layer of fusion, data from the neighboring nodes of the first node can be fused into the fused representation; through the second layer of fusion, data from the neighboring nodes of the first node's neighboring nodes can be further fused into the fused representation. And so on. By setting the number of hidden layers, the degree of fusion of related nodes is determined, thereby obtaining fused representations at different depths for model training and attribute prediction.

[0115] Step 403 will be explained below.

[0116] After obtaining the fused representation of the first node in the current sub-network through step 401, the loss function value of the first node in the current sub-network is calculated based on the fused representation of the first node and the pre-set label information. The loss function can be selected according to the types of upstream and downstream tasks, such as mean squared error loss and cross-entropy loss. In this way, N loss function values ​​can be obtained for N sub-networks.

[0117] Further in step 107, based on the values ​​of each loss function, the weight values ​​of the loss functions of each sub-network that satisfy the distribution drift constraint are calculated.

[0118] As mentioned earlier, data distribution drift exists in many business scenarios, meaning the evolution of graph networks over time is not always consistent. This inconsistency can lead to different time dependencies between the training and prediction data. In other words, if data distribution drift occurs, the trained model may not achieve optimal prediction results. Therefore, this solution considers constraining the inconsistent patterns of the training and prediction data over time by calculating weight values ​​under distribution drift constraints. For example, in one possible implementation, in step 107, when calculating the weight values ​​for each sub-network to satisfy the distribution drift constraint based on the loss function values, the weight value at which the loss function reaches its maximum value can be calculated if the distance between the probability distribution of the actual weight values ​​corresponding to each loss function value and the first probability distribution is less than a first preset value. Here, the first probability distribution is the probability distribution when the weight values ​​corresponding to each loss function value satisfy a uniform distribution. Thus, since the weight values ​​for each sub-network are calculated under the condition of satisfying the distribution drift constraint, the model trained based on these weight values ​​exhibits good robustness, thereby improving model performance.

[0119] In one possible implementation, step 107 can calculate the weight values ​​of each sub-network using the following formula:

[0120]

[0121] Among them, Q * The set of weights used to characterize the various subnetworks, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks. Used to characterize Q and The divergence between them, Q, is used to characterize the probability distribution satisfied by the actual weights corresponding to each loss function value. The probability distribution of the weight values ​​corresponding to each loss function value when they satisfy a uniform distribution is used to characterize the first preset value, and ρ is used to characterize the weight value when the loss value is maximized.

[0122] As shown in the calculation formula above, For Q and The divergence between them, where Q is the probability distribution satisfied by the actual weights corresponding to each loss function value. The probability distribution of the weights corresponding to each loss function value is such that it satisfies a uniform distribution, meaning that for N loss function values, the probability distribution of a uniform distribution is 1 / N. By limiting the distance between two probability distributions to a first preset value ρ when assigning weight values, the robustness of the model is enhanced, thereby mitigating the performance degradation caused by distribution drift.

[0123] Therefore, in step 107, the input is the values ​​of N loss functions, and the output is the final weights of the N loss function values. The goal is: the larger the loss function value, the larger the weight; when the loss function value is very small, the weight is set to 0. The specific weighting method depends on the preset distribution perturbation radius, i.e., the first preset value ρ. In the above formula, Q is a set of weights q_1, ..., q_n, satisfying q_1 + q_2 + ... + q_n = 1, and Q is the probability distribution satisfied by the actual weights corresponding to each loss function value. The D_fk divergence between Q and the probability distribution 1 / n of the uniform distribution corresponding to each loss function value is within the perturbation radius ρ, which is essentially a constraint on distribution drift.

[0124] Finally, in step 109, the attribute prediction model is trained using the weight values ​​of the obtained loss function.

[0125] In this step, after obtaining the weight values ​​of each loss function, we can first consider calculating the weighted average of the weight values ​​of the loss functions corresponding to each sub-network, and then use the stochastic gradient algorithm to train the attribute prediction model based on the weighted average of the weight values.

[0126] When calculating the weighted average of the weights of the loss function corresponding to each sub-network, the following formula can be used:

[0127]

[0128] Where q is used to characterize the weighted average of the weights of each sub-network, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks.

[0129] Therefore, this embodiment assigns different weights to different sub-networks during training. After obtaining the weights qi, the loss function is applied to q. i The weighted sum is used as the learning objective, and stochastic gradient descent with backpropagation is used to optimize the objective function, thereby training a higher-performing attribute prediction model.

[0130] like Figure 6 As shown in the embodiments of this specification, an attribute prediction method is also provided, which may include the following steps:

[0131] Step 601: Based on network behavior data occurring in the business scenario, obtain the nodes to be predicted for attributes;

[0132] Step 603: Input the information of the node to be attribute predicted into the attribute prediction model to obtain the attribute prediction result of the node to be attribute predicted output by the attribute prediction model; wherein, the attribute prediction model is trained using the training method of any of the attribute prediction models in the above embodiments.

[0133] Since the weight values ​​in the above-mentioned attribute prediction model are obtained under the condition of satisfying the loss function distribution drift constraint, that is, the inconsistency between the model training data and the predicted data over time is constrained. Therefore, the attribute prediction model obtained by this scheme can effectively alleviate the problem of model performance degradation caused by distribution drift when performing attribute prediction, thereby improving the accuracy of attribute prediction using the model.

[0134] like Figure 7 As shown in the embodiments of this specification, a training device for an attribute prediction model is also provided. The device includes: a graph network construction module 701, a sub-network splitting module 702, a loss calculation module 703, a weight calculation module 704, and a model training module 705.

[0135] Graph network construction module 701 is configured to construct a graph network based on network behavior data recorded in business scenarios.

[0136] The subnetwork splitting module 702 is configured to obtain at least two subnetworks from the graph network obtained by the graph network construction module 701; wherein each subnetwork includes a first node and at least one second node that has an edge relationship with the first node, and the nodes contained in any two subnetworks are not completely the same; the nodes are entities obtained based on network behavior data;

[0137] The loss calculation module 703 is configured to calculate the loss function value of the first node in the current sub-network relative to the label value for each sub-network obtained by the sub-network splitting module 702.

[0138] The weight calculation module 704 is configured to calculate the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on the loss function values ​​obtained by the loss calculation module 703; wherein, the distribution drift constraint is used to characterize the constraint on the inconsistent patterns of the model training data and prediction data over time.

[0139] The model training module 705 is configured to train the attribute prediction model using the weight values ​​of the loss function obtained by the weight calculation module 704.

[0140] In one possible implementation, the subnetwork splitting module 702, when obtaining at least two subnetworks from the graph network, is configured to perform the following operation:

[0141] In a graph network, starting from the first node, the network extends to other nodes with connected edges according to at least two paths; wherein the path includes at least one of the direction of extension and the depth of extension.

[0142] At least two sets of nodes obtained through at least two paths are defined as at least two subnetworks; wherein each path yields a set of nodes, and each set of nodes constitutes a subnetwork.

[0143] In one possible implementation, the loss calculation module 703 is configured to perform the following operation when calculating the loss function value of the first node in the current sub-network relative to the label value:

[0144] Calculate the fusion representation of the first node in the current sub-network; where the fusion representation is used to characterize the contribution of all nodes in the current sub-network to the prediction of the attributes of the first node;

[0145] Based on the fusion representation of the first node and the pre-defined label information, calculate the loss function value of the first node in the current sub-network.

[0146] In one possible implementation, the loss calculation module 703 is configured to perform the following operation when calculating the fusion representation of the first node in the current sub-network:

[0147] For each hidden layer during the training of the attribute prediction model, perform the following:

[0148] For each second node in the current sub-network, the representation of the second node is merged into the representation of the first node to obtain the first fused representation corresponding to the second node.

[0149] The first fusion representations obtained from each second node are aggregated to obtain the second fusion representation;

[0150] The second fusion representation is merged with the fusion representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fusion representation of the first node in the current hidden layer in the current sub-network.

[0151] In one possible implementation, when the loss calculation module 703 fuses the representation of the second node into the representation of the first node to obtain a first fused representation corresponding to the second node, it is configured to calculate the first fused representation using the following formula:

[0152] h′=δ(H u H v )

[0153] Where h′ is used to characterize the first fusion characterization, H u The representation used to characterize the first node, H v The representation used to characterize the second node, δ(·) is the activation function.

[0154] In one possible implementation, when the loss calculation module 703 aggregates the first fusion representations obtained from each second node to obtain the second fusion representation, it is configured to calculate the second fusion representation using the following formula:

[0155] h″=AGG(h′1,h′2,…,h′ j ,…,h′ n )

[0156] Where h″ is used to characterize the second fusion representation, h′ j This is used to characterize the first fusion representation obtained through the j-th second node in the current sub-network, where n is the number of second nodes in the current sub-network, and AGG is the aggregation function.

[0157] In one possible implementation, when the loss calculation module 703 merges the second fused representation with the fused representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fused representation of the first node in the current hidden layer of the current sub-network, it is configured to calculate the fused representation using the following formula:

[0158] h (l+1)=COMBINE(h (l) ,h″)

[0159] Among them, h (l+1) h is used to characterize the fusion representation in the current (l+1)th hidden layer. (l) The fusion representation obtained from the l-th hidden layer is used to characterize the l-th hidden layer, and the l-th hidden layer is the previous hidden layer of the (l+1)-th hidden layer. h″ is used to characterize the second fusion representation, and COMBINE(·) is the merging function.

[0160] In one possible implementation, when the weight calculation module 704 calculates the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on each loss function value, it is configured to calculate the weight value when the loss function reaches its maximum value, provided that the distance between the probability distribution satisfied by the actual weight values ​​corresponding to each loss function value and the first probability distribution is less than a first preset value; wherein, the first probability distribution is used to characterize the probability distribution when the weight values ​​corresponding to each loss function value satisfy a uniform distribution.

[0161] In one possible implementation, when calculating the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on the values ​​of each loss function, the weight calculation module 704 is configured to calculate the weight values ​​of each sub-network using the following formula:

[0162]

[0163] Among them, Q * The set of weights used to characterize the various subnetworks, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks. Used to characterize Q and The divergence between them, Q, is used to characterize the probability distribution satisfied by the actual weights corresponding to each loss function value. The probability distribution of the weight values ​​corresponding to each loss function value when they satisfy a uniform distribution is used to characterize the first preset value, and ρ is used to characterize the weight value when the loss value is maximized.

[0164] In one possible implementation, the model training module 705, when training the attribute prediction model using the weight values ​​of the obtained loss function, is configured to perform the following operation:

[0165] Calculate the weighted average of the weights of the loss functions corresponding to each subnetwork; and,

[0166] The attribute prediction model is trained using the stochastic gradient algorithm based on the weighted average of the weight values.

[0167] In one possible implementation, the model training module 705 is configured to calculate the weighted average of the weights of the loss function corresponding to each sub-network using the following formula:

[0168]

[0169] Where q is used to characterize the weighted average of the weights of each sub-network, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks.

[0170] like Figure 8 As shown in the embodiments of this specification, an attribute prediction device is also provided, which includes: a node acquisition module 801 to be predicted and an attribute prediction result output module 802.

[0171] The node acquisition module 801 is configured to obtain the nodes to be predicted based on network behavior data occurring in the business scenario.

[0172] The attribute prediction result output module 802 is configured to input the information of the node to be predicted, which is obtained by the node acquisition module 801, into the attribute prediction model to obtain the attribute prediction result of the node to be predicted output by the attribute prediction model; wherein, the attribute prediction model is trained using the training device of the attribute prediction model provided in the above embodiments.

[0173] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods in any of the embodiments of the specification.

[0174] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any of the embodiments in this specification.

[0175] It is understood that the structures illustrated in the embodiments of this specification do not constitute a specific limitation on the training device and the attribute prediction device for the attribute prediction model. In other embodiments of this specification, the training device and the attribute prediction device for the attribute prediction model may include more or fewer components than illustrated, or combine some components, split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0176] The information interaction and execution process between the various units in the above-mentioned device are based on the same concept as the method embodiments in this specification, and the specific details can be found in the descriptions in the method embodiments in this specification, so they will not be repeated here.

[0177] Those skilled in the art will recognize that, in one or more of the examples above, the functions described herein can be implemented using hardware, software, widgets, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0178] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects described in this specification. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. Training methods for attribute prediction models, including: A graph network is constructed based on network behavior data recorded in business scenarios. The business scenarios include: determining whether an account is a risky account in the field of risk prevention and control. The network behavior data includes: transaction connections between accounts and other accounts, device information associated with accounts, and bank card information associated with accounts. The graph network is constructed by establishing connections between accounts, using each account included in the network behavior data as a node. At least two sub-networks are obtained from the graph network; each sub-network includes a first node and at least one second node connected to the first node, and the nodes in any two sub-networks are not exactly the same; the nodes are entities obtained based on the network behavior data. For each subnetwork, calculate the loss function value of the first node in the current subnetwork relative to the label value; Based on the values ​​of each loss function, the weight values ​​of the loss functions of each sub-network that satisfy the distribution drift constraint are calculated; wherein, the distribution drift constraint is used to characterize the constraint on the inconsistent patterns of the model training data and prediction data over time. The attribute prediction model is trained using the weight values ​​of the obtained loss function.

2. The method according to claim 1, wherein, The process of obtaining at least two sub-networks from the graph network includes: In the graph network, with the first node as the base point, it extends to other nodes with connected edges according to at least two paths; wherein, the path includes at least one of the direction of extension and the depth of extension; The at least two sets of nodes obtained through the at least two paths are determined as the at least two sub-networks; wherein each path yields a set of nodes, and each set of nodes constitutes a sub-network.

3. The method according to claim 1, wherein, The calculation of the loss function value of the first node in the current sub-network relative to the label value includes: Calculate the fusion representation of the first node in the current sub-network; wherein the fusion representation is used to represent the contribution of all nodes in the current sub-network to the prediction of the attributes of the first node. Based on the fusion representation of the first node and the pre-set label information, the loss function value of the first node in the current sub-network is calculated.

4. The method according to claim 3, wherein, The calculation of the fusion representation of the first node in the current sub-network includes: For each hidden layer during the training of the attribute prediction model, the following is performed: For each second node in the current sub-network, the representation of the second node is fused into the representation of the first node to obtain the first fused representation corresponding to the second node. The first fusion representations obtained from each second node are aggregated to obtain the second fusion representation; The second fusion representation is merged with the fusion representation of the first node obtained in the previous hidden layer of the current sub-network to obtain the fusion representation of the first node in the current hidden layer of the current sub-network.

5. The method according to claim 4, wherein, The step of fusing the representation of the second node into the representation of the first node to obtain a first fused representation corresponding to the second node includes: The first fusion representation is calculated using the following formula: h′=δ(H u ,H v ) Where h′ is used to characterize the first fusion characterization, H u The representation used to characterize the first node, H v The representation used to characterize the second node, δ(·) is the activation function.

6. The method according to claim 4, wherein, The aggregation of the first fusion representations obtained for each second node to obtain the second fusion representation includes: The second fusion representation is calculated using the following formula: h″=AGG(h′1,h′2,…,h′ j ,…,h′ n ) Where h″ is used to characterize the second fusion characterization, h′ j This is used to characterize the first fusion representation obtained through the j-th second node in the current sub-network, where n is the number of second nodes in the current sub-network, and AGG is the aggregation function.

7. The method according to claim 4, wherein, The step of merging the second fused representation with the fused representation of the first node obtained from the previous hidden layer in the current sub-network to obtain the fused representation of the first node in the current hidden layer in the current sub-network includes: The fusion representation is calculated using the following formula: h (l+1) =COMBINE(h (l) ,h″) Among them, h (l+1) h is used to characterize the fusion representation in the current (l+1)th hidden layer. (l) The fusion representation obtained from the l-th hidden layer is used to characterize the l-th hidden layer, and the l-th hidden layer is the previous hidden layer of the (l+1)-th hidden layer. h″ is used to characterize the second fusion representation, and COMBINE(·) is the merging function.

8. The method according to claim 1, wherein, The step of calculating the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on each of the aforementioned loss function values ​​includes: Under the condition that the distance between the probability distribution of the actual weight values ​​corresponding to each loss function value and the first probability distribution is less than a first preset value, the weight value when the loss function reaches its maximum value is calculated; wherein, the first probability distribution is used to characterize the probability distribution when the weight values ​​corresponding to each loss function value satisfy a uniform distribution.

9. The method according to any one of claims 1 to 8, wherein, The step of calculating the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on each of the aforementioned loss function values ​​includes: The weights of each subnetwork are calculated using the following formula: Among them, Q * The set of weights used to characterize the various subnetworks, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks. Used to characterize Q and The divergence between them, Q, is used to characterize the probability distribution satisfied by the actual weights corresponding to each loss function value. The probability distribution of the weight values ​​corresponding to each loss function value when they satisfy a uniform distribution is used to characterize the first preset value, and ρ is used to characterize the weight value when the loss value is maximized.

10. The method according to claim 1, wherein, The step of training the attribute prediction model using the weight values ​​of the obtained loss function includes: Calculate the weighted average of the weights of the loss functions corresponding to each subnetwork; and, The attribute prediction model is trained using a stochastic gradient descent algorithm based on the weighted average of the weight values.

11. The method according to claim 10, wherein, The calculation of the weighted average of the weight values ​​of the loss function corresponding to each sub-network includes: Calculate the weighted average of the weights of each subnetwork using the following formula: Where q is used to characterize the weighted average of the weights of each sub-network, q i The weights of the i-th sub-network are used to characterize the weights, l is used to characterize the loss function, θ is used to characterize the parameters of the currently trained attribute prediction model, and X is used to characterize the weights of the i-th sub-network. i The loss function value is used to characterize the i-th subnetwork, and n is used to characterize the number of subnetworks.

12. Attribute prediction methods, including: Based on network behavior data occurring in business scenarios, nodes to be predicted for attributes are obtained; wherein, the business scenarios include: determining whether an account is a risky account in the field of risk prevention and control; wherein, the network behavior data includes: transaction connections between the account and other accounts, device information associated with the account, and bank card information associated with the account; The information of the node to be attribute predicted is input into the attribute prediction model to obtain the attribute prediction result of the node to be attribute predicted output by the attribute prediction model; wherein, the attribute prediction model is trained using the method described in any one of claims 1 to 11.

13. A training device for an attribute prediction model, comprising: The module includes a graph network construction module, a sub-network splitting module, a loss calculation module, a weight calculation module, and a model training module. The graph network construction module is configured to construct a graph network based on network behavior data recorded in business scenarios. The business scenarios include: determining whether an account is a risky account in the risk control field. The network behavior data includes: transaction connections between accounts and other accounts, device information associated with the account, and bank card information associated with the account. The graph network is constructed by establishing connections between the accounts, using each account included in the network behavior data as nodes. The sub-network splitting module is configured to obtain at least two sub-networks from the graph network obtained by the graph network construction module; wherein each sub-network includes a first node and at least one second node that has an edge relationship with the first node, and the nodes contained in any two sub-networks are not completely the same; the node is an entity obtained based on the network behavior data; The loss calculation module is configured to calculate the loss function value of the first node in the current sub-network relative to the label value for each sub-network obtained by the sub-network splitting module. The weight calculation module is configured to calculate the weight values ​​of each sub-network whose loss function satisfies the distribution drift constraint based on the loss function values ​​obtained by the loss calculation module; wherein, the distribution drift constraint is used to characterize the constraint on the inconsistent patterns of the model training data and prediction data over time. The model training module is configured to train the attribute prediction model using the weight values ​​of the loss function obtained by the weight calculation module.

14. Attribute prediction module, including: Module for acquiring nodes to be predicted, and module for outputting attribute prediction results; The node acquisition module is configured to obtain nodes for attribute prediction based on network behavior data occurring in a business scenario; wherein, the business scenario includes: determining whether an account is a risk account in the field of risk prevention and control; wherein, the network behavior data includes: transaction connections between the account and other accounts, device information associated with the account, and bank card information associated with the account; The attribute prediction result output module is configured to input the information of the node to be predicted, obtained by the node acquisition module, into the attribute prediction model to obtain the attribute prediction result of the node to be predicted output by the attribute prediction model; wherein, the attribute prediction model is trained using the training device of the attribute prediction model as described in claim 13.

15. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-12.

Images