Incremental prediction method and device for multiple service targets
By introducing expert modules, gated units and target towers into the incremental prediction model, combining feature separation and attention mechanisms, the problems of high complexity and poor accuracy of incremental prediction of multiple business targets in the prior art are solved, and incremental prediction with low complexity and high accuracy are achieved.
Patent Information
- Application Number
- CN202411825561.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-11
- Publication Date
- 2025-02-25
AI Technical Summary
The prior art has high complexity and poor prediction accuracy in incremental prediction of multiple business objectives.
The incremental prediction model is adopted, including expert modules, gated units and target towers, and the prediction of incremental values is performed through feature separation and attention mechanisms. The model is able to separate features related to each business objective and calculate incremental values through intervention features.
The incremental prediction complexity of multiple business goals is reduced, prediction accuracy is improved, and the relationship between multiple goals can be effectively modeled.
Smart Images
Figure CN119515467A_ABST
Abstract
Description
Technical Field
[0001] One or more embodiments of the present specification relate to the computer field, and in particular, to an incremental prediction method and device for multiple business objectives. Background Art
[0002] At present, in some business scenarios, we should not only pay attention to multiple business goals, but also focus on the increments of each business goal after intervening on users. Accurately predicting the increments will help to select users for intervention in a targeted manner, thereby improving the overall business effect. The above interventions can include issuing red envelopes, placing advertisements, etc.
[0003] Typically, in marketing advertising business scenarios for advertisers, unlike the advertising business of traditional advertising platforms, marketing advertising is advertising from the advertiser's perspective. It not only focuses on response indicators such as click-through rate, visit rate, and spending rate that reflect user interest and brand influence, but also focuses on the incremental behavior brought by advertising to each user. Therefore, it is necessary to make incremental predictions for multiple business goals.
[0004] In the prior art, the above incremental prediction is highly complex and has poor prediction accuracy. Summary of the invention
[0005] One or more embodiments of the present specification describe a method and device for incremental prediction of multiple business objectives, which has low complexity and good prediction accuracy.
[0006] In a first aspect, a method for incremental prediction of multiple business objectives is provided, which is implemented using an incremental prediction model, wherein the incremental prediction model includes an expert module, a first gating unit, a second gating unit, a first target tower, and a second target tower, and the method includes:
[0007] Obtain user characteristics of target users and current intervention characteristics;
[0008] Inputting the user features into the expert module, and separating, through the expert module, a first feature related to a first business goal, a second feature related to a second business goal, and a shared feature related to both the first business goal and the second business goal;
[0009] Input the first feature and the shared feature into a first gating unit to obtain a first target feature; input the first target feature and the intervention feature into a first target tower to obtain a first incremental value of intervention relative to no intervention under a first business goal;
[0010] The second feature and the shared feature are input into a second gating unit to obtain a second target feature; the second target feature and the intervention feature are input into a second target tower to obtain a second incremental value with intervention relative to no intervention under a second business goal.
[0011] In a possible implementation, the method further includes: performing the following first training on the expert module:
[0012] Obtaining user characteristics of the sample user and a label value of the first business goal; the label value represents whether the sample user has performed a target behavior of the first business goal;
[0013] Inputting the user characteristics of the sample users into the expert module, and separating the first sample characteristics related to the first business goal through the expert module;
[0014] Based on the first sample feature of the sample user, predict a first estimated value of the first business goal;
[0015] The model parameters of the expert module are adjusted according to the first estimated value and the label value.
[0016] In a possible implementation, the method further includes: performing the following second training on the expert module:
[0017] A first sample set and a second sample set are obtained, wherein each sample user in the first sample set performs a target behavior of a first business goal, and each sample user in the second sample set does not perform the target behavior of the first business goal;
[0018] Inputting user features of any sample user into the expert module, and separating second sample features related to the second business goal through the expert module;
[0019] Based on the second sample features of each sample user, calculate its first distribution in the first sample set and its second distribution in the second sample set;
[0020] The model parameters of the expert module are adjusted with the training goal of reducing the difference between the first distribution and the second distribution.
[0021] In a possible implementation, the first target tower includes: a first self-attention network and a first mutual attention network;
[0022] The first self-attention network is used to calculate the first target feature based on the self-attention mechanism to obtain a first business prediction value of the first business target without intervention;
[0023] The first mutual attention network is used to perform calculations on the first target feature and the intervention feature based on the mutual attention mechanism to obtain a first intervention prediction value of whether intervention exists, and the first incremental value.
[0024] Furthermore, the target user is a non-intervention user and has a first label value of a first service target under non-intervention; the method further includes:
[0025] Determining a first predicted loss according to the first service prediction value and the first label value;
[0026] With the goal of reducing the first prediction loss, the model parameters of the first self-attention network, the first gating unit and / or the expert module are adjusted.
[0027] Furthermore, the target user is an intervened user and has a second label value of the first service target under intervention; the method further includes:
[0028] Adding the first service prediction value to the first incremental value to obtain a first estimated value of the first service target with intervention of the target user;
[0029] Determining a first incremental prediction loss based on the first estimated value and the second label value;
[0030] With the goal of reducing the first incremental prediction loss, adjust model parameters of the first mutual attention network, the first self-attention network, the first gating unit and / or the expert module.
[0031] Furthermore, the target user has an intervention tag value indicating whether there is intervention; and the method further includes:
[0032] determining a first intervention prediction loss according to the first intervention prediction value and the intervention label value;
[0033] With the goal of reducing the first intervention prediction loss, the model parameters of the first mutual attention network, the first gating unit and / or the expert module are adjusted.
[0034] In a possible implementation, the second target tower includes: a second self-attention network and a second mutual attention network;
[0035] The second self-attention network is used to calculate the second target feature based on the self-attention mechanism to obtain a second business prediction value of the second business target without intervention;
[0036] The second mutual attention network is used to perform calculations on the second target feature and the intervention feature based on the mutual attention mechanism to obtain a second intervention prediction value of whether intervention occurs and the second incremental value.
[0037] Furthermore, the target user is a non-intervention user and has a third label value of the second service target without intervention; the method further includes:
[0038] Determining a second predicted loss according to the second service prediction value and the third label value;
[0039] With the goal of reducing the second prediction loss, the model parameters of the second self-attention network, the second gating unit and / or the expert module are adjusted.
[0040] Furthermore, the target user is an intervened user and has a fourth tag value of a second service target under intervention; the method further includes:
[0041] Adding the second service prediction value to the second incremental value to obtain a second estimated value of the second service target with intervention of the target user;
[0042] Determining a second incremental prediction loss based on the second estimated value and the fourth label value;
[0043] With the goal of reducing the second incremental prediction loss, adjust the model parameters of the second mutual attention network, the second self-attention network, the second gating unit and / or the expert module.
[0044] Furthermore, the target user has an intervention tag value indicating whether there is intervention; and the method further includes:
[0045] determining a second intervention prediction loss according to the second intervention prediction value and the intervention label value;
[0046] With the goal of reducing the second intervention prediction loss, the model parameters of the second mutual attention network, the second gating unit and / or the expert module are adjusted.
[0047] In a second aspect, an incremental prediction device for multiple business objectives is provided, which is implemented using an incremental prediction model, wherein the incremental prediction model includes an expert module, a first gating unit, a second gating unit, a first target tower, and a second target tower, and the device includes:
[0048] An acquisition unit, used for acquiring user characteristics of a target user and current intervention characteristics;
[0049] a feature separation unit, configured to input the user features acquired by the acquisition unit into the expert module, and separate a first feature related to the first business goal, a second feature related to the second business goal, and a shared feature related to both the first business goal and the second business goal through the expert module;
[0050] A first prediction unit is configured to input the first feature and the shared feature obtained by the feature separation unit into a first gating unit to obtain a first target feature; input the first target feature and the intervention feature obtained by the acquisition unit into a first target tower to obtain a first incremental value of the intervention relative to the non-intervention under the first business goal;
[0051] The second prediction unit is used to input the second feature and the shared feature obtained by the feature separation unit into the second gating unit to obtain a second target feature; and input the second target feature and the intervention feature obtained by the acquisition unit into the second target tower to obtain a second incremental value with intervention relative to no intervention under the second business goal.
[0052] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.
[0053] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.
[0054] Through the method and device provided in the embodiment of this specification, firstly, the user characteristics and the current intervention characteristics of the target user are obtained; then the user characteristics are input into the expert module, and the first characteristics related to the first business goal, the second characteristics related to the second business goal, and the shared characteristics related to both the first business goal and the second business goal are separated through the expert module; then the first characteristics and the shared characteristics are input into the first gating unit to obtain the first target characteristics; the first target characteristics and the intervention characteristics are input into the first target tower to obtain the first incremental value of the intervention relative to no intervention under the first business goal; finally, the second characteristics and the shared characteristics are input into the second gating unit to obtain the second target characteristics; the second target characteristics and the intervention characteristics are input into the second target tower to obtain the second incremental value of the intervention relative to no intervention under the second business goal. As can be seen from the above, in the embodiment of this specification, the incremental prediction for the first business goal and the incremental prediction for the second business goal are both based on the same expert module, that is, a multi-target incremental modeling scheme is adopted, which can model the relationship between multiple goals compared with independent modeling for each goal, with low complexity and good prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0056] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;
[0057] Figure 2 A flow chart of a method for incremental prediction of multiple business objectives according to an embodiment is shown;
[0058] Figure 3 A first training schematic diagram according to an embodiment is shown;
[0059] Figure 4 A second training schematic diagram according to an embodiment is shown;
[0060] Figure 5 A schematic diagram showing the structure of a first target tower according to one embodiment is shown;
[0061] Figure 6 A schematic diagram showing the structure of a second target tower according to one embodiment is shown;
[0062] Figure 7 A schematic block diagram of an incremental prediction device for multiple business objectives according to an embodiment is shown. DETAILED DESCRIPTION
[0063] The solution provided in this specification is described below in conjunction with the accompanying drawings.
[0064] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).
[0065] Figure 1 This is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. The implementation scenario involves incremental prediction of multiple business objectives, which is implemented using an incremental prediction model. The input of the incremental prediction model is the user characteristics of the target user and the current intervention characteristics. The output of the incremental prediction model is a first incremental value of intervention relative to no intervention under the first business objective, and a second incremental value of intervention relative to no intervention under the second business objective. Figure 1The incremental prediction model includes an expert module, a first gating unit G1, a second gating unit G2, a first target tower and a second target tower, wherein the expert module is used to separate a first feature related to a first business goal, a second feature related to a second business goal, and a shared feature related to both the first business goal and the second business goal from user features; the first gating unit G1 is used to obtain a first target feature based on the first feature and the shared feature; the first target tower is used to obtain the above-mentioned first incremental value based on the first target feature and the intervention feature; the second gating unit G2 is used to obtain the second target feature based on the second feature and the shared feature; the second target tower is used to obtain the above-mentioned second incremental value based on the second target feature and the intervention feature.
[0066] The embodiments of this specification are based on multi-objective modeling to perform incremental prediction of multiple business objectives, with low complexity and good prediction accuracy.
[0067] Multi-objective modeling: is a technique widely used in machine learning, especially deep learning and recommendation systems, which aims to optimize multiple business goals or indicators at the same time. These goals may be interrelated or conflicting, such as click-through rate (CTR), conversion rate (CVR), user stay time, gross merchandise volume (GMV), etc. The goal of multi-objective modeling is to optimize these goals simultaneously by building a unified model to improve the overall business effect.
[0068] Expert module: refers to a module used to represent and process knowledge related to a specific domain or task. In multi-objective modeling, an expert module usually contains multiple expert networks, which are combined in a specific way to cope with different tasks or scenarios. Expert modules can represent and store knowledge related to specific tasks or domains, which can be rules, model parameters, feature representations, etc. At the same time, in multi-objective modeling, expert modules can learn the correlation between different modules and realize knowledge transfer and sharing through shared or independent expert networks.
[0069] The embodiments of this specification propose multi-objective incremental modeling based on multi-objective modeling, and optimize feature separation, data correction, etc., in order to further improve the prediction accuracy.
[0070] Figure 2 A flowchart of an incremental prediction method for multiple business objectives according to an embodiment is shown, which is implemented using an incremental prediction model, wherein the incremental prediction model includes an expert module, a first gating unit, a second gating unit, a first target tower, and a second target tower. The method can be based on Figure 1The implementation scenario shown in Figure 1 is as follows. Figure 2 As shown, the incremental prediction method for multiple business goals in this embodiment includes the following steps: Step 21, obtaining the user characteristics of the target user and the current intervention characteristics; Step 22, inputting the user characteristics into the expert module, and separating the first characteristics related to the first business goal, the second characteristics related to the second business goal, and the shared characteristics related to both the first business goal and the second business goal through the expert module; Step 23, inputting the first characteristics and the shared characteristics into the first gating unit to obtain the first target characteristics; inputting the first target characteristics and the intervention characteristics into the first target tower to obtain the first incremental value of the intervention relative to the non-intervention under the first business goal; Step 24, inputting the second characteristics and the shared characteristics into the second gating unit to obtain the second target characteristics; inputting the second target characteristics and the intervention characteristics into the second target tower to obtain the second incremental value of the intervention relative to the non-intervention under the second business goal. The specific execution method of each of the above steps is described below.
[0071] First, in step 21, the user characteristics of the target user and the current intervention characteristics are obtained. It is understandable that the intervention means can be various, for example, issuing red envelopes or placing advertisements. When the intervention means is issuing red envelopes, the intervention characteristics can be the amount of the red envelopes, etc., and when the intervention means is placing advertisements, the intervention characteristics can be the advertisement time or advertisement display form, etc.
[0072] The user characteristics may include the user's attribute characteristics, behavior characteristics or context characteristics. The attribute characteristics may include, but are not limited to, language, etc., the behavior characteristics may include, but are not limited to, user historical access information, user historical shopping information, etc., and the context characteristics may include, but are not limited to, the device used by the user, the current time, etc.
[0073] In a marketing scenario, the above-mentioned intervention characteristics may specifically be exposure characteristics.
[0074] In the embodiments of the present specification, the above-mentioned user features and intervention features can be respectively subjected to feature embedding to obtain corresponding coding vectors. The feature embedding method can be flexibly selected according to the feature type. For example, for numerical type features, the corresponding coding vectors can be obtained through normalization processing, for multi-classification features, the corresponding coding vectors can be obtained through one-hot encoding processing, and for some numerical type features with large spans, the corresponding coding vectors can be obtained through bucket discretization processing.
[0075] Then, in step 22, the user features are input into an expert module, and the expert module is used to separate a first feature related to the first business goal, a second feature related to the second business goal, and a shared feature related to both the first business goal and the second business goal. It can be understood that the expert module can specifically include three expert networks, and each expert network outputs the first feature, the second feature, and the shared feature respectively.
[0076] It should be noted that the number of expert networks is related to the number of business objectives. When the multiple business objectives are specifically two business objectives, the expert module can specifically include three expert networks. When the number of multiple business objectives is more, the expert module needs to include more expert networks. The expert network can specifically adopt a multi-layer perceptron (MLP).
[0077] In one example, the method further includes: performing the following first training on the expert module:
[0078] Obtaining user characteristics of the sample user and a label value of the first business goal; the label value represents whether the sample user has performed a target behavior of the first business goal;
[0079] Inputting the user characteristics of the sample users into the expert module, and separating the first sample characteristics related to the first business goal through the expert module;
[0080] Based on the first sample feature of the sample user, predict a first estimated value of the first business goal;
[0081] The model parameters of the expert module are adjusted according to the first estimated value and the label value.
[0082] In this example, a first estimated value of the first business target is predicted using a first sample feature that is only related to the first business target, and the first sample feature is constrained to be only related to the first business target, thereby ensuring the accuracy of feature separation.
[0083] Figure 3 FIG. 1 shows a first training diagram according to an embodiment. Figure 3 The first training is implemented by a class correction module, and the class correction module includes a fully connected layer. Based on the first sample feature of the sample user, the first estimated value of the first business target is predicted by the fully connected layer. Assuming that the first sample feature is Xc, the label value of the first business target is Yc, and the loss function is a binary cross entropy loss, the model parameters of the expert module can be adjusted based on this.
[0084] In one example, the method further includes: performing the following second training on the expert module:
[0085] A first sample set and a second sample set are obtained, wherein each sample user in the first sample set performs a target behavior of a first business goal, and each sample user in the second sample set does not perform the target behavior of the first business goal;
[0086] Inputting user features of any sample user into the expert module, and separating second sample features related to the second business goal through the expert module;
[0087] Based on the second sample features of each sample user, calculate its first distribution in the first sample set and its second distribution in the second sample set;
[0088] The model parameters of the expert module are adjusted with the training goal of reducing the difference between the first distribution and the second distribution.
[0089] In this example, the distribution of the second sample features that are only related to the second business goal under different target behaviors of the first business goal is calculated to ensure that the distribution of the second sample features is as similar as possible regardless of whether there is a target behavior of the first business goal, thereby constraining the second sample features to be only related to the second business goal, thereby ensuring the accuracy of feature separation.
[0090] Specifically, each batch of training data can be divided into data with a target behavior of 1 for the first business goal (i.e., representing a target behavior) and data with a target behavior of 0 for the first business goal (i.e., representing no target behavior). The distribution of the second sample feature Xv only related to the second business goal on these two groups of data is calculated, and the maximum mean difference (MMD) distance of the two distributions is constrained to be as small as possible, that is, the distributions are guaranteed to be as similar as possible.
[0091] As you can understand, MMD is a metric used to measure the distance between two probability distributions. It is widely used in many occasions, especially when you need to compare two different but related distributions.
[0092] Figure 4 FIG. 2 shows a second training diagram according to an embodiment. Figure 4 , the second training is realized through the balancing module. It is assumed that the second sample feature is Xv, and the label value of the first business goal is Yc. The label value can indicate whether the sample user has performed the target behavior of the first business goal. Therefore, according to the label value, each sample user can be divided into the first sample set and the second sample set. The distribution of the second sample feature Xv on the two sample sets is calculated, and the MMD distance of the two distributions is constrained to be as small as possible. Based on this, the model parameters of the expert module can be adjusted.
[0093] In the embodiments of this specification, the balancing module and the class correction module can be used to perform short-distance strong constraints immediately after feature separation to ensure that the features separated by the expert module are only related to a certain target or are shared features of various targets.
[0094] Then, in step 23, the first feature and the shared feature are input into the first gating unit to obtain the first target feature; the first target feature and the intervention feature are input into the first target tower to obtain the first incremental value of intervention relative to no intervention under the first business goal. It can be understood that the first feature and the shared feature are outputs of the expert module, and the prediction accuracy of the first incremental value is not only related to the first target tower and the first gating unit, but also depends on the accuracy of feature separation of the expert module.
[0095] In one example, the first target tower includes: a first self-attention network and a first mutual attention network;
[0096] The first self-attention network is used to calculate the first target feature based on the self-attention mechanism to obtain a first business prediction value of the first business target without intervention;
[0097] The first mutual attention network is used to perform calculations on the first target feature and the intervention feature based on the mutual attention mechanism to obtain a first intervention prediction value of whether intervention exists, and the first incremental value.
[0098] In this example, the self-attention mechanism and the mutual attention mechanism are combined to divide the first target feature into a feature insensitive to intervention and a feature sensitive to intervention. The feature insensitive to intervention is used to obtain the first business prediction value, and the feature sensitive to intervention is used to obtain the first intervention prediction value and the first incremental value. Predicting the first business prediction value and the first intervention prediction value helps to improve the prediction accuracy of the first incremental value.
[0099] Figure 5 FIG. 1 is a schematic diagram showing the structure of a first target tower according to an embodiment. Figure 5 The first target tower includes: a first self-attention network, a first mutual attention network, and three MLPs, whose input is the first target feature and the intervention feature, and whose output is the first business prediction value First intervention prediction value First increment value It is understandable that the first business forecast value and the first increment value Add them together to get the business forecast value of the first business target under intervention
[0100] Furthermore, the target user is a non-intervention user and has a first label value of a first service target under non-intervention; the method further includes:
[0101] Determining a first predicted loss according to the first service prediction value and the first label value;
[0102] With the goal of reducing the first prediction loss, the model parameters of the first self-attention network, the first gating unit and / or the expert module are adjusted.
[0103] In this example, the first target feature is processed by the self-attention mechanism, feature cross-pollination is performed, and regression is performed to obtain the first business prediction value under natural conditions. Only data without intervention is used to update the model parameters of the first self-attention network, the first gating unit and / or the expert module to complete precise correction.
[0104] Furthermore, the target user is an intervened user and has a second label value of the first service target under intervention; the method further includes:
[0105] Adding the first service prediction value to the first incremental value to obtain a first estimated value of the first service target with intervention of the target user;
[0106] Determining a first incremental prediction loss based on the first estimated value and the second label value;
[0107] With the goal of reducing the first incremental prediction loss, adjust model parameters of the first mutual attention network, the first self-attention network, the first gating unit and / or the expert module.
[0108] In this example, the intervention feature and the first target feature are processed by the mutual attention mechanism to capture the features of the first target feature that are sensitive to the intervention. Incremental modeling is different from response modeling. The labels of the observed data cannot be directly used for supervision. Therefore, the first incremental value obtained by regression is First business forecast value with natural conditions Add and transform into business forecast value under intervention The labels of the observed data are converted into response indicators to indirectly constrain the estimation of the incremental scores and achieve precise correction.
[0109] Furthermore, the target user has an intervention tag value indicating whether there is intervention; and the method further includes:
[0110] determining a first intervention prediction loss according to the first intervention prediction value and the intervention label value;
[0111] With the goal of reducing the first intervention prediction loss, the model parameters of the first mutual attention network, the first gating unit and / or the expert module are adjusted.
[0112] In this example, the intervention feature and the first target feature are processed by the mutual attention mechanism to capture the features of the first target feature that are sensitive to the intervention, and the correction is completed by predicting whether there is intervention.
[0113] Finally, in step 24, the second feature and the shared feature are input into the second gating unit to obtain the second target feature; the second target feature and the intervention feature are input into the second target tower to obtain the second incremental value with intervention relative to without intervention under the second business goal. It can be understood that the second feature and the shared feature are the outputs of the expert module, and the prediction accuracy of the second incremental value is not only related to the second target tower and the second gating unit, but also depends on the accuracy of feature separation of the expert module.
[0114] In one example, the second target tower includes: a second self-attention network and a second mutual attention network;
[0115] The second self-attention network is used to calculate the second target feature based on the self-attention mechanism to obtain a second business prediction value of the second business target without intervention;
[0116] The second mutual attention network is used to perform calculations on the second target feature and the intervention feature based on the mutual attention mechanism to obtain a second intervention prediction value of whether intervention occurs and the second incremental value.
[0117] In this example, the self-attention mechanism and the mutual attention mechanism are combined to divide the second target features into features insensitive to intervention and features sensitive to intervention. The features insensitive to intervention are used to obtain the second business prediction value, and the features sensitive to intervention are used to obtain the second intervention prediction value and the second incremental value. Predicting the second business prediction value and the second intervention prediction value helps to improve the prediction accuracy of the second incremental value.
[0118] Figure 6 FIG. 1 is a schematic diagram showing the structure of a second target tower according to an embodiment. Figure 6 The second target tower includes: a second self-attention network, a second mutual attention network, and three MLPs, whose input is the second target feature and the intervention feature, and whose output is the second business prediction value Second intervention prediction value Second increment value It is understandable that the second business forecast value and the second increment value Add them together to get the business forecast value of the second business target under intervention
[0119] Furthermore, the target user is a non-intervention user and has a third label value of the second service target without intervention; the method further includes:
[0120] Determining a second predicted loss according to the second service prediction value and the third label value;
[0121] With the goal of reducing the second prediction loss, adjust the model parameters of the second self-attention network, the second gating unit and / or the expert module.
[0122] In this example, the second target feature is processed by the self-attention mechanism, feature cross-pollination is performed, and regression is performed to obtain the second business prediction value under natural conditions. Only data without intervention is used to update the model parameters of the second self-attention network, the second gating unit and / or the expert module to complete precise correction.
[0123] Furthermore, the target user is an intervened user and has a fourth tag value of a second service target under intervention; the method further includes:
[0124] Adding the second service prediction value to the second incremental value to obtain a second estimated value of the second service target with intervention of the target user;
[0125] Determining a second incremental prediction loss based on the second estimated value and the fourth label value;
[0126] With the goal of reducing the second incremental prediction loss, adjust the model parameters of the second mutual attention network, the second self-attention network, the second gating unit and / or the expert module.
[0127] In this example, the intervention feature and the second target feature are processed by the mutual attention mechanism to capture the features of the second target feature that are sensitive to the intervention. Incremental modeling is different from response modeling. The labels of the observed data cannot be directly used for supervision. Therefore, the second incremental value obtained by regression is Second business forecast value under natural conditions Add and transform into business forecast value under intervention The labels of the observed data are converted into response indicators to indirectly constrain the estimation of the incremental scores and achieve precise correction.
[0128] Furthermore, the target user has an intervention tag value indicating whether there is intervention; and the method further includes:
[0129] determining a second intervention prediction loss according to the second intervention prediction value and the intervention label value;
[0130] With the goal of reducing the second intervention prediction loss, the model parameters of the second mutual attention network, the second gating unit and / or the expert module are adjusted.
[0131] In this example, the intervention feature and the second target feature are processed by the mutual attention mechanism to capture the features of the second target feature that are sensitive to the intervention, and the correction is completed by predicting whether there is intervention.
[0132] In the embodiment of this specification, the output of each target tower is subjected to long-distance constraints according to the label value of the result to be predicted, so as to further ensure the accuracy of feature separation. Among them, the expert module has undergone dual constraints, including the aforementioned short-distance constraints and long-distance constraints. After the short-distance constraints, the expert module separates the input features according to multiple business goals, retains the target-related features and shared features, removes the interference features brought by other goals, reduces interference for multi-target correction at the feature level, and reduces the range of separated features to make subsequent corrections more accurate. Long-distance constraints, based on the target tower, find the precise range of the corresponding separation feature X according to each target Y, and complete the correction of the separation feature X to the intervention feature T in the precise range; the expert module with long and short dual constraints and the target tower based on attention are progressive, and the multi-tower architecture is combined to achieve accurate correction of multiple targets.
[0133] Through the method provided in the embodiment of this specification, firstly, the user characteristics of the target user and the current intervention characteristics are obtained; then the user characteristics are input into the expert module, and the first characteristics related to the first business goal, the second characteristics related to the second business goal, and the shared characteristics related to both the first business goal and the second business goal are separated through the expert module; then the first characteristics and the shared characteristics are input into the first gating unit to obtain the first target characteristics; the first target characteristics and the intervention characteristics are input into the first target tower to obtain the first incremental value of intervention relative to no intervention under the first business goal; finally, the second characteristics and the shared characteristics are input into the second gating unit to obtain the second target characteristics; the second target characteristics and the intervention characteristics are input into the second target tower to obtain the second incremental value of intervention relative to no intervention under the second business goal. As can be seen from the above, in the embodiment of this specification, the incremental prediction for the first business goal and the incremental prediction for the second business goal are both based on the same expert module, that is, a multi-target incremental modeling scheme is adopted, which can model the relationship between multiple goals compared with independent modeling for each goal, with low complexity and good prediction accuracy.
[0134] According to an embodiment of another aspect, an incremental prediction device for multiple business objectives is also provided, which is implemented using an incremental prediction model. The incremental prediction model includes an expert module, a first gating unit, a second gating unit, a first target tower and a second target tower. The device is used to execute the method provided in the embodiment of this specification. Figure 7 A schematic block diagram of an incremental prediction device for multiple business objectives according to an embodiment is shown. Figure 7As shown, the device 700 includes:
[0135] An acquisition unit 71, used to acquire user characteristics of a target user and current intervention characteristics;
[0136] A feature separation unit 72 is used to input the user features acquired by the acquisition unit 71 into the expert module, and separate the first features related to the first business goal, the second features related to the second business goal, and the shared features related to both the first business goal and the second business goal through the expert module;
[0137] The first prediction unit 73 is used to input the first feature and the shared feature obtained by the feature separation unit 72 into the first gating unit to obtain a first target feature; input the first target feature and the intervention feature obtained by the acquisition unit 71 into the first target tower to obtain a first incremental value of the intervention relative to the non-intervention under the first business goal;
[0138] The second prediction unit 74 is used to input the second feature and the shared feature obtained by the feature separation unit 72 into the second gating unit to obtain a second target feature; input the second target feature and the intervention feature obtained by the acquisition unit 71 into the second target tower to obtain a second incremental value with intervention relative to no intervention under the second business goal.
[0139] Optionally, as an embodiment, the device further includes: a first training unit, configured to perform the following first training on the expert module:
[0140] Obtaining user characteristics of the sample user and a label value of the first business goal; the label value represents whether the sample user has performed a target behavior of the first business goal;
[0141] Inputting the user characteristics of the sample users into the expert module, and separating the first sample characteristics related to the first business goal through the expert module;
[0142] Based on the first sample feature of the sample user, predict a first estimated value of the first business goal;
[0143] The model parameters of the expert module are adjusted according to the first estimated value and the label value.
[0144] Optionally, as an embodiment, the device further includes: a second training unit, configured to perform the following second training on the expert module:
[0145] A first sample set and a second sample set are obtained, wherein each sample user in the first sample set performs a target behavior of a first business goal, and each sample user in the second sample set does not perform the target behavior of the first business goal;
[0146] Inputting user features of any sample user into the expert module, and separating second sample features related to the second business goal through the expert module;
[0147] Based on the second sample features of each sample user, calculate its first distribution in the first sample set and its second distribution in the second sample set;
[0148] The model parameters of the expert module are adjusted with the training goal of reducing the difference between the first distribution and the second distribution.
[0149] Optionally, as an embodiment, the first target tower includes: a first self-attention network and a first mutual attention network;
[0150] The first self-attention network is used to calculate the first target feature based on the self-attention mechanism to obtain a first business prediction value of the first business target without intervention;
[0151] The first mutual attention network is used to perform calculations on the first target feature and the intervention feature based on the mutual attention mechanism to obtain a first intervention prediction value of whether intervention exists, and the first incremental value.
[0152] Furthermore, the target user is a non-intervention user and has a first label value of a first service target under non-intervention; the device further includes:
[0153] A first loss determining unit, configured to determine a first predicted loss according to the first service prediction value and the first label value;
[0154] The first parameter adjustment unit is used to adjust the model parameters of the first self-attention network, the first gating unit and / or the expert module with the goal of reducing the first prediction loss obtained by the first loss determination unit.
[0155] Furthermore, the target user is an intervention user and has a second label value of the first service target under intervention; the device further includes:
[0156] A first estimation unit, configured to add the first service prediction value to the first incremental value to obtain a first estimation value of a first service target with intervention of a target user;
[0157] A first incremental loss determining unit, configured to determine a first incremental prediction loss according to a first estimated value and a second label value obtained by the first estimating unit;
[0158] The first incremental parameter adjustment unit is used to adjust the model parameters of the first mutual attention network, the first self-attention network, the first gating unit and / or the expert module with the goal of reducing the first incremental prediction loss obtained by the first incremental loss determination unit.
[0159] Furthermore, the target user has an intervention tag value indicating whether there is intervention; and the device further includes:
[0160] a first intervention loss determining unit, configured to determine a first intervention prediction loss according to the first intervention prediction value and the intervention label value;
[0161] The first intervention parameter adjustment unit is used to adjust the model parameters of the first mutual attention network, the first gating unit and / or the expert module with the goal of reducing the first intervention prediction loss obtained by the first intervention loss determination unit.
[0162] Optionally, as an embodiment, the second target tower includes: a second self-attention network and a second mutual attention network;
[0163] The second self-attention network is used to calculate the second target feature based on the self-attention mechanism to obtain a second business prediction value of the second business target without intervention;
[0164] The second mutual attention network is used to perform calculations on the second target feature and the intervention feature based on the mutual attention mechanism to obtain a second intervention prediction value of whether intervention occurs and the second incremental value.
[0165] Furthermore, the target user is a non-intervention user and has a third label value of the second service target under non-intervention; the device further includes:
[0166] a second loss determining unit, configured to determine a second predicted loss according to the second service prediction value and a third tag value;
[0167] The second parameter adjustment unit is used to adjust the model parameters of the second self-attention network, the second gating unit and / or the expert module with the goal of reducing the second prediction loss obtained by the second loss determination unit.
[0168] Furthermore, the target user is an intervened user and has a fourth label value of a second service target under intervention; the device further includes:
[0169] A second estimation unit, configured to add the second service prediction value to the second incremental value to obtain a second estimation value of a second service target with intervention of the target user;
[0170] A second incremental loss determining unit, configured to determine a second incremental prediction loss according to the second estimated value obtained by the second estimating unit and the fourth label value;
[0171] The second incremental parameter adjustment unit is used to adjust the model parameters of the second mutual attention network, the second self-attention network, the second gating unit and / or the expert module with the goal of reducing the second incremental prediction loss obtained by the second incremental loss determination unit.
[0172] Furthermore, the target user has an intervention tag value indicating whether there is intervention; and the device further includes:
[0173] a second intervention loss determining unit, configured to determine a second intervention prediction loss according to the second intervention prediction value and the intervention label value;
[0174] The second intervention parameter adjustment unit is used to adjust the model parameters of the second mutual attention network, the second gating unit and / or the expert module with the goal of reducing the second intervention prediction loss obtained by the second intervention loss determination unit.
[0175] Through the device provided in the embodiment of this specification, first, the acquisition unit 71 acquires the user features and the current intervention features of the target user; then the feature separation unit 72 inputs the user features into the expert module, and separates the first feature related to the first business goal, the second feature related to the second business goal, and the shared feature related to both the first business goal and the second business goal through the expert module; then the first prediction unit 73 inputs the first feature and the shared feature into the first gating unit to obtain the first target feature; the first target feature and the intervention feature are input into the first target tower to obtain the first incremental value of intervention relative to no intervention under the first business goal; finally, the second prediction unit 74 inputs the second feature and the shared feature into the second gating unit to obtain the second target feature; the second target feature and the intervention feature are input into the second target tower to obtain the second incremental value of intervention relative to no intervention under the second business goal. As can be seen from the above, in the embodiment of this specification, the incremental prediction for the first business goal and the incremental prediction for the second business goal are both based on the same expert module, that is, a multi-target incremental modeling scheme is adopted, which can model the relationship between multiple goals compared to independent modeling for each goal, with low complexity and good prediction accuracy.
[0176] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described.
[0177] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 2 The method described.
[0178] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.
[0179] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for incremental prediction of multiple business objectives, implemented by using an incremental prediction model, wherein the incremental prediction model includes an expert module, a first gating unit, a second gating unit, a first target tower and a second target tower, and the method includes: Obtain user characteristics of target users and current intervention characteristics; Inputting the user features into the expert module, and separating, through the expert module, a first feature related to a first business goal, a second feature related to a second business goal, and a shared feature related to both the first business goal and the second business goal; Input the first feature and the shared feature into a first gating unit to obtain a first target feature; input the first target feature and the intervention feature into a first target tower to obtain a first incremental value of intervention relative to no intervention under a first business goal; Inputting the second feature and the shared feature into a second gating unit to obtain a second target feature; The second target feature and the intervention feature are input into a second target tower to obtain a second incremental value with intervention relative to without intervention under a second business target.
2. The method of claim 1, wherein: The method further comprises: performing the following first training on the expert module: Obtaining user characteristics of the sample user and a label value of the first business goal; the label value represents whether the sample user has performed a target behavior of the first business goal; Inputting the user characteristics of the sample users into the expert module, and separating the first sample characteristics related to the first business goal through the expert module; Based on the first sample feature of the sample user, predict a first estimated value of the first business goal; The model parameters of the expert module are adjusted according to the first estimated value and the label value.
3. The method of claim 1, wherein: The method further comprises: performing the following second training on the expert module: A first sample set and a second sample set are obtained, wherein each sample user in the first sample set performs a target behavior of a first business goal, and each sample user in the second sample set does not perform the target behavior of the first business goal; Inputting user features of any sample user into the expert module, and separating second sample features related to the second business goal through the expert module; Based on the second sample features of each sample user, calculate its first distribution in the first sample set and its second distribution in the second sample set; The model parameters of the expert module are adjusted with the training goal of reducing the difference between the first distribution and the second distribution.
4. The method of claim 1, wherein: The first target tower includes: a first self-attention network and a first mutual attention network; The first self-attention network is used to calculate the first target feature based on the self-attention mechanism to obtain a first business prediction value of the first business target without intervention; The first mutual attention network is used to perform calculations on the first target feature and the intervention feature based on the mutual attention mechanism to obtain a first intervention prediction value of whether intervention exists, and the first incremental value.
5. The method of claim 4, wherein: The target user is a non-intervention user and has a first label value of a first service target under non-intervention; the method further includes: Determining a first predicted loss according to the first service prediction value and the first label value; With the goal of reducing the first prediction loss, the model parameters of the first self-attention network, the first gating unit and / or the expert module are adjusted.
6. The method of claim 4, wherein: The target user is an intervened user and has a second label value of a first service target under intervention; the method further includes: Adding the first service prediction value to the first incremental value to obtain a first estimated value of the first service target with intervention of the target user; Determining a first incremental prediction loss based on the first estimated value and the second label value; With the goal of reducing the first incremental prediction loss, adjust model parameters of the first mutual attention network, the first self-attention network, the first gating unit and / or the expert module.
7. The method according to any one of claims 4 to 6, wherein: The target user has an intervention tag value indicating whether there is intervention; and the method further includes: determining a first intervention prediction loss according to the first intervention prediction value and the intervention label value; With the goal of reducing the first intervention prediction loss, the model parameters of the first mutual attention network, the first gating unit and / or the expert module are adjusted.
8. The method of claim 1, wherein: The second target tower includes: a second self-attention network and a second mutual attention network; The second self-attention network is used to calculate the second target feature based on the self-attention mechanism to obtain a second business prediction value of the second business target without intervention; The second mutual attention network is used to perform calculations on the second target feature and the intervention feature based on the mutual attention mechanism to obtain a second intervention prediction value of whether intervention occurs and the second incremental value.
9. The method of claim 8, wherein: The target user is a non-intervention user and has a third label value of a second service target without intervention; the method further includes: Determining a second predicted loss according to the second service prediction value and the third label value; With the goal of reducing the second prediction loss, the model parameters of the second self-attention network, the second gating unit and / or the expert module are adjusted.
10. The method of claim 8, wherein: The target user is an intervened user and has a fourth tag value of a second service target under intervention; the method further includes: Adding the second service prediction value to the second incremental value to obtain a second estimated value of the second service target with intervention of the target user; Determining a second incremental prediction loss based on the second estimated value and the fourth label value; With the goal of reducing the second incremental prediction loss, adjust the model parameters of the second mutual attention network, the second self-attention network, the second gating unit and / or the expert module.
11. The method according to any one of claims 8 to 10, wherein: The target user has an intervention tag value indicating whether there is intervention; and the method further includes: determining a second intervention prediction loss according to the second intervention prediction value and the intervention label value; With the goal of reducing the second intervention prediction loss, the model parameters of the second mutual attention network, the second gating unit and / or the expert module are adjusted.
12. An incremental prediction device for multiple business objectives, implemented using an incremental prediction model, the incremental prediction model comprising an expert module, a first gating unit, a second gating unit, a first target tower and a second target tower, the device comprising: An acquisition unit, used for acquiring user characteristics of a target user and current intervention characteristics; a feature separation unit, configured to input the user features acquired by the acquisition unit into the expert module, and separate a first feature related to the first business goal, a second feature related to the second business goal, and a shared feature related to both the first business goal and the second business goal through the expert module; A first prediction unit is configured to input the first feature and the shared feature obtained by the feature separation unit into a first gating unit to obtain a first target feature; input the first target feature and the intervention feature obtained by the acquisition unit into a first target tower to obtain a first incremental value of the intervention relative to the non-intervention under the first business goal; A second prediction unit, configured to input the second feature and the shared feature obtained by the feature separation unit into a second gating unit to obtain a second target feature; The second target feature and the intervention feature acquired by the acquisition unit are input into a second target tower to obtain a second incremental value with intervention relative to without intervention under the second business target.
13. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 11.
14. A computing device, comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 11 is implemented.