Incremental model training method and computing device

Abstract

A method for training an incremental model and a computing device, comprising parameter adjustment performed by multi-round iteration: obtaining a plurality of training samples of a current round, each training sample comprising a plurality of user features of a sample user and an intervention tag and a conversion tag of the sample user; for any training sample, determining a comprehensive feature corresponding to the training sample according to the plurality of user features of the training sample; determining a prediction conversion result corresponding to the training sample according to the comprehensive characteristics; determining prediction loss according to the difference between the prediction conversion result of each training sample and the conversion label; determining distribution difference loss according to the intervention labels corresponding to the training samples and the comprehensive features; determining a weight corresponding to the distribution difference loss according to the number of rounds of the current round; the parameters of the incremental model are adjusted according to the prediction loss, the distribution difference loss and the weight, the convergence speed of the parameters of the incremental model in the model training process can be increased, and the calculation power consumption in the model training process is reduced.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of data processing technology, and in particular relate to a method for training an incremental model and a computing device. Background Technology

[0002] Uplift prediction is a key application of causal inference in machine learning. It aims to quantify the true causal effect of interventions (such as marketing and treatment) on individual behavior, rather than simply predicting outcomes. Specifically, assuming an individual's behavior is A before receiving an intervention, and B after receiving an intervention, the difference between A and B represents the true causal effect of the intervention on that individual behavior. For example, in marketing, if the goal is to predict whether a user will convert after receiving marketing (in this scenario, marketing is the intervention) (common conversion behaviors include subscription, purchase, and viewing), and the predicted probability of subscription under the control condition (no marketing) is 10%, while the predicted probability under the intervention condition (receiving marketing) is 50%, then the true causal effect of marketing on user subscription is the difference between the two—50% - 10% = 40%.

[0003] In real-world scenarios, since marketing incurs costs, the optimal strategy for relevant organizations is naturally to identify users with a higher probability of genuine causal effect and conduct targeted marketing (imagine, if a user has a high probability of subscription without marketing, there is naturally no need to market to them). Therefore, incremental models based on Counterfactual Regression (CFR), with their ability to model and predict separately for the treated and control groups, have become a popular modeling architecture in this field.

[0004] Specifically, Figure 1 A schematic diagram of the incremental model based on CFR is shown. This incremental model includes a feature embedding layer, a shared representation extraction layer, and a backbone prediction network. The feature embedding layer transforms the various original features of the input sample from their disparate data dimensions to a unified data dimension. The shared representation extraction layer extracts the transformed features to obtain comprehensive features that represent the input sample. The backbone prediction network outputs the predicted conversion probability based on the comprehensive features. The key feature of this incremental model is that the backbone prediction network includes two independent branches—an intervention branch and a control branch—which respectively output the predicted conversion probabilities under intervention conditions (i.e., intervention measures applied) and control conditions (i.e., no intervention measures applied). Therefore, the difference between the conversion probabilities under intervention and control conditions reflects the true causal effect of the intervention on the conversion behavior.

[0005] During the training of incremental models, the training sample set is often divided into an intervention group (receiving intervention) and a control group (not receiving intervention) based on whether the training samples actually received intervention. To balance the feature distributions of the intervention and control groups and eliminate selection bias (imagine that the data in the training sample set is collected in real-world application scenarios; in real-world application scenarios, high-value users with high spending and high activity are more likely to be selected as marketing targets and are more likely to convert naturally (regardless of whether they receive marketing), meaning that the training samples in the intervention group naturally have a higher conversion rate than those in the control group. However, this is not due to "whether or not intervention was received," but rather to the difference in the characteristics of the samples themselves—this is selection bias), the distribution difference loss, which measures the difference in feature distributions between the intervention and control groups, is introduced as a commonly used loss function to adjust the parameters of the incremental model.

[0006] However, by introducing distributional discrepancy loss into the training process, the incremental model also introduces an additional training objective (compared to simply predicting the conversion probability, the incremental model also needs to exclude the influence of selection bias on the conversion probability). As a result, the training difficulty of the incremental model is greatly increased, often requiring several times more training epochs than the ordinary model (which cannot predict the true causal effect) to achieve convergence, which greatly increases the computational cost of the model training process.

[0007] Therefore, there is an urgent need for a scheme to train incremental models in order to at least partially solve the above problems. Summary of the Invention

[0008] This specification provides an embodiment of a method and computing device for training an incremental model, including:

[0009] This specification provides a method for training an incremental model, the method relating to an incremental model including a shared representation extraction layer and a backbone prediction network, the method including parameter tuning through multiple iterations, wherein the parameter tuning in any one iteration specifically includes:

[0010] Obtain several training samples corresponding to the current round, wherein each training sample includes several user features of the sample user as well as intervention tags and conversion tags for the sample user;

[0011] For any training sample, the shared representation extraction layer is used to determine the comprehensive features corresponding to the training sample based on several user features of the training sample.

[0012] Using the backbone prediction network, the prediction conversion result corresponding to the training sample is determined based on the comprehensive features;

[0013] The prediction loss is determined based on the difference between the predicted conversion results and conversion labels of the training samples; the distribution difference loss is determined based on the intervention labels and comprehensive features of the training samples; and the weight corresponding to the distribution difference loss is determined based on the number of rounds in the current round.

[0014] The parameters of the incremental model are adjusted based on the prediction loss, the distribution difference loss, and the weights.

[0015] A second aspect of this specification provides a computer-readable storage medium having a computer program stored thereon that, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0016] A third aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described in the first aspect.

[0017] The proposed method for training an incremental model in this embodiment adjusts the weights of the distribution difference loss based on the iteration rounds, which can accelerate the convergence speed of the incremental model parameters during model training and reduce the computational consumption of the model training process. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure of an incremental model in one embodiment of this specification;

[0020] Figure 2 This is a schematic diagram illustrating the technical concept of a training incremental model in one embodiment of this specification;

[0021] Figure 3 This is a flowchart illustrating a method for training an incremental model in one embodiment of this specification.

[0022] Figure 4 This is a flowchart illustrating a method for determining an intervention strategy in one embodiment of this specification. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0024] First, it should be noted that this embodiment provides a technical solution for training an incremental model. This specification does not limit the specific architecture of the trained incremental model. The construction method of each model layer in this incremental model can refer to common structures of related models in the prior art (such as various common neural network architectures), and this specification does not impose any limitations on it.

[0025] Figure 2 This specification illustrates a schematic diagram of a technical concept for training an incremental model in one embodiment (the incremental model in this schematic diagram refers to...). Figure 1 (Construction). Generally speaking, in the model training process that does not involve distribution difference loss, the training process can be simply summarized as adjusting the model parameters based on the difference between the prediction result (in this example, the prediction transformation result) and the label (in this example, the transformation label), with the goal of making the prediction result as close to the label as possible. Figure 2 L Outcome This represents the predicted loss determined based on the predicted conversion results and conversion tags, L Outcome The solid lines represent the parameter adjustments made during training based on the prediction loss (generally, parameter adjustments are necessary for the shared representation extraction layer and the backbone prediction network, while adjustments to the feature embedding layer can be decided by the trainer). The purpose of adding the distribution difference loss is to eliminate selection bias, that is, to ensure that the features used for prediction (in this embodiment, the comprehensive features) are shared by both the intervention and control groups. Therefore, the parameter adjustments made based on the distribution difference loss are also performed on the shared representation extraction layer. Figure 2 L Distribution This represents the loss due to distributional differences, L Distribution The connected dashed lines represent the parameter adjustments made during training based on the distribution difference loss.

[0026] Note that for the shared representation extraction layer, the parameters of this model layer need to be adjusted regardless of whether the prediction loss or the distribution difference loss is used. However, if the prediction loss and the distribution difference loss are considered independently, the direction of the parameter adjustments made based on the prediction loss and the distribution difference loss often differs greatly.

[0027] Imagine that the training objective of adjusting parameters based on distributional difference loss is to eliminate the distributional differences in the comprehensive features corresponding to the intervention and control groups. Ideally, the comprehensive features should only contain feature information shared by both groups. However, if the training objective is solely to make the predicted results as close to the labels as possible (i.e., training only based on prediction loss), due to the selection bias effect, adding feature information that leads to selection bias to the features could improve the similarity between the predicted results and the labels. However, the feature information that leads to selection bias is precisely the feature information unique to either the intervention or control group.

[0028] In other words, the training objectives corresponding to prediction loss and distribution difference loss are inherently conflicting, leading to a conflict in the parameter adjustment directions corresponding to prediction loss and distribution difference loss. Due to the conflict in parameter adjustment directions, the parameter adjustment direction and magnitude obtained by combining the two types of losses often fail to meet either the goal of eliminating distribution difference or the goal of improving prediction accuracy. This also makes it difficult for the parameters of incremental models in existing technologies to converge during training.

[0029] Therefore, in the training process of the incremental model, this embodiment adjusts the weight of the distribution difference loss according to the iteration rounds of parameter adjustment, thereby adjusting the degree of influence of the distribution difference loss on the training process and the timing of its effect. After the parameter adjustment based on the prediction loss is relatively stable, the distribution difference loss is introduced into the training process. In the time dimension, the misalignment of the two types of loss on the training process reduces the problem of model parameters being difficult to converge due to the conflict between the two types of loss in the prior art.

[0030] Specifically, this embodiment provides a method for training an incremental model. This method can be executed by a computing device or computing node with computing capabilities to train the incremental model. The incremental model can be as follows: Figure 1 As shown, it may also include only a shared representation extraction layer and a backbone prediction model; this specification does not impose any limitations. The method for training the incremental model may include multi-round iterative parameter tuning, where the parameter tuning process for each round can be as follows: Figure 3 As shown, it includes:

[0031] Step S301: Obtain several training samples corresponding to the current round, wherein each training sample includes several user features of the sample user and intervention and conversion tags for the sample user.

[0032] In this embodiment, the incremental model can be applied to e-commerce scenarios to predict user conversion behavior after receiving marketing measures. Therefore, considering the characteristics of e-commerce scenarios, the training samples may include several user characteristics and corresponding labels (including intervention labels indicating whether the user accepted intervention and conversion labels indicating whether the user made a conversion). The user characteristics can be of the types commonly used in incremental prediction, such as attribute features (user's age, gender, etc.) and behavioral features (user's page browsing time, historical conversion frequency, etc.), and this specification does not impose specific limitations.

[0033] Furthermore, this specification does not impose any limit on the number of training samples obtained in each round.

[0034] Step S303: For any training sample, using the shared representation extraction layer, determine the comprehensive features corresponding to the training sample based on several user features of the training sample.

[0035] It should be noted that the processing performed on each training sample obtained in the current round is the same in steps S303 and S305. The following will only take one training sample as an example to introduce steps S303 and S305.

[0036] After obtaining the training sample, feature extraction can be performed. In some implementations, if the incremental model only includes a shared representation extraction layer and a backbone prediction network, the shared representation extraction layer can be used directly to extract several user features corresponding to the training sample to obtain comprehensive features. That is, the shared representation extraction layer simultaneously transforms several user features from different data dimensions (e.g., gender is categorical data, while page view time is continuous data) to a unified dimension, and extracts comprehensive features based on the transformed user features.

[0037] In other implementations, if the structure of the incremental model is as follows: Figure 1 As shown, the feature embedding layer can first be used to transform several user features of the training sample to a unified dimension. Then, the shared representation extraction layer is used to extract the transformed user features to obtain comprehensive features. Therefore, in subsequent parameter adjustments, the parameter adjustments made to the shared representation extraction layer can be more focused on enhancing the comprehensive feature extraction capability of the shared representation extraction layer.

[0038] Step S305: Using the backbone prediction network, determine the prediction transformation result corresponding to the training sample based on the comprehensive features.

[0039] After determining the comprehensive features corresponding to the training sample, the backbone prediction network can be used to determine the prediction transformation result corresponding to the training sample.

[0040] Specifically, the backbone prediction network may include two branches—an intervention branch and a control branch. These branches can have independent parameters. The intervention branch is used to predict the conversion outcome after intervention, while the control branch predicts the conversion outcome without intervention. In step S305, since it is a training process, based on the intervention labels corresponding to the training samples, only the branch corresponding to the intervention label is used to determine the predicted conversion outcome (if several pre-labels are "received intervention," the intervention branch is used to process this comprehensive feature; if several pre-labels are "not received intervention," the control branch is used to process this comprehensive feature), while the other branch does not need to be activated. The predicted conversion outcome can be represented as the predicted probability of the conversion behavior occurring.

[0041] Since the intervention label for a training sample can only be either "received intervention" or "did not receive intervention," the corresponding conversion label for that training sample can only represent the actual conversion result under the corresponding intervention label. Therefore, the conversion label can only provide accurate guidance for the output (i.e., the predicted conversion result) of the corresponding branch. The output of the other branch, lacking a corresponding conversion label, cannot be used for subsequent parameter adjustments. Therefore, in step S305, only one branch is used to process the comprehensive feature for a single training sample.

[0042] Step S307: Determine the prediction loss based on the difference between the prediction conversion results and conversion labels of the training samples; determine the distribution difference loss based on the intervention labels and comprehensive features corresponding to the training samples; determine the weight corresponding to the distribution difference loss based on the number of rounds in the current round.

[0043] After obtaining the prediction transformation results for all training samples in the current round, the comprehensive loss L for the current round can be further determined. Specifically, this comprehensive loss may include the prediction loss and the distribution difference loss:

[0044] L = L Outcome +λL Distribution

[0045] The prediction loss can be determined using a preset prediction loss function based on the difference between the predicted conversion result and the conversion label for each training sample. This prediction loss function can be, for example, the cross-entropy loss function or various variant loss functions constructed based on the cross-entropy loss function; this specification does not impose any restrictions.

[0046] The distributional difference loss can be determined using a preset distributional difference loss function based on the differences in the comprehensive characteristics of training samples in the intervention group and the control group. Specifically, for any training sample, if its intervention label is "received intervention," then the training sample belongs to the intervention group; if its intervention label is "did not receive intervention," then the training sample belongs to the control group. This distributional difference loss function can be, for example, the Maximum Mean Discrepancy (MMD) function or various variant loss functions constructed based on the MMD function; this specification does not impose any restrictions.

[0047] To fuse the prediction loss and the distribution difference loss to obtain the comprehensive loss, it is also necessary to determine the weight λ corresponding to the distribution difference loss during fusion. In this embodiment, the weight corresponding to the distribution difference loss is determined based on the number of rounds in the current round.

[0048] The following section details the technical approach of adjusting the weights corresponding to the distribution difference loss. First, it's crucial to understand that for an incremental model, prediction accuracy is its most critical evaluation criterion. During training, the prediction loss aims to improve the prediction accuracy of the incremental model. (The distribution difference loss, on the other hand, reduces selection bias introduced by training samples, improving the generalization ability of the incremental model.) Therefore, in this embodiment, the prediction loss for each round should not be ignored.

[0049] Furthermore, considering the conflict between prediction loss and distribution difference loss, this embodiment determines the weight corresponding to the distribution difference loss based on the number of rounds in the current round. In some implementations, the weight corresponding to the distribution difference loss can be set to be lower as the number of rounds in the current round decreases. This reduces the interference of distribution difference loss on parameter adjustments caused by prediction loss in lower rounds, allowing the incremental model to focus more on learning the ability to accurately predict conversion results in lower rounds. In higher rounds, the incremental model has gradually learned the ability to accurately predict conversion results through learning in previous rounds, and the prediction loss naturally decreases with parameter adjustments in each iteration. Therefore, even if the weight corresponding to distribution difference loss is increased in higher rounds, since the prediction loss has been greatly reduced compared to lower rounds, the prediction loss will not cause excessive interference to parameter adjustments caused by distribution difference loss. This allows the incremental model to focus more on learning the ability to eliminate selection bias in higher rounds. On the other hand, even though the parameter adjustments caused by the distribution difference loss may interfere with the ability to make accurate predictions, the overall loss always includes the prediction loss, regardless of whether it is a high-round or low-round period, and each round can make up for the interference caused by the distribution difference loss accordingly.

[0050] Therefore, this embodiment adjusts the weights corresponding to the distribution difference loss, thereby relatively decoupling the learning processes of different dimensions (prediction accuracy, elimination of selection bias) of the incremental model, realizing the organic integration of distribution difference loss and prediction loss, which can reduce the number of iterations required for model parameter convergence and reduce the computational power consumption of the model training process.

[0051] In some implementations, if the number of rounds in the current round is lower than a preset round threshold, the weight corresponding to the distribution difference loss can be set to 0.

[0052] It should be noted that when the weight corresponding to the distribution difference loss is 0, there is no need to determine the distribution difference loss in step S307. Specifically, the round threshold can be determined based on the number of training samples obtained in each round (usually, the number of training samples obtained in each round is the same). For example, the number of training samples obtained in each round can be multiplied by a transformation coefficient, and the result of the multiplication can be used as the round threshold. This transformation coefficient can be any value between 0.5 and 2, for example, it can be set to 1.

[0053] Therefore, in lower rounds, the incremental model can adjust its parameters only based on the prediction loss, thus accelerating the learning of the incremental model to achieve the ability to make accurate predictions.

[0054] Step S309: Adjust the parameters of the incremental model according to the prediction loss, the distribution difference loss, and the weights.

[0055] After determining the comprehensive loss corresponding to the current round, the gradients for each model layer of the incremental model (including at least the shared representation extraction layer and the backbone prediction network) can be determined based on the comprehensive loss. The parameters of each model layer of the incremental model can then be adjusted based on the calculated gradients.

[0056] Additionally, after completing step S309, it can be confirmed whether the preset termination condition has been met. If not, the process continues according to the following steps: Figure 3 The method shown is used to adjust the parameters in the next round.

[0057] The termination condition may be, for example, reaching a certain number of iterations or the parameter adjustment amount in the current iteration being less than a certain threshold, but this specification does not impose any restrictions on it.

[0058] In some implementations, the incremental model adjusted in the current round can be used to determine the prediction results after intervention and the prediction results without intervention for each test sample in the test set; based on the conversion results after intervention and the prediction results without intervention for each test sample, the gain curve corresponding to the current round can be determined; based on the area under the gain curve of the current round and each previous round, it can be determined whether to adjust the parameters for the next round.

[0059] Specifically, the gain curve is the curve showing the cumulative gain as a function of the sample proportion after sorting each test sample by predicted gain (the difference between the predicted result after intervention and the predicted result without intervention) from high to low. The area under the uplift curve (AUUC) measures the effectiveness of the incremental model on the test set. If the difference between the area under the gain curve of the current round and the average area under the gain curve of each previous round in the preset historical period is less than the preset area difference threshold, then it can be considered that the parameter adjustment of the current round has not brought about improvement, and thus it can be regarded as reaching the preset termination condition; otherwise, the parameter adjustment of the next round continues.

[0060] like Figure 3 The method shown here is to train an incremental model by adjusting the weights of the distribution difference loss according to the iteration rounds, which can accelerate the convergence speed of the parameters of the incremental model during the model training process and reduce the computational power consumption of the model training process.

[0061] In addition to the conflict between loss functions, another disruptive factor in e-commerce model training is the sheer number of user features. Specifically, in e-commerce, user features can encompass dozens or even hundreds of different types, each with a specific value range. However, the number of training samples obtained in each round is limited (typically no more than 2000). Considering the presence of intervention labels, these 2000 users need to be divided into intervention and control groups. The dimensionality of user features can even exceed the number of users in a single group. Consequently, the feature distribution of the intervention and control groups in each round is highly susceptible to random error interference, and the magnitude of this interference is also substantial. Adjusting the incremental model parameters based on the distribution difference loss caused by random error results in a level of adjustment that is difficult to recover from even after multiple rounds of parameter adjustments.

[0062] Therefore, this embodiment selects to further control the value of the distribution difference loss in order to improve the stability of the training process and reduce the impact of random errors on the parameters of the incremental model.

[0063] In some implementations, in step S307, each comprehensive feature can be normalized to obtain a normalized feature corresponding to each comprehensive feature with the same feature size; based on the difference in the normalized features between each training sample of the intervention group and each training sample of the control group, the distribution difference loss is determined.

[0064] Typically, composite features are represented in vector form. Therefore, L2 normalization can be used to process each composite feature into a normalized feature with a vector length of 1. This limits the size of the composite features, and the value space of the distribution difference loss determined by the composite features will also be limited accordingly.

[0065] In some other implementations, in step S307, radial basis functions can also be used to determine the first distribution similarity of each training sample in the intervention group, the second distribution similarity of each training sample in the control group, and the cross-distribution similarity between the intervention group and the control group based on the comprehensive features; and the distribution difference loss is determined based on the first distribution similarity, the second distribution similarity, and the cross-distribution similarity.

[0066] Specifically, the first distribution similarity, the second distribution similarity, and the cross-distribution similarity can be determined using the kernel matrix calculation method in the radial basis function. Since the radial basis function is equivalent to mapping the comprehensive features to a higher-dimensional representation space, the value space of the first distribution similarity, the second distribution similarity, and the cross-distribution similarity is more compact than that of the comprehensive features, and the value space of the distribution difference loss will also be correspondingly limited.

[0067] Furthermore, in step S307, radial basis functions can be used to determine, based on comprehensive features, a first kernel matrix representing the similarity of training samples within the intervention group, a second kernel matrix representing the similarity of training samples within the control group, and a third kernel matrix representing the similarity of training samples between the intervention group and the control group; numerical truncation is performed on each element in the first kernel matrix, the second kernel matrix, and the third kernel matrix to determine a first truncation matrix corresponding to the first kernel matrix, a second truncation matrix corresponding to the second kernel matrix, and a third truncation matrix corresponding to the third kernel matrix; the first distribution similarity is determined based on the first truncation matrix; the second distribution similarity is determined based on the second truncation matrix; and the third distribution similarity is determined based on the third truncation matrix.

[0068] Therefore, the numerical values ​​of the elements in the kernel matrix are limited by the numerical truncation process, and the similarity of the distributions determined by the kernel matrix is ​​also correspondingly limited. In some implementations, the upper limit of this numerical truncation process can be set to 10. 7 The lower limit of the value can be set to 10. -7 .

[0069] In addition, some implementations may limit the number of training samples acquired in each round to reduce the risk of training crashes due to numerical overflow during the calculation of distribution difference loss.

[0070] Figure 4This specification illustrates a flowchart of a method for determining an intervention strategy according to an embodiment of the present invention. This method for determining an intervention strategy can utilize, for example... Figure 3 The incremental model trained after the method shown in the diagram is executed, specifically including:

[0071] Step S401: Obtain several user characteristics of the target user.

[0072] The types of user features obtained in step S401 can be the same as those obtained in step S301.

[0073] Step S403: Using the shared representation extraction layer, determine the comprehensive characteristics of the target user based on several user characteristics of the target user.

[0074] Specifically, the execution method of step S403 can be found in step S303.

[0075] Step S405: Based on the comprehensive characteristics of the target user, determine the post-intervention prediction result corresponding to the target user using the intervention branch, and determine the non-intervention prediction result corresponding to the target user using the control branch.

[0076] Specifically, the execution method of step S405 can be found in step S305. It should be noted that, unlike step S305, in the actual prediction task, both the intervention branch and the control branch need to be enabled, and the intervention branch and the control branch make predictions based on the same comprehensive features.

[0077] Step S407: Determine the actual conversion effect of the target user based on the difference between the prediction results after intervention and the prediction results without intervention.

[0078] As mentioned above, the difference between the predicted results after intervention and the predicted results without intervention represents the actual conversion effect of the intervention measures on the target user.

[0079] Step S409: Based on the actual conversion effect, determine the intervention strategy for the target user.

[0080] Specifically, a conversion effect threshold can be preset. When the actual conversion effect corresponding to the target user reaches the conversion effect threshold, the target user is taken as the intervention target and intervention measures are applied to the target user.

[0081] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0082] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0083] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0084] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0085] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0090] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0091] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0092] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0094] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0095] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for training an incremental model, the method relating to an incremental model including a shared representation extraction layer and a backbone prediction network, the method including parameter tuning in multiple iterations, wherein the parameter tuning in any iteration specifically includes: Obtain several training samples corresponding to the current round, wherein each training sample includes several user features of the sample user as well as intervention tags and conversion tags for the sample user; For any training sample, the shared representation extraction layer is used to determine the comprehensive features corresponding to the training sample based on several user features of the training sample. Using the backbone prediction network, the prediction conversion result corresponding to the training sample is determined based on the comprehensive features; The prediction loss is determined based on the difference between the predicted conversion results and conversion labels of the training samples; the distribution difference loss is determined based on the intervention labels and comprehensive features of the training samples; and the weight corresponding to the distribution difference loss is determined based on the number of rounds in the current round. The parameters of the incremental model are adjusted based on the prediction loss, the distribution difference loss, and the weights.

2. The method as described in claim 1, wherein the distribution difference loss is determined based on the intervention labels and comprehensive features corresponding to each of the plurality of training samples, specifically including: The distributional difference loss is determined based on the differences in the comprehensive characteristics of each training sample in the intervention group and each training sample in the control group, wherein the group to which each training sample belongs is determined according to the intervention label.

3. The method as described in claim 2, wherein the distribution difference loss is determined based on the differences in the comprehensive characteristics of each training sample in the intervention group and each training sample in the control group, specifically including: Normalize each comprehensive feature to obtain the normalized feature corresponding to each comprehensive feature with the same feature size; The distributional difference loss is determined based on the differences in the normalized characteristics of each training sample in the intervention group and each training sample in the control group.

4. The method as described in claim 2, wherein the distribution difference loss is determined based on the differences in the comprehensive characteristics of each training sample in the intervention group and each training sample in the control group, specifically including: Using radial basis functions, the first distribution similarity of each training sample within the intervention group, the second distribution similarity of each training sample within the control group, and the cross-distribution similarity between the intervention group and the control group are determined based on comprehensive features. The distribution difference loss is determined based on the first distribution similarity, the second distribution similarity, and the cross-distribution similarity.

5. The method as described in claim 4, utilizing radial basis functions to determine the first distribution similarity of each training sample within the intervention group, the second distribution similarity of each training sample within the control group, and the cross-distribution similarity between the intervention group and the control group based on comprehensive features, specifically includes: Using radial basis functions, a first kernel matrix representing the similarity of training samples within the intervention group, a second kernel matrix representing the similarity of training samples within the control group, and a third kernel matrix representing the similarity of training samples between the intervention group and the control group are determined based on comprehensive features. Numerical truncation is performed on each element in the first kernel matrix, the second kernel matrix, and the third kernel matrix to determine the first truncation matrix corresponding to the first kernel matrix, the second truncation matrix corresponding to the second kernel matrix, and the third truncation matrix corresponding to the third kernel matrix. The first distribution similarity is determined based on the first truncation matrix; the second distribution similarity is determined based on the second truncation matrix; and the third distribution similarity is determined based on the third truncation matrix.

6. The method as described in claim 1, wherein the weight corresponding to the distribution difference loss is determined based on the number of rounds in the current round, specifically includes: If the number of rounds in the current round is lower than the preset round threshold, the weight corresponding to the distribution difference loss is set to 0.

7. The method of claim 1, wherein, The backbone prediction network includes an intervention branch and a control branch; Using the backbone prediction network, the prediction transformation result corresponding to the training sample is determined based on the comprehensive features, specifically including: Based on the intervention label corresponding to the training sample, determine the target branch between the intervention branch and the control branch; Using the target branch, the predicted transformation result corresponding to the training sample is determined based on the comprehensive features.

8. The method of claim 7, further comprising: Obtain several user characteristics of the target user; Using the shared representation extraction layer, the comprehensive characteristics of the target user are determined based on several user characteristics of the target user; Based on the comprehensive characteristics of the target user, the intervention branch is used to determine the post-intervention prediction result for the target user, and the control branch is used to determine the non-intervention prediction result for the target user. The actual conversion effect of the target user is determined based on the difference between the prediction results after intervention and the prediction results without intervention. Based on the actual conversion effect, determine the intervention strategy for the target user.

9. The method of claim 8, further comprising, after adjusting the parameters of the incremental model according to the prediction loss, the distribution difference loss, and the weights: Using an incremental model, we determined the predicted results after intervention and the predicted results without intervention for each test sample in the test set. Based on the post-intervention conversion results and non-intervention prediction results corresponding to each test sample, determine the gain curve corresponding to the current round; Based on the area under the gain curve of the current round and each previous round, determine whether to adjust the parameters for the next round.

10. A computing device comprising a memory and a processor, the memory storing executable code, wherein the processor, when executing the executable code, implements the method as claimed in any one of claims 1-9.

Images