Training methods, devices, electronic equipment, and storage media for neural networks

Abstract

This application discloses a training method, apparatus, electronic device, and storage medium for a neural network, belonging to the field of data processing technology. The method includes: obtaining a first predicted probability value predicted by a first neural network among two adjacent neural networks in M ​​neural networks when a training sample is used as input; a second neural network in the M neural networks uses the first predicted probability value as the initial value of the input weights to perform weighted processing on the training samples to obtain a weighted result; the second neural network obtains a calculation result based on the weighted result and a preset deviation value, and determines the prediction error value between the calculation result and the actual result of the training samples based on the calculation result and a preset loss function, wherein the loss function has a reward score; updating the reward score based on the prediction error value; and updating the input weights and deviation value based on the prediction error value and the updated reward score until the loss function converges, at which point the training of the second neural network ends.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and specifically relates to a training method, apparatus, electronic device and storage medium for a neural network. Background Technology

[0002] Machine learning in related technical solutions mainly involves single-task learning. For more complex multi-task application scenarios, it is usually decomposed into multiple single tasks that can be learned separately, and the results of the learning are combined to obtain the final learning result.

[0003] However, in multi-task application scenarios, the relevant technical solutions only consider one transformation goal and cannot take into account other transformation goals. Therefore, the trained neural network has the drawback of imbalance between different transformation goals. Summary of the Invention

[0004] The purpose of this application embodiment is to provide a training method, apparatus, electronic device, and storage medium for a neural network. In this embodiment, the training of the second neural network takes into account the training results of other neural networks, which improves the problem of imbalance between different transformation targets in existing trained neural networks. This enables the trained multi-task neural network to balance multiple transformation targets and improves the accuracy of the trained neural network.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for training a neural network. The number of neural networks is M, where M is a positive integer greater than or equal to 2. The task links of the M neural networks are connected, and the M neural networks are trained using the same sample training set, which includes training samples. The training method includes: obtaining a first prediction probability value predicted by the first neural network of two adjacent neural networks in the M neural networks when the training samples are used as input; the second neural network in the M neural networks uses the first prediction probability value as the initial assignment of input weights to perform weighted processing on the training samples to obtain a weighted result; the second neural network obtains a calculation result based on the weighted result and a preset deviation value, and determines the prediction error value between the calculation result and the actual result of the training samples based on the calculation result and a preset loss function, wherein the loss function has a reward score; updating the reward score based on the prediction error value; and updating the input weights and deviation value based on the prediction error value and the updated reward score, until the loss function converges and the training of the second neural network ends.

[0007] Secondly, embodiments of this application provide a training device for a neural network. The number of neural networks is M, where M is a positive integer greater than or equal to 2. The task links of the M neural networks are connected, and the M neural networks are trained using the same sample training set, which includes training samples. The training device for the neural network includes: an acquisition module, used to acquire a first prediction probability value predicted by the first neural network of two adjacent neural networks in the M neural networks when the training samples are used as input; a weighting module, used by the second neural network in the M neural networks to use the first prediction probability value as the initial value of the input weights, and to perform weighting processing on the training samples to obtain a weighted result; a calculation module, used by the second neural network to obtain a calculation result based on the weighted result and a preset deviation value, and to determine the prediction error value between the calculation result and the actual result of the training samples based on the calculation result and a preset loss function, wherein the loss function has a reward score; and an update module, used to update the reward score based on the prediction error value; and to update the input weights and deviation value based on the prediction error value and the updated reward score, until the loss function converges and the training of the second neural network ends.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method as described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, for a multi-task neural network, specifically M interconnected neural networks involved in this application, during neural network training, the first predicted probability value obtained by the first neural network in two adjacent neural networks when using training samples as input is used as the initial value of the input weights for the second neural network. This allows the first predicted probability value of the first neural network to participate in the training of the second neural network as an influencing factor. During this training process, the training of the second neural network takes into account the training results of other neural networks, improving the imbalance problem existing in trained neural networks between different transformation targets. This enables the trained multi-task neural network to balance multiple transformation targets, improving the accuracy of the trained neural network. Attached Figure Description

[0013] Figure 1 A schematic flowchart illustrating the neural network training method provided in this application embodiment;

[0014] Figure 2 This is one of the neural network connection diagrams provided in the embodiments of this application;

[0015] Figure 3 This is a second schematic diagram of the neural network connection provided in the embodiments of this application;

[0016] Figure 4 A schematic diagram illustrating the expression of the activation function provided in the embodiments of this application;

[0017] Figure 5 A schematic diagram illustrating parameter derivation in the neural network during the backpropagation stage, provided in an embodiment of this application;

[0018] Figure 6 A schematic block diagram of a neural network training apparatus provided in an embodiment of this application;

[0019] Figure 7 One of the hardware schematic diagrams of the electronic device provided in the embodiments of this application;

[0020] Figure 8 This is a second hardware schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0023] The training method, apparatus, electronic device, and storage medium of the neural network provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0024] like Figure 1 As shown, this application embodiment provides a method for training a neural network. The number of neural networks is M, where M is a positive integer greater than or equal to 2. The task links of the M neural networks are connected, and the M neural networks are trained using the same sample training set, which includes training samples. The training method includes:

[0025] Step 102: Obtain the first prediction probability value of the first neural network among two adjacent neural networks in the M neural networks when the training sample is used as input.

[0026] In this system, each of the M neural networks has a task, and the tasks are linked together. It can be understood that the tasks of different neural networks have a sequential relationship.

[0027] Specifically, for example, the tasks corresponding to the M neural networks include: downloading, activating, registering, and purchasing. Activation occurs after downloading, registration occurs after activation, and similarly, purchasing occurs after registration. Obviously, downloading, activating, registering, and purchasing occur sequentially, and the above relationship constitutes the task chain in this application.

[0028] For example, taking app promotion as an example, the different tasks in app promotion ads are interconnected, and the entire path (i.e., the chain) is download-activation-registration and purchase. Let's take this path as an example. Figure 2As shown, the entire model architecture comprises a common subnetwork and four task subnetworks. The four subnetworks are a download subnetwork (i.e., a download neural network), an activation subnetwork (i.e., an activation neural network), a registration subnetwork (i.e., a registration neural network), and a purchase subnetwork (i.e., a purchase neural network). These subnetworks communicate with the common subnetwork. The common subnetwork is used to extract features related to the relationships between tasks, i.e., the input data mentioned above. The download, activation, registration, and purchase subnetworks correspond to download, activation, registration, and purchase, respectively, and are used to train different conversion goals. The model ultimately obtains four outputs, namely the conversion rates for each task: pdCVR, pacVR, prCVR, and ppCVR. pdCVR represents the probability of downloading the ad task, pacVR represents the probability of activating the ad task, prCVR represents the probability of registering the ad task, and ppCVR represents the probability of purchasing the ad task.

[0029] In the embodiments of this application, a neural network containing four task sub-networks can be implemented to achieve balanced training of the four objectives of downloading, activating, registering, and purchasing.

[0030] In some embodiments of this application, such as Figure 3 As shown, M neural networks can be modified to add new task chains based on the actual needs of the task. Specifically, for example, on the existing M neural networks, a new task can be added as a second-retention sub-network, i.e., a second-retention neural network. In this case, the task chains of downloading, activation, and second-retention can be added without affecting the original task chains.

[0031] Specifically, the tasks corresponding to the M neural networks include download and second-day retention. Second-day retention, also known as the next-day retention rate, obviously occurs after download. That is, download and second-day retention happen sequentially, and the above relationship is the task chain in this application.

[0032] In the embodiments of this application, when the task of the first neural network is downloading, the first predicted probability value is the download rate; when the task of the first neural network is registration, the first predicted probability value is the registration rate, and so on, which will not be elaborated further here.

[0033] Step 104: The second neural network in the M neural networks uses the first predicted probability value as the initial value of the input weight to perform weighted processing on the training samples to obtain the weighted result.

[0034] Specifically, the input weight is set to the first predicted probability value, and the input weight is multiplied with the training sample to obtain the calculation result, which is the weighted result in this application. This result is then used as the training sample in the alternative sample training set to train the second neural network.

[0035] Step 106: The second neural network obtains the calculation result based on the weighted result and the preset deviation value, and determines the prediction error value between the calculation result and the actual result of the training sample based on the calculation result and the preset loss function. The loss function has a reward score.

[0036] In this embodiment, the neural network has a multi-layered structure, where the output of the previous layer can be used as the input of the next layer. Therefore, each layer has an input weight and a bias value. After the training samples are input into the second neural network, the input weights are multiplied by the samples to obtain the weighted result as described above. This weighted result is then calculated with the bias value and used as the output of the current layer. Simultaneously, it serves as the input to the next layer, ultimately yielding the output of the second neural network, which is the computational result in this application.

[0037] By using a preset loss function, the prediction error value of the second neural network is estimated by comparing the computation result with the actual result of the training sample, and then the prediction error value is used to train the second neural network.

[0038] Step 108: Update the reward score based on the prediction error value; and update the input weights and bias values ​​based on the prediction error value and the updated reward score, until the loss function converges and the training of the second neural network ends.

[0039] In this embodiment, the reward score is updated using the prediction error value to update the loss function, thereby selecting a suitable loss function for evaluating the second neural network. The input weights and bias values ​​are updated to train the second neural network. When the prediction error value estimated by the loss function approaches a set threshold, the loss function converges, and the training of the second neural network ends.

[0040] In this embodiment, for a multi-task neural network, specifically M interconnected neural networks involved in this application, during neural network training, the first predicted probability value obtained by the first neural network in two adjacent neural networks when using training samples as input is used as the initial value of the input weights for the second neural network. This allows the first predicted probability value of the first neural network to participate in the training of the second neural network as an influencing factor. During this training process, the training of the second neural network takes into account the training results of other neural networks, improving the imbalance problem existing in trained neural networks between different transformation targets. This enables the trained multi-task neural network to balance multiple transformation targets, improving the accuracy of the trained neural network.

[0041] In one embodiment, the preset loss function can be obtained by modifying an existing loss function. Specifically, the existing loss function includes:

[0042]

[0043] Where LOSS is the loss function, N is the number of input samples, yi represents the true class of the instance (in this application, it is a training sample), and pi represents the predicted probability of the input instance.

[0044] The preset loss function, modified from the existing loss function, is expressed as follows:

[0045]

[0046] In the case of the second neural network task being activation, DLD represents the reward score given for the download action, and pd is the probability that the download is not activated.

[0047] Specifically, in the relevant technical solution, when the second neural network is trained, its training labels include 0 and 1, where 0 represents inactive and 1 represents downloaded and activated. At this time, the trained model performs well for the activation conversion target but pays little attention to the download target. By modifying the existing loss function, the training labels of the second neural network include 0, 1, and 2, where 0 represents not downloaded, 1 represents downloaded but not activated, and 2 represents downloaded and activated. During label conversion, they are mapped to 0, DLD_REWARD and 1. DLD_REWARD represents how much reward score should be given for the download behavior, for example, 0.3, which means that the download behavior is considered while predicting activation. This reward score can be selected through expert methods or loss function convergence. Initially, 0.5 is given as a reward score. The loss function is used for backpropagation to continuously update the reward score until the model converges. By adjusting DLD_REWARD, a balance can be made between the download rate and the activation rate, thereby achieving a balance between multiple conversion targets under multiple neural networks. Specifically, when calculating the prediction error, the neural network considers both downloading and activation behaviors simultaneously. Activation, as the final goal, requires error calculation, while downloading, as an intermediate behavior, requires rewarding samples exhibiting this behavior for error calculation. In other words, in a scenario where activation is the goal, samples exhibiting only downloading behavior are also considered to need to be rewarded. By doing so, the model focuses on activation as the primary goal during training, and downloading behavior as the secondary goal. During backpropagation based on the loss value, both behaviors are updated, which is equivalent to two steps converging simultaneously in one model. After convergence, the model outputs the pdaCVR probability value.

[0048] The formula is as follows:

[0049]

[0050] Here, yilogpi+(1-yi)log(1-pi) can be understood as having activation as the main objective, and DLD×logpd+(1-DLD)log(1-Pd) is the reward for downloading.

[0051] In this process, the pdaCVR probability value obtained after the model converges is based on activating the primary target and downloading the secondary target model. The generated probability value takes into account both behaviors, that is, a trade-off is made between the download rate and the activation rate.

[0052] In one embodiment, since the M neural networks are trained using the same sample training set, different neural networks can be trained using the full amount of data, which reduces the impact of data sparsity and sample selection bias on the training results and improves the accuracy of the training results.

[0053] In some embodiments of this application, the training samples include one or more of the following: user attribute data, historical behavior data of electronic devices used by the user, user behavior data on historical delivery information, and contextual environment data of historical delivery information.

[0054] In this embodiment, the user's attribute data may be gender, age, and / or the user's location.

[0055] Among them, the historical behavior data of the user's electronic devices can be usage data of applications installed on the electronic devices, historical browsing data of products, etc.

[0056] In one embodiment, user behavior data in response to historical information can be understood as user response data in response to historical information. For example, if the historical information is a product advertisement, the response data may be clicks on the advertisement and / or purchases of the product. Similarly, if the historical information is an application promotion advertisement, the response data may be downloads, activations, registrations, and purchases of the application.

[0057] In one embodiment, the contextual data of historical delivery information can be understood as the environmental data of historical delivery information in the information viewed by the user. For example, in the scenario of browsing a game list, an advertisement for a game application is inserted into the game list, or in the scenario of browsing a product list, an advertisement for a product is inserted into the product list.

[0058] In some embodiments of this application, training samples are obtained through feature processing. Specifically, the processed features are continuous features that are directly input into the model. Categorical features are transformed into a numerical vector using a one-hot encoding method, such as gender, which is transformed into [0, 1] or [1, 0], so that it can be used as input to the neural network.

[0059] In some embodiments of this application, the second neural network obtains the calculation result based on the weighted result and a preset deviation value, including: determining the sum of the weighted result and the deviation value; and inputting the sum value as input to the activation function in the second neural network to obtain the calculation result.

[0060] The training of the second neural network includes a forward propagation phase and a backward propagation phase. In the forward propagation phase, the first predicted probability value from the previous round of the neural network is used as an influencing factor, and the initial magnitude of the input weights of the training samples is used as the input to the second neural network. The calculation results are then used to update the input weights and bias values ​​in the backward propagation phase.

[0061] Specifically, taking the activation conversion rate paCVR as an example, it is necessary to estimate the download conversion rate pdCVR. The input weights are based on the first predicted probability value (i.e., conversion rate) predicted by the previous round of neural network. Based on this, the input weights w will initially be assigned the probability predicted by the pdCVR model, i.e., w1 = p(d = 1|x1), w2 = p(d = 1|x2).

[0062] Where d=1 represents the download action.

[0063] like Figure 4 As shown, the input is then weighted, summed, and the bias b is added before the result is passed to the activation function f(x). The activation function f(x) is expressed as follows:

[0064] y = f(x1×w1 + x2×w2 + b);

[0065] In the formula, y represents the output, f(x) uses the sigmoid function, x1 is training sample 1, w1 is the probability of x1 predicted on the pdCVR model, x2 is training sample 2, w2 is the probability of x2 predicted on the pdCVR model, and b is the random bias.

[0066] In some embodiments of this application, updating the input weights and deviation values ​​based on the prediction error value and the updated reward score includes: taking the derivative of the input weights and deviation values ​​based on the prediction error value and the updated reward score to obtain the gradient values ​​of the input weights and deviation values; updating the input weights based on the gradient values ​​of the input weights, the updated reward score, and the initial assignment of the input weights; and updating the deviation values ​​based on the gradient values ​​of the deviation values, the updated reward score, and the initial assignment of the deviation values.

[0067] In this embodiment, which belongs to the backpropagation stage of the second neural network, the prediction error value between the calculation result and the actual result of the training sample is determined using the loss function, and the reward score is updated according to the prediction error value so as to update the input weights and bias values ​​using the updated reward score.

[0068] Specifically, since there is an error between the output of forward propagation and the actual result, the modified loss function is used to calculate the error between the estimated value and the actual value. In other words, the reward score is adjusted within a preset range to obtain the minimum prediction error value.

[0069] In some embodiments of this application, error estimation is performed using two actions: downloading and activation. The initial value of the reward score DLD is 0.5, and the range is [0.1, 0.9].

[0070] During the backpropagation phase, the input weights and biases are updated using the reward score values, with the input weights and biases being updated from the back to the front.

[0071] like Figure 5 As shown, specifically, the prediction error value of the last layer is obtained, the derivative of the input weights and the bias value is calculated, and the gradient of the input weights and the bias value is obtained. Here, due to the adjustment of the loss function, the reward score is used as a constant. Then, the gradients of the input weights and the bias value of the first two layers are obtained by taking the derivatives respectively. Then, the input weights and the bias value are updated starting from the first layer.

[0072] The formula is as follows:

[0073]

[0074]

[0075] Where W represents the input weights in the previous layer, B represents the bias value in the previous layer, w represents the input weights in the current layer, b represents the bias value in the current layer, DLD is the reward score, and η is the learning rate set in the model. It is the calculated gradient of w. The calculated gradient b is used to update the input weights and biases for each layer. The gradients of the input weights and biases are different for each layer. The reward score corresponding to the lowest error value is used to update the input weights and biases until the loss function converges.

[0076] In some embodiments of this application, after the training of the second neural network is completed, the method further includes: outputting a second predicted probability value of the second neural network trained with the training samples as input.

[0077] In this embodiment, the second prediction probability value of the second neural network is output to enable the training of other neural networks. At the same time, it is also convenient to use the output second prediction probability value for information processing when using neural networks for data prediction.

[0078] Information processing includes, for example, information recommendation.

[0079] In some embodiments of this application, the first prediction probability value is the prediction probability value obtained by the first neural network predicting with the first transformation target; the second prediction probability value is the prediction probability value obtained by the second neural network predicting with the second transformation target; the second neural network also has a third prediction probability value, which is determined based on the first prediction probability value and the second prediction probability value.

[0080] In this embodiment, each neural network can be trained with a transformation target. In the embodiments of this application, the implicit learning of the third prediction probability value can be achieved by using the first prediction probability value of the first neural network and the second prediction probability value of the second neural network, thereby improving the sampling bias and overcoming the problem of data sparsity as the number of posterior task samples increases.

[0081] Specifically, taking pdCVR, pdaCVR, and pacVR as examples:

[0082]

[0083] Where p(a=1|d=1,x) represents paCVR, p((d=1,a=1||d=1,a=0|x) represents pdaCVR, and p(d=1|x) represents pdCVR.

[0084] In the relevant technical solutions, the pdaCVR probability value only considers the download and activation scenarios. The conversion probability is calculated as P(d=1,a=1|x). The adjusted new pdaCVR is shown in the following formula. When calculating the pdaCVR value, predictions are made based on both download and activation and download without activation. As shown in the above conditional probability formula, the download rate is inversely proportional to the activation rate. During backpropagation, the neural network automatically finds the balance point between the two through multiple iterations, thus learning the activation rate while simultaneously incentivizing download behavior. This obtains a new pdaCVR while keeping pdCVR constant, avoiding situations where some neural networks predict a high activation rate but a low download rate.

[0085] The above principle can be applied to other conversion rates, such as registration conversion rate. It is necessary to first estimate the download and activation conversion rate pdaCVR and the download, activation and registration conversion rate pdarCVR. Based on the same principle, the registration conversion rate prCVR can be obtained by dividing them.

[0086] Specifically, p(r=1|d=1,a=1,x)=p(d=1,a=1,r=1|x) / p(d=1,a=1|x).

[0087] Where p(r=1|d=1,a=1,x) represents prCVR, p(d=1,a=1,r=1|x) represents pdarCVR, and p(d=1,a=1|x) represents pdaCVR.

[0088] Based on the probability chain rule, the conversion rate corresponding to each conversion target can be calculated.

[0089] Specifically, taking activation conversion rate (paCVR) as an example, it is necessary to estimate download conversion rate (pdCVR) and download and activation conversion rate (pdaCVR). From the perspective of conditional probability, the relationship among the three is as follows:

[0090] p(d=1,a=1│x)=p(d=1│x)×p(a=1│d=1,x)

[0091] Among them, p(d=1,a=1│x) is also known as padCVR, p(d=1│x) is also known as pdCVR, and p(a=1│d=1,x) is also known as paCVR.

[0092] In this formula, d represents download, a represents activation, x represents user feature data, and p is both the probability and the model's prediction result. It can be seen that the prior for pdaCVR and pdCVR is x, meaning that the entire user base can be used to estimate these probabilities. However, the prior for pacVR is d=1 and x, meaning the users after downloading cannot be directly derived from the model training. The entire user base is needed to predict pdaCVR and pdCVR, and then pacVR is used as an intermediate variable. That is, after inputting all samples, parameters are shared in the common sub-network, and then different tasks are trained in each task sub-network. The download sub-network is trained with download as the conversion target, and the activation sub-network is trained with download and activation as the conversion target. After obtaining the two probability values, they are divided to obtain the activation conversion rate. This method bypasses the requirement of "using only download users as training samples."

[0093] The neural network training method provided in this application can be executed by a neural network training device. This application uses an example of a neural network training device executing the neural network training method to illustrate the neural network training device provided in this application.

[0094] In some embodiments of this application, such as Figure 6As shown, a neural network training device 600 is proposed. The number of neural networks is M, where M is a positive integer greater than or equal to 2. The task links of the M neural networks are connected, and the M neural networks are trained using the same sample training set, which includes training samples. The neural network training device 600 includes: an acquisition module 602, used to acquire the first prediction probability value predicted by the first neural network of two adjacent neural networks in the M neural networks when the training samples are used as input; a weighting module 604, used by the second neural network in the M neural networks to use the first prediction probability value as the initial value of the input weights, and to perform weighted processing on the training samples to obtain a weighted result; an operation module 606, used by the second neural network to obtain the operation result based on the weighted result and a preset deviation value, and to determine the prediction error value between the operation result and the actual result of the training samples based on the operation result and a preset loss function, where the loss function has a reward score; and an update module 608, used to update the reward score based on the prediction error value; and to update the input weights and deviation value based on the prediction error value and the updated reward score, until the loss function converges and the training of the second neural network ends.

[0095] In this embodiment, for a multi-task neural network, specifically M interconnected neural networks involved in this application, during neural network training, the first predicted probability value obtained by the first neural network in two adjacent neural networks when using training samples as input is used as the initial value of the input weights for the second neural network. This allows the first predicted probability value of the first neural network to participate in the training of the second neural network as an influencing factor. During this training process, the training of the second neural network takes into account the training results of other neural networks, improving the imbalance problem existing in trained neural networks between different transformation targets. This enables the trained multi-task neural network to balance multiple transformation targets, improving the accuracy of the trained neural network.

[0096] In some embodiments of this application, the weighting module 604 is used to: determine the sum of the weighted result and the deviation value; and input the sum as input to the activation function in the second neural network to obtain the calculation result.

[0097] In some embodiments of this application, the computation module 606 is used to: differentiate the input weights and the deviation value based on the prediction error value and the updated reward score to obtain the gradient value of the input weights and the gradient value of the deviation value; update the input weights based on the gradient value of the input weights, the updated reward score and the initial assignment of the input weights; and update the deviation value based on the gradient value of the deviation value, the updated reward score and the initial assignment of the deviation value.

[0098] In some embodiments of this application, the update module 608 is also used to: output a second predicted probability value of the second neural network trained with the training samples as input.

[0099] In some embodiments of this application, the first prediction probability value is the prediction probability value obtained by the first neural network predicting with the first transformation target; the second prediction probability value is the prediction probability value obtained by the second neural network predicting with the second transformation target; the second neural network also has a third prediction probability value, which is determined based on the first prediction probability value and the second prediction probability value.

[0100] In some embodiments of this application, an information push method is proposed, including: receiving input data, the input data being associated with a user; inputting the input data into a neural network trained by the neural network training method of any of the above embodiments, to obtain multiple predicted probability values ​​output by the neural network; and pushing information to the user based on the multiple predicted probability values.

[0101] In this embodiment, the proposed information push method pushes information to users based on multiple predicted probability values ​​output by the neural network trained above, thereby achieving targeted information delivery, improving the conversion rate of delivered information, and ensuring the effectiveness of information delivery.

[0102] In some embodiments of this application, users are sorted according to multiple predicted probability values, such as selecting users ranked higher for information push.

[0103] Taking activation as the conversion goal as an example, users are sorted by activation rate. Based on the needs of the target audience, users with the highest conversion rate are selected for advertising (e.g., the top 100% A, where A is selected based on actual usage needs). This group of people will perform better in terms of activation rate and will also have relatively better performance in terms of download rate.

[0104] The information push method provided in this application can be executed by an information push device. This application uses an information push device executing the information push method as an example to illustrate the information push device provided in this application.

[0105] In some embodiments of this application, an information push device is proposed, comprising: a receiving module for receiving input data, wherein the input data is associated with a user; a sending module for inputting the input data into a neural network trained by the neural network training method of any of the above embodiments, thereby obtaining multiple predicted probability values ​​output by the neural network; and a push module for pushing information to the user based on the multiple predicted probability values.

[0106] In this embodiment, information is pushed to users based on multiple predicted probability values ​​output by the neural network trained above, which realizes targeted information delivery, improves the conversion rate of delivered information, and ensures the effectiveness of information delivery.

[0107] In some embodiments of this application, users are sorted according to multiple predicted probability values, such as selecting users ranked higher for information push.

[0108] The neural network training device or information push device in this application embodiment can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit it.

[0109] The neural network training device or information push device in the embodiments of this application can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application embodiment does not specifically limit it.

[0110] In one embodiment, such as Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 702 and a memory 704. The memory 704 stores a program or instructions that can run on the processor 702. When the program or instructions are executed by the processor 702, they implement the various steps of the above-mentioned neural network training device or information push device embodiments and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0111] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0112] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0113] like Figure 8 As shown, the electronic device 800 includes, but is not limited to, components such as: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810. Those skilled in the art will understand that the electronic device 800 may also include a power supply (such as a battery) for powering the various components. The power supply can be logically connected to the processor 810 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0114] In one embodiment, the processor 810 is used to obtain a first predicted probability value predicted by the first neural network of two adjacent neural networks in M ​​neural networks when the training sample is used as input; the second neural network in the M neural networks uses the first predicted probability value as the initial value of the input weights to perform weighted processing on the training sample to obtain a weighted result; the second neural network obtains the calculation result based on the weighted result and a preset deviation value, and determines the prediction error value between the calculation result and the actual result of the training sample based on the calculation result and a preset loss function, wherein the loss function has a reward score; the reward score is updated according to the prediction error value; and the input weights and deviation value are updated according to the prediction error value and the updated reward score until the loss function converges, and the training of the second neural network ends.

[0115] In one embodiment, the processor 810 is used to determine the sum of the weighted result and the deviation value; the sum is used as input to the activation function in the second neural network to obtain the calculation result.

[0116] In one embodiment, the processor 810 is configured to differentiate the input weights and the deviation value based on the prediction error value and the updated reward score to obtain the gradient value of the input weights and the gradient value of the deviation value; update the input weights based on the gradient value of the input weights, the updated reward score, and the initial assignment of the input weights; and update the deviation value based on the gradient value of the deviation value, the updated reward score, and the initial assignment of the deviation value.

[0117] In one embodiment, the processor 810 is used to output a second predicted probability value of the second neural network trained with training samples as input.

[0118] In one embodiment, the first prediction probability value is the prediction probability value obtained by the first neural network making a prediction with the first transformation target; the second prediction probability value is the prediction probability value obtained by the second neural network making a prediction with the second transformation target; the second neural network also has a third prediction probability value, which is determined based on the first prediction probability value and the second prediction probability value.

[0119] In one embodiment, the processor 810 is configured to receive input data associated with a user; input the input data into a neural network trained by a neural network training method to obtain multiple predicted probability values ​​output by the neural network; and push information to the user based on the multiple predicted probability values.

[0120] In this embodiment, for a multi-task neural network, specifically M interconnected neural networks involved in this application, during neural network training, the first predicted probability value obtained by the first neural network in two adjacent neural networks when using training samples as input is used as the initial value of the input weights for the second neural network. This allows the first predicted probability value of the first neural network to participate in the training of the second neural network as an influencing factor. During this training process, the training of the second neural network takes into account the training results of other neural networks, improving the imbalance problem existing in trained neural networks between different transformation targets. This enables the trained multi-task neural network to balance multiple transformation targets, improving the accuracy of the trained neural network.

[0121] It should be understood that, in this embodiment, the input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042. The GPU 8041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0122] The memory 809 can be used to store software programs and various data. The memory 809 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 809 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 809 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0123] Processor 810 may include one or more processing units; optionally, processor 810 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 810.

[0124] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described neural network training method or information delivery method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0125] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0126] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described neural network training method or information push method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0127] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0128] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described neural network training method or information push method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the embodiments of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0131] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for training a neural network, characterized in that, Executed by a processor in an electronic device, the number of neural networks is M, where M is a positive integer greater than or equal to 2. The task chains of the M neural networks are interconnected. The tasks corresponding to the M neural networks include at least two of download, activation, registration, purchase, and retention. The download, activation, registration, and purchase occur sequentially, and the retention occurs after the download. The M neural networks are trained using the same sample training set, which includes training samples. The training method includes: Obtain the first prediction probability value predicted by the first neural network among two adjacent neural networks in the M neural networks when the training sample is used as input; The second neural network in the M neural networks uses the first predicted probability value as the initial value of the input weight to perform weighted processing on the training samples to obtain a weighted result. The second neural network obtains the calculation result based on the weighted result and the preset deviation value, and determines the prediction error value between the calculation result and the actual result of the training sample based on the calculation result and the preset loss function, wherein the loss function has a reward score; The reward score is updated based on the prediction error value; and the input weights and the deviation value are updated based on the prediction error value and the updated reward score, until the loss function converges, and the training of the second neural network ends. The second neural network outputs a second predicted probability value when the training sample is used as input. The second predicted probability value is used for information processing, which includes information recommendation. The training samples include one or more of the following: user attribute data, historical behavior data of the electronic devices used by the user, user behavior data on historical delivery information, and contextual environment data of the historical delivery information. The first neural network is any one of the download sub-network, activation sub-network, registration sub-network, purchase sub-network, and secondary retention sub-network, and the second neural network is the neural network adjacent to the first neural network among the download sub-network, activation sub-network, registration sub-network, purchase sub-network, and secondary retention sub-network.

2. The neural network training method according to claim 1, characterized in that, The second neural network obtains a calculation result based on the weighted result and a preset deviation value, including: Determine the sum of the weighted result and the deviation value; The sum is used as input to the activation function in the second neural network to obtain the calculation result.

3. The neural network training method according to claim 1, characterized in that, Based on the prediction error value and the updated reward score, update the input weights and the deviation value, including: Based on the prediction error value and the updated reward score, the gradient values ​​of the input weights and the deviation value are obtained by taking the derivatives of the input weights and the deviation value. The input weight is updated based on the gradient value of the input weight, the updated reward score, and the initial value of the input weight. The deviation value is updated based on the gradient value of the deviation value, the updated reward score, and the initial value of the deviation value.

4. The training method for a neural network according to any one of claims 1 to 3, characterized in that, The first predicted probability value is the predicted probability value obtained by the first neural network making a prediction with the first transformation target; The second prediction probability value is the prediction probability value obtained by the second neural network in making a prediction with the second transformation target. The second neural network also has a third prediction probability value, which is determined based on the first prediction probability value and the second prediction probability value.

5. A training device for a neural network, characterized in that, Applied to electronic devices, the number of neural networks is M, where M is a positive integer greater than or equal to 2. The task chains of the M neural networks are interconnected. The tasks corresponding to the M neural networks include at least two of download, activation, registration, purchase, and retention. The download, activation, registration, and purchase occur sequentially, and the retention occurs after the download. The M neural networks are trained using the same sample training set, which includes training samples. The training device for the neural networks includes: A module is used to obtain the first prediction probability value predicted by the first neural network in two adjacent neural networks among the M neural networks when the training sample is used as input. A weighting module is used in the second neural network of the M neural networks to perform weighting processing on the training samples by taking the first predicted probability value as the initial value of the input weight, and to obtain the weighted result. The computation module is used by the second neural network to obtain a computation result based on the weighted result and a preset deviation value, and to determine the prediction error value between the computation result and the actual result of the training sample based on the computation result and a preset loss function, wherein the loss function has a reward score; The module is updated to update the reward score based on the prediction error value; and the input weights and the deviation value are updated based on the prediction error value and the updated reward score, until the loss function converges and the training of the second neural network ends. The update module is further configured to output a second prediction probability value of the second neural network when the training sample is used as input. The second prediction probability value is used for information processing, which includes information recommendation. The training samples include one or more of the following: user attribute data, historical behavior data of the electronic devices used by the user, user behavior data on historical delivery information, and contextual environment data of the historical delivery information. The first neural network is any one of the download sub-network, activation sub-network, registration sub-network, purchase sub-network, and secondary retention sub-network, and the second neural network is the neural network adjacent to the first neural network among the download sub-network, activation sub-network, registration sub-network, purchase sub-network, and secondary retention sub-network.

6. The neural network training apparatus according to claim 5, characterized in that, The weighting module is used to: determine the sum of the weighting result and the deviation value; The sum is used as input to the activation function in the second neural network to obtain the calculation result.

7. The neural network training apparatus according to claim 5, characterized in that, The computing module is used for: Based on the prediction error value and the updated reward score, the gradient values ​​of the input weights and the deviation value are obtained by taking the derivatives of the input weights and the deviation value. The input weight is updated based on the gradient value of the input weight, the updated reward score, and the initial value of the input weight. The deviation value is updated based on the gradient value of the deviation value, the updated reward score, and the initial value of the deviation value.

8. The training apparatus for a neural network according to any one of claims 5 to 7, characterized in that, The first prediction probability value is the prediction probability value obtained by the first neural network predicting with the first transformation target; the second prediction probability value is the prediction probability value obtained by the second neural network predicting with the second transformation target; the second neural network also has a third prediction probability value, which is determined based on the first prediction probability value and the second prediction probability value.

9. An electronic device, characterized in that, include: A processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in any one of claims 1 to 4.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 4.

Images