Multi-task model training method, multi-task prediction method, related device and medium

By using the attention module to pass parameter weights in the multi-task model and combining task correlation information, the problem of correlation information loss in traditional multi-task prediction methods is solved, and the accuracy and correlation of prediction results are improved.

CN115049108BActive Publication Date: 2024-11-05ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210552195.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-20
Publication Date
2024-11-05
Estimated Expiration
2042-05-20

AI Technical Summary

Technical Problem

Traditional multi-task prediction methods tend to lose correlation information between tasks when processing complex tasks, resulting in insufficient accuracy of prediction results.

Method used

A multi-task model training method is adopted, in which each sub-model contains an attention module. By determining the parameter weights of sample features and target features, the parameters of the sub-model are adjusted to each other to retain the correlation information between tasks, thereby improving the accuracy of the prediction results. .

Benefits of technology

By conveying the parameter weight of the attention module in the multi-task model, it is possible to more accurately combine the correlation information of adjacent tasks, improve the accuracy and correlation of the final prediction results, and reduce data processing time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115049108B_ABST
    Figure CN115049108B_ABST
Patent Text Reader

Abstract

The embodiments of this specification disclose a multi-task model training method, a multi-task prediction method, a related device and a medium, and the training method may include: first determining the parameter weight of the sample feature in the attention module of the mth sub-model, determining the parameters in the attention module of the m+1th sub-model according to the parameter weight in the attention module of the mth sub-model, and training the multi-task model based on the parameter weight in the attention module of the mth sub-model and the parameters in the attention module of the m+1th sub-model. The multi-task model can correspond each task in the multi-task to a sub-model, and migrate the associated information of the previous task to the next task through the attention module between adjacent tasks, so that the next task can combine the associated information of the previous task to obtain a more accurate and more relevant prediction result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-task model training method, a multi-task prediction method, related devices and media. Background Art

[0002] Generally, the way to predict a task is to input the data corresponding to the task into a trained model. This type of model can be trained based on the learning pattern of the task data. When faced with complex tasks, it can be decomposed first and then multiple tasks can be trained separately, and the final prediction result can be obtained by combining the results corresponding to the multiple tasks.

[0003] Since multiple tasks may be correlated, the prediction results obtained by traditional prediction methods will lose the correlation information, so it is necessary to provide a technical solution with higher accuracy of prediction results. Summary of the invention

[0004] The embodiments of this specification provide a multi-task model training method, a multi-task prediction method, related devices and media, and the technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present specification provides a multi-task model training method, wherein the multi-task model includes M sub-models, each sub-model corresponds to a task, and each sub-model includes an attention module, and the method includes:

[0006] Determine the parameter weight of the sample feature in the attention module of the mth sub-model; where m is a positive integer less than M;

[0007] Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the sample features include sample user features, sample product features, and sample results of the user for the task corresponding to the m+1th sub-model;

[0008] The multi-task model is trained based on sample features, parameter weights in the attention module of the mth sub-model, and parameters in the attention module of the m+1th sub-model.

[0009] In a second aspect, an embodiment of the present specification provides a multi-task prediction method, which is applied to a multi-task model, wherein the multi-task model includes M sub-models, each sub-model corresponds to a task, and each sub-model includes an attention module, and the method includes:

[0010] Determine the parameter weight of the target feature in the attention module of the mth sub-model; where m is a positive integer less than M;

[0011] Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the target features include target user features and target product features;

[0012] Based on the target features and the parameters in the attention module of the m+1th sub-model, the prediction result of the corresponding task of the m+1th sub-model is obtained.

[0013] In a third aspect, an embodiment of the present specification provides a multi-task model training device, wherein the multi-task model includes M sub-models, each sub-model corresponds to a task, each sub-model includes an attention module, and the multi-task model training device includes:

[0014] The first processing module is used to determine the parameter weight of the sample feature in the attention module of the mth sub-model; wherein m is a positive integer less than M;

[0015] The second processing module is used to determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the sample features include sample user features, sample product features, and sample results of the user for the task corresponding to the m+1th sub-model;

[0016] A training module is used to train the multi-task model based on sample features, parameter weights in the attention module of the m-th sub-model, and parameters in the attention module of the m+1-th sub-model.

[0017] In a fourth aspect, an embodiment of the present specification also provides a multi-task model training device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned multi-task model training method steps.

[0018] In a fifth aspect, an embodiment of the present specification provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above-mentioned multi-task model training method steps.

[0019] In a sixth aspect, an embodiment of the present specification provides a multi-task prediction device, which is applied to a multi-task model, wherein the multi-task model includes M sub-models, each sub-model corresponds to a task, each sub-model includes an attention module, and the device includes:

[0020] The third processing module is used to determine the parameter weight of the target feature in the attention module of the mth sub-model; wherein m is a positive integer less than M;

[0021] A fourth processing module is used to determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the target features include target user features and target product features;

[0022] The prediction module is used to obtain the prediction result of the task corresponding to the m+1th sub-model based on the target features and the parameters in the attention module of the m+1th sub-model.

[0023] In a seventh aspect, an embodiment of the present specification also provides a multi-task prediction device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned multi-task prediction method steps.

[0024] In an eighth aspect, an embodiment of the present specification further provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above-mentioned multi-task prediction method steps.

[0025] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0026] In one or more embodiments of the present specification, when training a multi-task model, the parameter weights of the sample features in the attention module of the mth sub-model can be first determined, and the parameters in the attention module of the m+1th sub-model can be determined based on the parameter weights in the attention module of the mth sub-model, and the multi-task model can be trained based on the parameter weights in the attention module of the mth sub-model and the parameters in the attention module of the m+1th sub-model. The multi-task model can correspond each task in the multi-task to a sub-model, and migrate the associated information of the previous task to the next task through the attention module between adjacent tasks, so that the next task can combine the associated information of the previous task to obtain a more accurate and more relevant prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A schematic diagram of a prediction process of an existing multi-task model provided in an embodiment of this specification;

[0029] Figure 2 A schematic diagram of a prediction process of a multi-task model provided in an embodiment of this specification;

[0030] Figure 3 A flowchart of a multi-task model training method provided in an embodiment of this specification;

[0031] Figure 4 A schematic diagram of the structure of a sub-model provided in an embodiment of this specification;

[0032] Figure 5 A schematic diagram of the structure of a multi-task model provided in an embodiment of this specification;

[0033] Figure 6 A flowchart of a multi-task prediction method provided in an embodiment of this specification;

[0034] Figure 7 A schematic diagram of the structure of a multi-task model training device provided in an embodiment of this specification;

[0035] Figure 8 A schematic diagram of the structure of a multi-task prediction device provided in an embodiment of this specification;

[0036] Fig. 9 A schematic diagram of the structure of another multi-task model training device provided in an embodiment of this specification;

[0037] Fig.10 A schematic diagram of the structure of another multi-task prediction device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification.

[0039] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0040] When a user uses a general life application, it will provide the user with a recommendation service that can better improve the user's needs according to the user's actions. Here, taking a third-party application that can perform a payment function as an example, the user can browse and click on the relevant information of interest on the recommendation interface of the third-party application, and can pay to purchase the relevant information of interest, wherein the entire process of the user can be divided into multiple tasks, for example, when the user browses on the recommendation interface of the third-party application, it can correspond to the exposure task of the relevant information of interest to the user, when the user clicks on the relevant information of interest on the recommendation interface of the third-party application, it can correspond to the click task of the relevant information of interest to the user, and when the user pays to purchase the relevant information of interest on the third-party application, it can correspond to the purchase task of the relevant information of interest to the user. It can be understood that there is a sequential dependency between the exposure task, click task and purchase task mentioned above, that is, the click task can only occur after the exposure task occurs, and the purchase task can only occur after the click task occurs, and there is a correlation between adjacent tasks.

[0041] Of course, the number of multiple tasks corresponding to the user's action may not be limited to the three mentioned above. For example, for a third-party purchase application, when a user places an order in the third-party purchase application, he can first browse the third-party purchase application, then click on the items that meet the needs, then receive the discount coupons in the store corresponding to the items that meet the needs, then verify the items that meet the needs, and place an order for the items that meet the needs after verification. Among them, when the user browses the third-party purchase application, it can correspond to the exposure task of the items that meet the user's needs, when the user clicks on the items that meet the needs in the third-party purchase application, it can correspond to the clicking task of the items that meet the user's needs, when the user receives the discount coupons in the store corresponding to the items that meet the needs, it can correspond to the receiving coupon task of the items that meet the user's needs, when the user verifies the items that meet the needs according to the received discount coupons, it can correspond to the verification task of the items that meet the user's needs, and when the user places an order after the verification of the items that meet the needs, it can correspond to the placing task of the items that meet the user's needs.

[0042] Based on this, in order to obtain the prediction results of user needs, model learning can generally be used to train multiple tasks corresponding to the user needs, and the final prediction results can be obtained by combining the prediction results of each task.

[0043] Specifically, as a common way to predict multiple tasks, multiple tasks can be divided into multiple tasks first, and multiple tasks can be learned separately, and the prediction results corresponding to each of the multiple tasks can be combined to obtain the final prediction results of the multiple tasks. Among them, each task can be learned and trained through a task model, for example, the sample feature vector corresponding to each task can be input into the task model for training, so that the feature vector corresponding to each task is input into the trained task model to obtain the corresponding prediction results. It can be understood that a task model can be set for each task to ensure the accuracy of the prediction results of each task model. Here, the multi-task can be divided into three tasks A, B and C as an example. Inputting task A into the trained task model can obtain the corresponding prediction result a, inputting task B into the trained task model can obtain the corresponding prediction result b, and inputting task C into the trained task model can obtain the corresponding prediction result c. The final prediction result of the multi-task can be obtained by combining the prediction result a, the prediction result and the prediction result c, for example but not limited to a×b×c.

[0044] Also refer to Figure 1 The following is a schematic diagram of a prediction process of an existing multi-task model provided in an embodiment of this specification. Figure 1As shown, taking the multi-task model including two sub-models, and the multi-tasks corresponding to the event including the first task and the second task as an example, the prediction result of the first task can be expressed as the predicted probability of the event being clicked, the prediction result of the second task can be expressed as the predicted probability of the event being converted if it is clicked, and the prediction result of the multi-task can be expressed as the predicted probability of the event being clicked and then converted. Specifically, when predicting the occurrence probability of the multi-tasks corresponding to the event, the feature vector corresponding to the multi-task can be first input into the first model to obtain the first prediction result corresponding to the first task. Among them, the first model can be a sub-model of the multi-task model, which corresponds to the task model of the first task, and can be obtained by training the sample features of the known prediction results. Then the feature vector corresponding to the multi-task can be input into the second model to obtain the second prediction result corresponding to the second task. Among them, the second model can be a sub-model of the multi-task model, which corresponds to the task model of the second task, and can be obtained by training the sample features of the known prediction results. Further, the target prediction result of the multi-task can be obtained by combining the first prediction result corresponding to the first task and the second prediction result corresponding to the second task, for example, by multiplying the first prediction result corresponding to the first task by the second prediction result corresponding to the second task. It can be understood that the first prediction result corresponding to the first task and the second prediction result corresponding to the second task are not limited to the order of precedence, for example, the second prediction result corresponding to the second task can be obtained first, and then the first prediction result corresponding to the first task can be obtained, or the first prediction result corresponding to the first task and the second prediction result corresponding to the second task can be obtained at the same time.

[0045] It can be seen that the above-mentioned method of predicting multiple tasks does not take into account the strong correlation between the tasks. It only obtains the final prediction result with loss of related information through a simple method of scalar multiplication, which is easy to affect the accuracy of the final prediction result.

[0046] Next, in order to better solve the above technical problems, one or more embodiments of this specification will be explained.

[0047] See also Figure 2 , Figure 2 FIG. 2 shows a schematic diagram of a prediction process of a multi-task model provided in an embodiment of this specification. Figure 2As shown, the multi-task model may include two sub-models (which may be represented as a first model and a second model), and the corresponding events may include a first task and a second task. Here, the first model may correspond to predicting the probability of occurrence of the first task, which may be represented as the predicted probability of the event being clicked with reference to the prediction result of the first task mentioned above. Here, the second model may correspond to predicting the probability of occurrence of the second task, which may be represented as the predicted probability of the event being converted if clicked with reference to the prediction result of the second task mentioned above. The prediction results of the multi-tasks may be represented as the predicted probability of the event being clicked and then converted.

[0048] When predicting the probability of occurrence of multiple tasks corresponding to an event, the feature vector corresponding to the multiple tasks can be first input into the first model to obtain the parameter weights in the attention module of the first model. Wherein, the first model may include a first module and an attention module. Specifically, the feature vector corresponding to the multiple tasks can be first input into the first module to obtain the first conversion information, and then the first conversion information can be input into the attention module to obtain the parameter weights in the attention module of the first model. It can be understood that the parameter weights in the attention module of the first model here can be used to characterize the parameters corresponding to the association information between the first task and the second task, which can also correspond to partial parameters of all parameters in the attention module of the first model. Here, all parameters in the attention module of the first model can be expressed as A, B, C, D and E as an example. The parameter weights in the attention module of the first model can be, but are not limited to, expressed as (0, 1, 1, 0, 0), that is, the parameters corresponding to the association information between the first task and the second task include B and C.

[0049] The feature vector corresponding to the multi-task mentioned here may include, but is not limited to, user features and task product features, wherein user features may be understood as feature information of the target user to execute the event, such as the identity information of the target user, which may specifically include any at least one of the information such as user name, user category or user address. Here, taking the executed event applied to a certain purchase-type third-party application as an example, the feature information of the target user may be determined by querying the user information filled in by the target user. In addition to the user name, user category and user address mentioned above, the feature information of the target user may also include the browsing history or purchase history of the target user within a preset time, and may also record the redemption vouchers or product coupons that the user has received, but this embodiment is not limited to this.

[0050] Among them, the task product feature can be understood as the feature information used to characterize the product corresponding to the event, such as any at least one of the product name, product production information or product definition information of the event, and the product definition information can be understood as the information used to characterize the product function. Here, the executed event is applied to a third-party purchase application as an example. When a product to be purchased is determined, the feature information corresponding to the product can be queried in the store where the product is located, such as the product name, product production time and product function information mentioned above, and the preferential information corresponding to the product can also be recorded. This embodiment is not limited to this.

[0051] Further, after obtaining the parameter weights in the attention module of the first model, the parameter weights in the attention module of the first model can be transferred to the attention module of the second model to determine the parameters in the attention module of the second model according to the parameter weights in the attention module of the first model. Wherein, the second model may include a second module and an attention module, and the attention module of the second model may adjust its own parameters according to the parameter weights received in the attention module of the first model, so that the adjusted parameters in the attention module of the second model may have the association information between the first task and the second task. Here, taking the parameters in the attention module of the second model as B, D, E and F, and the parameters corresponding to the parameter weights in the attention module of the first model as B and C as an example, the parameters in the attention module of the second model after adjustment may be expressed as B, C, D, E and F. It can be understood that the structure of the attention module of the second model here may be consistent with the structure of the attention module of the first model, but the corresponding parameters are different.

[0052] It can also be understood that in the process of transferring the parameter weights in the attention module of the first model to the attention module of the second model, the transfer can be but not limited to being performed through a connection module. For example, the transfer process can be implemented through a fully connected layer to ensure the integrity of the parameter weights in the attention module of the first model during the transfer process.

[0053] Further, after determining the parameters in the attention module of the second model, the feature vector corresponding to the multi-task can be input into the second model, and the occurrence probability of the multi-task corresponding to the event can be predicted by the attention module of the second model. Wherein, referring to the second model mentioned above, including the second module and the attention module, specifically, the feature vector corresponding to the multi-task can be first input into the second module to obtain the second conversion information, and then the second conversion information can be input into the attention module with adjusted parameters to directly obtain the occurrence probability of the multi-task corresponding to the event. It can be understood that the prediction result obtained according to the second model is combined with the association information between the first task corresponding to the first model and the second task corresponding to the second model. Compared with first obtaining the prediction results of the tasks corresponding to each sub-model separately, and then obtaining the prediction results of the final multi-task by combining the prediction results of each corresponding task, the entire prediction process only needs to obtain a final prediction result, which not only reduces the data processing time, but also retains the association information between adjacent tasks in the final prediction result, so that the final prediction result is more accurate.

[0054] Of course, this embodiment can be but is not limited to a multi-task model including two sub-models. Here, the multi-task mentioned above may include exposure task, click task, coupon collection task, verification task and next task as an example. The multi-task model can also be set to include five sub-models (which can be represented as the first model, the second model, the third model, the fourth model and the fifth model respectively), wherein the first model can correspond to the predicted probability of occurrence of the exposure task, the second model can correspond to the predicted probability of occurrence of the click task, the third model can correspond to the probability of occurrence of the coupon collection task, the fourth model can correspond to the probability of occurrence of the verification task, and the fifth model can correspond to the probability of occurrence of the next task, and each model can include an attention module. Specifically, the feature vector corresponding to the multi-task can be first input into the first model to obtain the parameter weight in the attention module of the first model, and the parameter weight in the attention module of the first model is transferred to the attention module of the second model to adjust the parameters in the attention module of the second model. Further, the feature vector corresponding to the multi-task can be input into the second model again to obtain the parameter weight in the attention module of the second model, and the parameter weight in the attention module of the second model is transferred to the attention module of the third model to adjust the parameters in the attention module of the third model. Further, the feature vector corresponding to the multi-task can be input into the third model again to obtain the parameter weight in the attention module of the third model, and the parameter weight in the attention module of the third model is transferred to the attention module of the fourth model to adjust the parameters in the attention module of the fourth model. Further, the feature vector corresponding to the multi-task can be input into the fourth model to obtain the parameter weight in the attention module of the fourth model, and the parameter weight in the attention module of the fourth model is transferred to the attention module of the fifth model to adjust the parameters in the attention module of the fifth model. Further, the feature vector corresponding to the multi-task can be input into the fifth model, and the final prediction result can be obtained through the attention module of the fifth model.

[0055] See also Figure 3 , Figure 3 A flowchart of a multi-task model training method provided in an embodiment of this specification is shown.

[0056] like Figure 3 As shown, the multi-task model training method may at least include the following steps:

[0057] Step 302: Determine the parameter weight of the sample feature in the attention module of the mth sub-model.

[0058] When the probability of occurrence of multiple tasks needs to be predicted, the target features corresponding to the multiple tasks can be input into the trained multi-task model mentioned in this embodiment to obtain a more accurate prediction result. Based on this, the process of training the multi-task model mentioned in this embodiment is particularly important.

[0059] Specifically, before training the multi-task model, the number of tasks in the multi-task corresponding to the sample features and the sample results corresponding to each task can be determined first. Among them, the sample features may include sample user features, sample product features, and sample results of users for each task. The sample user features can be understood as sample feature information of the user who is to perform the event, such as the sample identity information of the user, and may specifically include any at least one information such as user name, user classification, or user address. The sample product features can be understood as sample feature information used to characterize the product corresponding to the event, such as any at least one information such as the sample product name of the event, sample product production information, or sample product definition information. The sample product definition information can be understood as information used to characterize the product function. It can be understood that the demand for the sample product can correspond to multiple tasks, and each task can correspond to a known sample result. The sample result can be, but is not limited to, when it is determined that the corresponding task does not occur, it can be represented as character 0, and when it is determined that the corresponding task has occurred, it can be represented as character 1.

[0060] After determining the number of tasks in the multi-task corresponding to the sample feature and the sample results corresponding to each task, the parameter weight in the attention module of the model corresponding to the mth task in the multi-task can be obtained according to the sample user feature and the sample product feature. Among them, the multi-task model of this embodiment may include M sub-models, each sub-model may include an attention module, each task in the multi-task corresponding to the sample feature may correspond to a sub-model, and adjacent tasks may correspond to two adjacent sub-models. For example, the first task in the multi-task corresponding to the sample feature may correspond to the first sub-model in the multi-task model, the second task in the multi-task corresponding to the sample feature may correspond to the second sub-model in the multi-task model, and the mth task in the multi-task corresponding to the sample feature may correspond to the mth sub-model in the multi-task model, that is, the parameter weight in the attention module of the model corresponding to the mth task in the multi-task mentioned above can be understood as the parameter weight in the attention module of the mth sub-model in the multi-task model. Here, M can be a positive integer greater than m, for example, when m is 2, M can be a positive integer greater than 2.

[0061] It can be understood that the mth task in the multi-task corresponding to the sample features mentioned above can correspond to the mth sub-model in the multi-task model, and the parameter weights in the attention module in the mth sub-model can be used to characterize the parameters corresponding to the association information between the mth task and the m+1th task, which can also correspond to partial parameters of all parameters in the attention module of the sub-model corresponding to the mth task. Here, taking the example that all parameters in the attention module of the sub-model corresponding to the mth task can be expressed as A, B, C, D and E, the parameter weights in the attention module of the sub-model corresponding to the mth task can be, but are not limited to, expressed as (0, 1, 1, 0, 0), that is, the parameters corresponding to the association information between the mth task and the m+1th task include B and C.

[0062] It can also be understood that the task corresponding to the mth sub-model in this embodiment can be any task from the first task to the second to last task in the multi-tasks corresponding to the sample features. Possibly, when the number of multi-tasks corresponding to the sample features is 5, the value of m can be 1, 2, 3 or 4. Possibly, when the number of multi-tasks corresponding to the sample features is 2, the value of m can only be 1.

[0063] Step 304: Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model.

[0064] Specifically, after determining the parameter weights in the attention module of the m-th sub-model, the parameter weights in the attention module of the m-th sub-model can be transferred to the attention module of the adjacent m+1-th sub-model to adjust the parameters in the attention module of the m+1-th sub-model. Here, taking the example that the parameters in the attention module of the m+1-th model can be expressed as B, D, E and F, and the parameters corresponding to the parameter weights in the attention module of the m-th model can be expressed as B and C, the parameters in the attention module of the m+1-th model after adjustment can be expressed as B, C, D, E and F.

[0065] It can be understood that the attention module included in each sub-model in the multi-task model of this embodiment can adopt the same structure, the difference is that the parameters in the attention module included in each sub-model are different.

[0066] Step 306: Train the multi-task model based on the sample features, the parameter weights in the attention module of the mth sub-model, and the parameters in the attention module of the m+1th sub-model.

[0067] Specifically, after determining the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model, the sample user features and sample product features in the sample features can be input into the m+1th sub-model, and the occurrence probability of the m+1th task can be obtained through the attention module of the m+1th sub-model. It can be understood that the occurrence probability of the m+1th task obtained in this way contains the correlation information between the first m tasks, and is more accurate than the occurrence probability of the m+1th task obtained in the prior art.

[0068] Furthermore, after obtaining the occurrence probability of the m+1th task, the m+1th sub-model and the m+1th sub-model of the multi-task model can be trained for parameter optimization in combination with the sample result of the m+1th task in the sample feature, so that the occurrence probability of the m+1th task is closer to the sample result of the m+1th task in the sample feature. It is understandable that in the training process of the multi-task model, as the parameters in the m+1th sub-model change, the parameter weights in the attention module of the m+1th sub-model will also change, and then the parameters in the attention module of the m+1th sub-model will also change, which can cause the occurrence probability of the m+1th task to be closer to the sample result of the m+1th task in the sample feature.

[0069] It should be noted that when m is any positive integer greater than 1, the parameter weights in the attention module of each corresponding sub-model can be obtained in sequence according to the arrangement order of each task, and then the parameters in the attention module of the m+1th sub-model can be determined according to the parameter weights in the attention module of the mth sub-model, and all m+1 sub-models can be trained during the training process to improve the training efficiency. For example, when m is 3, the parameter weights in the attention module of the first sub-model, the parameter weights in the attention module of the second sub-model, and the parameter weights in the attention module of the third sub-model can be determined in sequence, and then the parameters in the attention module of the fourth sub-model can be determined according to the parameter weights in the attention module of the third sub-model, and the occurrence probability of the fourth task can be obtained, and the four sub-models can be trained for parameter optimization according to the occurrence probability of the fourth task and the sample results corresponding to the fourth task.

[0070] In an embodiment of the present specification, the multi-task model may correspond each task in the multi-task to a sub-model, and migrate the relevant information of the previous task to the next task through the attention module between adjacent tasks, so that the next task can combine the relevant information of the previous task to obtain a more accurate and more relevant prediction result.

[0071] As an option of this embodiment, the mth sub-model further includes an embedding module, a combination module and a first conversion module;

[0072] Determine the parameter weights of the sample features in the attention module of the mth sub-model, including:

[0073] Inputting sample user features and sample product features into an embedding module to obtain feature vectors corresponding to the sample user features and sample product features respectively;

[0074] Inputting the feature vectors corresponding to the sample user features and the sample product features respectively into the combination module to obtain a combined feature vector;

[0075] Inputting the combined feature vector into a first conversion module to obtain first conversion information;

[0076] The first conversion information is input into the attention module of the mth sub-model to obtain the parameter weight of the sample feature in the mth sub-model.

[0077] Specifically, see here Figure 4 The structure diagram of a sub-model provided in the embodiment of this specification is shown. Figure 4As shown, this structural schematic diagram can be used to represent the structural schematic diagram of the m-th sub-model in this embodiment. The m-th sub-model may sequentially include an embedding module, a combination module, a first transformation module, and an attention module in the connection order. Among them, the embedding module can be used to input the sample user feature and the sample product feature in the sample feature, and obtain the feature vectors corresponding to the sample user feature and the sample product feature respectively. It can be understood that the sample user feature here can be expressed as the identity feature of the sample user, such as but not limited to including the user name, user classification, and user address, etc. The sample product feature can be expressed as the sample feature information used to characterize the product corresponding to the event, such as but not limited to including the sample product name of the event, the sample product production information, and the sample product definition information. The sample product definition information can be understood as the information used to characterize the product function. The embedding module in this embodiment can be but not limited to the embedding layer (i.e., the Embedding layer) in the speech processing technology. In this embedding module, a corpus containing a dictionary and characters can be preset, and each character can correspond to a Chinese character in the dictionary. For example, the Chinese character "你" can correspond to the character 1, and the Chinese character "好" can correspond to the character 3. When the sample user feature and the sample product feature are input into this embedding module, each Chinese character in the sample user feature and the sample product feature can be converted into the corresponding character according to the corpus, and the feature vectors corresponding to the sample user feature and the sample product feature can be output to the combination module in the form of a matrix. It can also be understood that the order of inputting the sample user feature and the sample product feature here is not limited. Possibly, the sample user feature can be input into the embedding module first to obtain the feature vector corresponding to the sample user feature, and then the sample product feature can be input into the embedding module to obtain the feature vector corresponding to the sample product feature. Possibly, the sample product feature can be input into the embedding module first to obtain the feature vector corresponding to the sample product feature, and then the sample user feature can be input into the embedding module to obtain the feature vector corresponding to the sample user feature. Possibly, the sample user feature and the sample product feature can be input into the embedding module at the same time, and the feature vector corresponding to the sample user feature and the feature vector corresponding to the sample product feature can be obtained at the same time.

[0078] Further, after the feature vectors corresponding to the sample user feature and the sample product feature are input into the combination module, the combination module can output the combined feature vector. Among them, the combined feature vector can be understood as the combination of the feature vector corresponding to the sample user feature and the feature vector corresponding to the sample product feature. Here, taking the feature vector corresponding to the sample user feature can be expressed as [A, B], and the feature vector corresponding to the sample product feature can be expressed as [C, D, E] as an example, the combined feature vector obtained through the combination module can be expressed as [A, B, C, D, E].

[0079] Furthermore, after obtaining the combined feature vector according to the combination module, the combined feature vector can be input into the first conversion module to obtain the first conversion information. The first conversion module can be understood as a neural network corresponding to the mth task, which is used to obtain vector information that can characterize the corresponding features of the mth task according to the combined feature vector. It can be understood that the first conversion module can be but is not limited to any deep learning neural network such as deep neural networks (DNN), deep interest learning networks (DIN) or deep recommendation networks (Factorization-Machinebased Neural Network, DeepFM).

[0080] Further, after obtaining the first conversion information according to the first conversion module, the first conversion information can be input into the attention module to learn the parameter weights of the sample features in the mth sub-model. The attention module can learn the vector information of the features corresponding to the association information between the mth task and the m+1th task in the first conversion information to obtain the parameter weights corresponding to the vector information of the features corresponding to the association information between the mth task and the m+1th task. It can be understood that the attention module here can be but is not limited to the attention mechanism (ie, attention mechanism) in the application of common technology, and the present embodiment is not limited thereto.

[0081] As another option of this embodiment, the (m+1)th sub-model further includes an embedding module, a combining module and a second conversion module;

[0082] The multi-task model is trained based on the sample features, the parameter weights in the attention module of the m-th sub-model, and the parameters in the attention module of the m+1-th sub-model, including:

[0083] Inputting sample user features and sample product features into an embedding module to obtain feature vectors corresponding to the sample user features and sample product features respectively;

[0084] Inputting the feature vectors corresponding to the sample user features and the sample product features respectively into the combination module to obtain a combined feature vector;

[0085] Inputting the combined feature vector into a second conversion module to obtain second conversion information;

[0086] Input the second conversion information into the attention module of the m+1th sub-model, and obtain the prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters in the attention module of the m+1th sub-model;

[0087] The multi-task model is trained according to the prediction results of the task corresponding to the m+1th sub-model and the sample results of the user for the task corresponding to the m+1th sub-model.

[0088] Specifically, see here Figure 5 The structure diagram of a multi-task model provided by the embodiment of this specification is shown in FIG. Figure 5 As shown, the multi-task model may include the mth sub-model and the m+1th sub-model, the mth sub-model may correspond to the mth task in the multi-tasks corresponding to the sample features, and the m+1th sub-model may correspond to the m+1th task in the multi-tasks corresponding to the sample features. Among them, the mth sub-model may include an embedding module, a combination module, a first conversion module and an attention module in the order of connection, and the m+1th sub-model may include an embedding module, a combination module, a second conversion module and an attention module in the order of connection. It can be understood that the embedding module in the mth sub-model and the embedding module in the m+1th sub-model here can be the same embedding module (it can also be understood as sharing the same embedding module), and this method can effectively retain the association information between the mth task and the m+1th task. Similarly, the combination module in the mth sub-model here can also be the same combination module as the combination module in the m+1th sub-model to ensure the reliability of the association information between the mth task and the m+1th task.

[0089] In the (m + 1)-th sub-model, the embedding module can be used to input the sample user features and sample product features in the sample feature, and obtain the feature vectors corresponding to the sample user features and sample product features respectively. It can be understood that the sample user features here can be expressed as the identity features of the sample user, such as but not limited to including the user name, user classification, and user address, etc. The sample product features can be expressed as the sample feature information used to characterize the product corresponding to the event, such as but not limited to including the sample product name of the event, sample product production information, and sample product definition information. The sample product definition information can be understood as the information used to characterize the product function. The embedding module in this embodiment can be but not limited to the embedding layer (i.e., the Embedding layer) in speech processing technology. In this embedding module, a corpus containing a dictionary and characters can be preset, and each character can correspond to a Chinese character in the dictionary. For example, the Chinese character "你" can correspond to character 1, and the Chinese character "好" can correspond to character 3. When the sample user features and sample product features are input into this embedding module, each Chinese character in the sample user features and sample product features can be converted into the corresponding character according to the corpus, and the feature vectors corresponding to the sample user features and sample product features respectively can be output to the combination module in the form of a matrix. It can also be understood that the order of inputting the sample user features and sample product features here is not limited. Possibly, the sample user features can be input into the embedding module first to obtain the feature vector corresponding to the sample user features, and then the sample product features can be input into the embedding module to obtain the feature vector corresponding to the sample product features. Possibly, the sample product features can be input into the embedding module first to obtain the feature vector corresponding to the sample product features, and then the sample user features can be input into the embedding module to obtain the feature vector corresponding to the sample user features. Possibly, the sample user features and sample product features can be input into the embedding module simultaneously, and the feature vector corresponding to the sample user features and the feature vector corresponding to the sample product features can be obtained simultaneously.

[0090] Further, after the feature vectors corresponding to the sample user features and sample product features are input into the combination module, the combination module can output the combined feature vector. Among them, the combined feature vector can be understood as the combination of the feature vector corresponding to the sample user features and the feature vector corresponding to the sample product features. Here, taking the feature vector corresponding to the sample user features as [A, B] and the feature vector corresponding to the sample product features as [C, D, E] as an example, the combined feature vector obtained through the combination module can be expressed as [A, B, C, D, E].

[0091] Furthermore, after obtaining the combined feature vector according to the combination module, the combined feature vector can be input into the second conversion module to obtain the second conversion information. The second conversion module can be understood as a neural network corresponding to the m+1th task, which is used to obtain vector information that can characterize the corresponding features of the m+1th task according to the combined feature vector. It can be understood that the second conversion module can be but is not limited to any deep learning neural network such as deep neural networks (DNN), deep interest learning networks (DIN) or deep recommendation networks (Factorization-Machinebased Neural Network, DeepFM).

[0092] Furthermore, after obtaining the second conversion information according to the second conversion module, the second conversion information can be input into the attention module, and the prediction result of the m+1th task can be obtained according to the parameters adjusted in the attention module and the activation function sigmoid in the attention module. It can be understood that the parameters of the attention module in the m+1th sub-model can include the association information with the mth task, and the occurrence probability of the m+1th task obtained by the attention module in the m+1th sub-model has the association with the mth task, thereby making the prediction result more reliable and accurate compared with the common technology.

[0093] It can also be understood that a fully connected module can be provided between the attention module in the m+1th sub-model and the attention module in the mth sub-model. After determining the parameter weights in the attention module of the mth sub-model, the parameter weights in the attention module of the mth sub-model can be input into the fully connected module to obtain connection information containing the parameter weights in the attention module of the mth sub-model. It should be noted here that the fully connected module in this embodiment can be used to retain the parameter weights in the attention module of the mth sub-model and transfer the parameter weights in the attention module of the mth sub-model, which can effectively ensure the integrity of the parameter weights in the attention module of the mth sub-model. After the fully connected module obtains the connection information containing the parameter weights in the attention module of the mth sub-model, the fully connected module can input the connection information containing the parameter weights in the attention module of the m+1th sub-model into the attention module of the m+1th sub-model, so that the parameters in the attention module of the m+1th sub-model are adjusted in combination with the parameter weights in the attention module of the mth sub-model.

[0094] Further, after obtaining the prediction result of the task corresponding to the m+1th sub-model, the parameters in the multi-task model can be optimized and trained by combining the prediction result of the task corresponding to the m+1th sub-model with the user's sample result for the task corresponding to the m+1th sub-model, until the prediction result of the task corresponding to the m+1th sub-model is closer to the user's sample result for the task corresponding to the m+1th sub-model. It can be understood that in this embodiment, the multi-task model can be trained by the prediction result and sample result of the m+1th sub-model, but is not limited to it. For example, the multi-task model can also be trained by the prediction result and sample result of the mth sub-model, but is not limited to it.

[0095] It should be noted that the multi-task model mentioned above may include but is not limited to only the mth sub-model and the m+1th sub-model, for example, it may also include the first sub-model, the second sub-model...the m+2th sub-model, the m+3th sub-model, etc. Each sub-model may correspond to a task in the multi-tasks corresponding to the sample features, and each sub-model may include an embedding module, a combination module, a corresponding transformation module and an attention module in sequence according to the connection order. It is understandable that a fully connected module may be provided between the attention modules of any two adjacent sub-models to facilitate the transfer of the parameter weights in the attention module of the previous sub-model to the attention module of the next sub-model.

[0096] As another option of this embodiment, after obtaining the prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters of the m+1th sub-model, the method further includes:

[0097] According to the prediction results of the task corresponding to the m+1th sub-model and the sample characteristics, the first loss function is obtained;

[0098] The m+1th sub-model is optimized based on the first loss function.

[0099] Specifically, in the process of training the multi-task model, after obtaining the prediction result of the task corresponding to the m+1th sub-model, the parameters of the m+1th sub-model can be optimized by calculating the cross entropy loss function (corresponding to the first loss function mentioned above). The calculation method of the cross entropy loss function is shown in the following formula (1):

[0100]

[0101] In formula (1), L ce It can be expressed as the cross entropy loss function of the m+1th sub-model, θ is the parameter of the m+1th model, N is the number of sample features, D is the sample set, (x, y t) is a set of samples in the sample set D, x can be expressed as the sample user features and sample product features corresponding to the m+1th task in the sample, y t It can be expressed as the sample result corresponding to the user's m+1th task, It can be expressed as the prediction result of the task corresponding to the m+1th sub-model. Here y t The value of can be but is not limited to 1 or 0, 0 can represent that the m+1th task is not completed, and 1 can represent that the m+1th task is completed.

[0102] As another option of this embodiment, after obtaining the first loss function according to the prediction result of the task corresponding to the m+1th sub-model and the sample characteristics, before optimizing the m+1th sub-model based on the first loss function, the method further includes:

[0103] Determine whether the prediction result of the task corresponding to the m+1th sub-model meets the preset conditions;

[0104] When it is determined that the prediction result of the task corresponding to the m+1th sub-model does not meet the preset condition, obtaining the prediction result of the task corresponding to the mth sub-model according to the first conversion information and the parameters in the attention module of the mth sub-model;

[0105] A second loss function is obtained according to the prediction result of the task corresponding to the m-th sub-model, the prediction result of the task corresponding to the m+1-th sub-model, and the sample characteristics;

[0106] The m+1th sub-model is optimized based on the first loss function, including:

[0107] The m+1th sub-model is optimized based on the first loss function and the second loss function.

[0108] According to the actual situation, only when the mth task is completed can the m+1th task be completed, that is, the prediction result corresponding to the mth task is higher than the prediction result corresponding to the m+1th task. Based on this, we can first determine whether the prediction result corresponding to the m+1th task meets the requirements, and then optimize the parameters of the m+1th sub-model according to the determination result.

[0109] Specifically, after calculating the cross entropy loss function of the m+1th sub-model, the prediction result of the task corresponding to the m+1th sub-model can be compared with the preset result, and when it is determined that the prediction result of the task corresponding to the m+1th sub-model is higher than the preset result, the prediction result of the mth task is first obtained according to the parameters in the attention module of the mth sub-model and the activation function sigmoid in the attention module, and then the prediction result of the task corresponding to the mth sub-model, the prediction result of the task corresponding to the m+1th sub-model and the sample features are combined to obtain the probability calibration loss function (that is, the second loss function mentioned above). Before obtaining the prediction result of the mth task, the parameters in the attention module of the m-1th sub-model can be determined according to the parameter weights in the attention module of the m-1th sub-model, and then the sample user features and the sample product features are input into the mth sub-model, and the prediction result of the mth task is obtained through the activation function sigmoid in the attention module through the embedding module, the combination module, the first conversion module and the attention module in sequence.

[0110] It is understandable that the preset result mentioned above can be determined based on the sample result corresponding to the user in the mth task, for example but not limited to, the preset result can be set to be smaller than the sample result corresponding to the user in the mth task.

[0111] The calculation method of the probability calibration loss function here can be obtained by the following formula (2):

[0112]

[0113] In formula (2), L le It can be expressed as the probability calibration loss function of the m+1th sub-model, It can be expressed as the prediction result of the task corresponding to the mth sub-model, It can be expressed as the prediction result of the task corresponding to the m+1th sub-model. Greater than When ∫ ∫ ∫ ∫ ∫ (that is, the prediction result of the task corresponding to the m+1th sub-model is greater than the prediction result of the task corresponding to the mth sub-model), it indicates that the prediction result of the task corresponding to the m+1th sub-model does not conform to the actual situation, and the probability calibration loss function needs to be introduced to calibrate the multi-task model.

[0114] For ease of understanding, when optimizing the m+1th sub-model based on the first loss function and the second loss function, the target loss function of the m+1th sub-model can be determined based on the first loss function and the second loss function, and then the parameters of the m+1th sub-model can be optimized using the target loss function. The target loss function of the m+1th sub-model can be calculated using the following formula (3):

[0115] L(θ)=L ce (θ)+αL le (θ) (3)

[0116] In formula (3), L(θ) can be expressed as the target loss function of the m+1th sub-model, L ce (θ) can be expressed as the cross entropy loss function of the m+1th sub-model, L le (θ) can be expressed as the probability calibration loss function of the m+1th sub-model, and α can be expressed as the calibration parameter. The larger the α, the greater the weight of the probability calibration loss function of the m+1th sub-model.

[0117] It can be understood that the above formula (3) can also be used to optimize the parameters of any other sub-model in the multi-task model, and this embodiment is not limited thereto.

[0118] It should be noted that in order to further improve the parameter optimization effect of the m+1th sub-model of the multi-task model, a mean square error module can be added to the mth sub-model and the m+1th sub-model respectively. When the sample user characteristics and the sample product characteristics pass through the embedding module, they can be abstracted into a mapping function. The mean square error loss function of the m+1th sub-model is constructed based on the mapping function of the mth sub-model and the mapping function of the m+1th sub-model, respectively, and the parameters of the m+1th sub-model are optimized in combination with the above-mentioned cross entropy loss function, probability calibration loss function and the mean square error loss function.

[0119] Among them, the mean square error loss function of the m+1th sub-model can be calculated by the following formula (4):

[0120]

[0121] In formula (4), L mse It can be expressed as the mean square error loss function of the m+1th sub-model, f t (x i ) can be expressed as the mapping function of the m+1th sub-model, f t-1 (x i ) can be expressed as the mapping function of the mth sub-model.

[0122] Here, we can combine the above formula (3) and the mean square error loss function to obtain the new target loss function of the m+1th sub-model through the following formula (5):

[0123] L(θ)=L ce (θ)+αL le (θ)+γL mse (5)

[0124] In formula (5), L(θ) can be expressed as the target loss function of the m+1th sub-model, L ce (θ) can be expressed as the cross entropy loss function of the m+1th sub-model, L le (θ) can be expressed as the probability calibration loss function of the m+1th sub-model, L mse It can be expressed as the mean square error loss function of the m+1th sub-model, α can be expressed as a calibration parameter, γ can be expressed as an error parameter, and γ can be any natural number.

[0125] As another option of this embodiment, after obtaining the prediction result of the task corresponding to the m-th sub-model according to the first conversion information and the parameter weight of the m-th sub-model, the method further includes:

[0126] According to the prediction results of the task corresponding to the m-th sub-model and the sample characteristics, the third loss function is obtained;

[0127] The mth sub-model is optimized based on the third loss function.

[0128] During the training of the multi-task model, in addition to optimizing the parameters of the m+1th sub-model, you can also optimize the parameters of any sub-model in the multi-task model to further ensure the accuracy of the prediction results of the multi-task model.

[0129] Specifically, after obtaining the prediction result of the task corresponding to the mth sub-model, the parameters of the mth sub-model can be optimized by calculating the cross entropy loss function (corresponding to the third loss function mentioned above), wherein the calculation method of the cross entropy loss function can refer to the above formula (1), which will not be elaborated here.

[0130] See also Figure 6 , Figure 6 A flowchart of a multi-task prediction method provided in an embodiment of this specification is shown.

[0131] like Figure 6 As shown, the multi-task prediction method may at least include the following steps:

[0132] Step 602: Determine the parameter weight of the target feature in the attention module of the mth sub-model.

[0133] Specifically, the target features may include target user features and target product features, wherein the target user features may be understood as target feature information of the user who is to execute the event, such as the target identity information of the user, and may specifically include any at least one of the user name, user category, or user address. The target product features may be understood as target feature information used to characterize the product corresponding to the event, such as any at least one of the target product name of the event, target product production information, or target product definition information, and the target product definition information may be understood as information used to characterize the product function.

[0134] After first determining the target user features and the target product features, the parameter weights in the attention module of the model corresponding to the mth task in the multi-task can be obtained according to the target user features and the target product features. Among them, the multi-task model of this embodiment may include M sub-models, each sub-model may include an attention module, each task in the multi-task corresponding to the target feature may correspond to a sub-model, and adjacent tasks may correspond to two adjacent sub-models. For example, the first task in the multi-task corresponding to the target feature may correspond to the first sub-model in the multi-task model, the second task in the multi-task corresponding to the target feature may correspond to the second sub-model in the multi-task model, and the mth task in the multi-task corresponding to the target feature may correspond to the mth sub-model in the multi-task model, that is, the parameter weights in the attention module of the model corresponding to the mth task in the multi-task mentioned above can be understood as the parameter weights in the attention module of the mth sub-model in the multi-task model. Here, M can be a positive integer greater than m, for example, when m is 2, M can be a positive integer greater than 2.

[0135] It can be understood that the mth task in the multi-task corresponding to the target features mentioned above can correspond to the mth sub-model in the multi-task model, and the parameter weights in the attention module in the mth sub-model can be used to characterize the parameters corresponding to the association information between the mth task and the m+1th task, which can also correspond to partial parameters of all parameters in the attention module of the sub-model corresponding to the mth task. Here, taking the example that all parameters in the attention module of the sub-model corresponding to the mth task can be expressed as A, B, C, D and E, the parameter weights in the attention module of the sub-model corresponding to the mth task can be, but are not limited to, expressed as (0, 1, 1, 0, 0), that is, the parameters corresponding to the association information between the mth task and the m+1th task include B and C.

[0136] It can also be understood that the task corresponding to the mth sub-model in this embodiment can be any task from the first task to the second to last task in the multi-tasks corresponding to the target feature. Possibly, when the number of multi-tasks corresponding to the target feature is 5, the value of m can be 1, 2, 3 or 4. Possibly, when the number of multi-tasks corresponding to the target feature is 2, the value of m can only be 1.

[0137] Step 604: Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model.

[0138] Specifically, after determining the parameter weights in the attention module of the m-th sub-model, the parameter weights in the attention module of the m-th sub-model can be transferred to the attention module of the adjacent m+1-th sub-model to adjust the parameters in the attention module of the m+1-th sub-model. Here, taking the example that the parameters in the attention module of the m+1-th model can be expressed as B, D, E and F, and the parameters corresponding to the parameter weights in the attention module of the m-th model can be expressed as B and C, the parameters in the attention module of the m+1-th model after adjustment can be expressed as B, C, D, E and F.

[0139] It can be understood that the attention module included in each sub-model in the multi-task model of this embodiment can adopt the same structure, the difference is that the parameters in the attention module included in each sub-model are different.

[0140] Step 606: Based on the target features and the parameters in the attention module of the m+1th sub-model, obtain the prediction result of the task corresponding to the m+1th sub-model.

[0141] In this embodiment, the m-th sub-model may include an embedding module, a combination module, a first conversion module and an attention module in the order of connection, and the m+1-th sub-model may include an embedding module, a combination module, a second conversion module and an attention module in the order of connection. It can be understood that the embedding module in the m-th sub-model and the embedding module in the m+1-th sub-model here can be the same embedding module (it can also be understood as sharing the same embedding module), and this method can effectively retain the association information between the m-th task and the m+1-th task. Similarly, the combination module in the m-th sub-model here can also be the same combination module as the combination module in the m+1-th sub-model to ensure the reliability of the association information between the m-th task and the m+1-th task.

[0142] Specifically, in the (m + 1)-th sub-model, the embedding module can be used to input the target user feature and the target product feature in the target features, and obtain the feature vectors corresponding to the target user feature and the target product feature respectively. It can be understood that the target user feature here can represent the identity feature of the target user, such as but not limited to including the user name, user classification, and user address, etc. The target product feature can represent the target feature information used to characterize the product corresponding to the event, such as but not limited to including the target product name of the event, the target product production information, and the target product definition information. The target product definition information can be understood as the information used to characterize the product function. The embedding module in this embodiment can be but not limited to the embedding layer (i.e., the Embedding layer) in the speech processing technology. In this embedding module, a corpus containing a dictionary and characters can be preset, and each character can correspond to a Chinese character in the dictionary. For example, the Chinese character "你" can correspond to the character 1, and the Chinese character "好" can correspond to the character 3. When the target user feature and the target product feature are input into this embedding module, each Chinese character in the target user feature and the target product feature can be converted into the corresponding character according to the corpus, and the feature vectors corresponding to the target user feature and the target product feature respectively can be output to the combination module in the form of a matrix. It can also be understood that the order of inputting the target user feature and the target product feature is not limited here. Possibly, the target user feature can be input into the embedding module first to obtain the feature vector corresponding to the target user feature, and then the target product feature can be input into the embedding module to obtain the feature vector corresponding to the target product feature. Possibly, the target product feature can be input into the embedding module first to obtain the feature vector corresponding to the target product feature, and then the target user feature can be input into the embedding module to obtain the feature vector corresponding to the target user feature. Possibly, the target user feature and the target product feature can be input into the embedding module simultaneously, and the feature vector corresponding to the target user feature and the feature vector corresponding to the target product feature can be obtained simultaneously.

[0143] Further, after the feature vectors corresponding to the target user feature and the target product feature are input into the combination module, the combination module can output the combined feature vector. Among them, the combined feature vector can be understood as the combination of the feature vector corresponding to the target user feature and the feature vector corresponding to the target product feature. Here, taking the feature vector corresponding to the target user feature as [A, B] and the feature vector corresponding to the target product feature as [C, D, E] as an example, the combined feature vector obtained through the combination module can be expressed as [A, B, C, D, E].

[0144] Furthermore, after obtaining the combined feature vector according to the combination module, the combined feature vector can be input into the second conversion module to obtain the second conversion information. The second conversion module can be understood as a neural network corresponding to the m+1th task, which is used to obtain vector information that can characterize the corresponding features of the m+1th task according to the combined feature vector. It can be understood that the second conversion module can be but is not limited to any deep learning neural network such as deep neural networks (DNN), deep interest learning networks (DIN) or deep recommendation networks (Factorization-Machinebased Neural Network, DeepFM).

[0145] Furthermore, after obtaining the second conversion information according to the second conversion module, the second conversion information can be input into the attention module, and the prediction result of the m+1th task can be obtained according to the parameters adjusted in the attention module and the activation function sigmoid in the attention module. It can be understood that the parameters of the attention module in the m+1th sub-model can include the association information with the mth task, and the occurrence probability of the m+1th task obtained by the attention module in the m+1th sub-model has the association with the mth task, thereby making the prediction result more reliable and accurate compared with the common technology.

[0146] See also Figure 7 , Figure 7 A schematic diagram of the structure of a multi-task model training device provided in an embodiment of this specification is shown.

[0147] The multi-task model includes M sub-models, each of which corresponds to a task and includes an attention module. Figure 7 As shown, the multi-task model training device 700 may further include at least a first processing module 701, a second processing module 702 and a training module 703, wherein:

[0148] The first processing module 701 is used to determine the parameter weight of the sample feature in the attention module of the mth sub-model; wherein m is a positive integer less than M;

[0149] The second processing module 702 is used to determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the sample features include sample user features, sample product features, and sample results of the user for the task corresponding to the m+1th sub-model;

[0150] The training module 703 is used to train the multi-task model based on the sample features, the parameter weights in the attention module of the m-th sub-model, and the parameters in the attention module of the m+1-th sub-model.

[0151] In some possible embodiments, the mth sub-model further includes an embedding module, a combining module and a first conversion module;

[0152] The first processing module 701 includes:

[0153] A first embedding unit, used to input the sample user features and the sample product features into the embedding module to obtain feature vectors corresponding to the sample user features and the sample product features respectively;

[0154] A first combining unit is used to input feature vectors corresponding to sample user features and sample product features respectively into a combining module to obtain a combined feature vector;

[0155] A first conversion unit, used for inputting the combined feature vector into a first conversion module to obtain first conversion information;

[0156] The first generating unit is used to input the first conversion information into the attention module of the m-th sub-model to obtain the parameter weight of the sample feature in the m-th sub-model.

[0157] In some possible embodiments, a fully connected module is provided between the attention module of the mth sub-model and the attention module of the m+1th sub-model;

[0158] The second processing module 702 includes:

[0159] A connection unit, used to input the parameter weights in the attention module of the m-th sub-model into the fully connected module, and obtain connection information including the parameter weights in the attention module of the m-th sub-model;

[0160] The second generating unit is used to input the connection information into the attention module of the m+1th sub-model to obtain the parameters in the attention module of the m+1th sub-model.

[0161] In some possible embodiments, the (m+1)th sub-model further includes an embedding module, a combining module, and a second conversion module;

[0162] The training module 703 includes:

[0163] A second embedding unit, used to input the sample user features and the sample product features into the embedding module to obtain feature vectors corresponding to the sample user features and the sample product features respectively;

[0164] A second combining unit is used to input feature vectors corresponding to sample user features and sample product features respectively into a combining module to obtain a combined feature vector;

[0165] A second conversion unit, used for inputting the combined feature vector into a second conversion module to obtain second conversion information;

[0166] a third generating unit, configured to input the second conversion information into the attention module of the m+1th sub-model, and obtain a prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters in the attention module of the m+1th sub-model;

[0167] The training unit is used to train the multi-task model according to the prediction results of the task corresponding to the m+1th sub-model and the sample results of the user for the task corresponding to the m+1th sub-model.

[0168] In some possible embodiments, the training module 703 further includes:

[0169] a first calculation unit, configured to obtain a first loss function according to the prediction result of the task corresponding to the m+1th sub-model and the sample feature after obtaining the prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameter of the m+1th sub-model;

[0170] The first optimization unit is used to optimize the (m+1)th sub-model based on the first loss function.

[0171] In some possible embodiments, the training module 703 further includes:

[0172] A judgment unit, configured to judge whether the prediction result of the task corresponding to the m+1th sub-model meets a preset condition after obtaining the first loss function according to the prediction result of the task corresponding to the m+1th sub-model and the sample characteristics and before optimizing the m+1th sub-model based on the first loss function;

[0173] a second computing unit, configured to obtain a prediction result of the task corresponding to the mth sub-model according to the first conversion information and a parameter in the attention module of the mth sub-model when it is determined that the prediction result of the task corresponding to the m+1th sub-model does not meet a preset condition;

[0174] a third calculation unit, configured to obtain a second loss function according to a prediction result of the task corresponding to the m-th sub-model, a prediction result of the task corresponding to the m+1-th sub-model, and sample features;

[0175] The first optimization unit is specifically used for:

[0176] The m+1th sub-model is optimized based on the first loss function and the second loss function.

[0177] In some possible embodiments, the training module 703 further includes:

[0178] a fourth calculation unit, configured to obtain a third loss function according to the prediction result of the task corresponding to the m-th sub-model and the sample feature after obtaining the prediction result of the task corresponding to the m-th sub-model according to the first conversion information and the parameter weight of the m-th sub-model;

[0179] The second optimization unit is used to optimize the m-th sub-model based on the third loss function.

[0180] See also Figure 8 , Figure 8 A schematic diagram of the structure of a multi-task prediction device provided in an embodiment of this specification is shown.

[0181] like Figure 8 As shown, the multi-task prediction device 800 is applied to a multi-task model, the multi-task model includes M sub-models, each sub-model corresponds to a task, each sub-model includes an attention module, and the multi-task prediction device 800 may include at least a third processing module 801, a fourth processing module 802 and a prediction module 803, wherein:

[0182] The third processing module 801 is used to determine the parameter weight of the target feature in the attention module of the mth sub-model; wherein m is a positive integer less than M;

[0183] The fourth processing module 802 is used to determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the target features include target user features and target product features;

[0184] The prediction module 803 is used to obtain the prediction result of the task corresponding to the m+1th sub-model based on the target features and the parameters in the attention module of the m+1th sub-model.

[0185] See also Fig. 9 , Fig. 9 A structural schematic diagram of another multi-task model training device provided in an embodiment of this specification is shown.

[0186] The multi-task model includes M sub-models, each of which corresponds to a task and includes an attention module. Fig. 9 As shown, the multi-task model training device 900 may also include: at least one processor 901, at least one network interface 904, a user interface 903, a memory 905 and at least one communication bus 902.

[0187] The communication bus 902 may be used to realize the connection and communication among the above-mentioned components.

[0188] The user interface 903 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0189] The network interface 904 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, and the like.

[0190] Among them, the processor 901 may include one or more processing cores. The processor 901 uses various interfaces and lines to connect the various parts of the entire multi-task model training device 900, and executes various functions and processes data of the multi-task model training device 900 by running or executing instructions, programs, code sets or instruction sets stored in the memory 905, and calling data stored in the memory 905. Optionally, the processor 901 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 901 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 901, but may be implemented separately by a chip.

[0191] The memory 905 may include RAM or ROM. Optionally, the memory 905 includes a non-transitory computer-readable medium. The memory 905 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 905 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 905 may optionally be at least one storage device located away from the aforementioned processor 901. As Fig. 9 As shown, the memory 905 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a multi-task model training application.

[0192] Specifically, the processor 901 may be used to call the multi-task model training application stored in the memory 905, and specifically perform the following operations:

[0193] Determine the parameter weight of the sample feature in the attention module of the mth sub-model; where m is a positive integer less than M;

[0194] Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the sample features include sample user features, sample product features, and sample results of the user for the task corresponding to the m+1th sub-model;

[0195] The multi-task model is trained based on sample features, parameter weights in the attention module of the mth sub-model, and parameters in the attention module of the m+1th sub-model.

[0196] In some possible embodiments, the mth sub-model further includes an embedding module, a combining module and a first conversion module;

[0197] When the processor 901 determines the parameter weight of the sample feature in the attention module of the mth sub-model, it specifically performs:

[0198] Inputting sample user features and sample product features into an embedding module to obtain feature vectors corresponding to the sample user features and sample product features respectively;

[0199] Inputting the feature vectors corresponding to the sample user features and the sample product features respectively into the combination module to obtain a combined feature vector;

[0200] Inputting the combined feature vector into a first conversion module to obtain first conversion information;

[0201] The first conversion information is input into the attention module of the mth sub-model to obtain the parameter weight of the sample feature in the mth sub-model.

[0202] In some possible embodiments, a fully connected module is provided between the attention module of the mth sub-model and the attention module of the m+1th sub-model;

[0203] When the processor 901 determines the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model, it specifically executes:

[0204] Input the parameter weights in the attention module of the m-th sub-model into the fully connected module to obtain connection information including the parameter weights in the attention module of the m-th sub-model;

[0205] The connection information is input into the attention module of the m+1th sub-model to obtain the parameters in the attention module of the m+1th sub-model.

[0206] In some possible embodiments, the (m+1)th sub-model further includes an embedding module, a combining module, and a second conversion module;

[0207] When the processor 901 trains the multi-task model based on the sample features, the parameter weights in the attention module of the m-th sub-model, and the parameters in the attention module of the m+1-th sub-model, the processor 901 specifically performs:

[0208] Inputting sample user features and sample product features into an embedding module to obtain feature vectors corresponding to the sample user features and sample product features respectively;

[0209] Inputting the feature vectors corresponding to the sample user features and the sample product features respectively into the combination module to obtain a combined feature vector;

[0210] Inputting the combined feature vector into a second conversion module to obtain second conversion information;

[0211] Input the second conversion information into the attention module of the m+1th sub-model, and obtain the prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters in the attention module of the m+1th sub-model;

[0212] The multi-task model is trained according to the prediction results of the task corresponding to the m+1th sub-model and the sample results of the user for the task corresponding to the m+1th sub-model.

[0213] In some possible embodiments, after obtaining the prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters of the m+1th sub-model, the processor 901 is further configured to execute:

[0214] According to the prediction results of the task corresponding to the m+1th sub-model and the sample characteristics, the first loss function is obtained;

[0215] The m+1th sub-model is optimized based on the first loss function.

[0216] In some possible embodiments, after obtaining the first loss function according to the prediction result of the task corresponding to the m+1th sub-model and the sample feature, the processor 901 is further configured to execute the following before optimizing the m+1th sub-model based on the first loss function:

[0217] Determine whether the prediction result of the task corresponding to the m+1th sub-model meets the preset conditions;

[0218] When it is determined that the prediction result of the task corresponding to the m+1th sub-model does not meet the preset condition, obtaining the prediction result of the task corresponding to the mth sub-model according to the first conversion information and the parameters in the attention module of the mth sub-model;

[0219] A second loss function is obtained according to the prediction result of the task corresponding to the m-th sub-model, the prediction result of the task corresponding to the m+1-th sub-model, and the sample characteristics;

[0220] The m+1th sub-model is optimized based on the first loss function, including:

[0221] The m+1th sub-model is optimized based on the first loss function and the second loss function.

[0222] In some possible embodiments, after obtaining the prediction result of the task corresponding to the m-th sub-model according to the first conversion information and the weight parameter of the m-th sub-model, the processor 901 is further configured to execute:

[0223] According to the prediction results of the task corresponding to the m-th sub-model and the sample characteristics, the third loss function is obtained;

[0224] The mth sub-model is optimized based on the third loss function.

[0225] See also Fig.10 , Fig.10 A schematic diagram of the structure of another multi-task prediction device provided in an embodiment of this specification is shown.

[0226] The multi-task prediction device 1000 is applied to a multi-task model, which includes M sub-models, each sub-model corresponds to a task, and each sub-model includes an attention module. Fig.10 As shown, the multi-task prediction device 1000 may further include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 and at least one communication bus 1002 .

[0227] The communication bus 1002 may be used to realize the connection and communication among the above-mentioned components.

[0228] The user interface 1003 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0229] The network interface 1004 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0230] Among them, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect the various parts of the entire multi-task prediction device 1000, and executes various functions and processes data of the multi-task prediction device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 1001 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1001, and it can be implemented separately through a chip.

[0231] Among them, the memory 1005 may include RAM and may also include ROM. Optionally, the memory 1005 includes a non-transitory computer-readable medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Fig.10 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a multi-task prediction application.

[0232] Specifically, the processor 1001 may be used to call the multi-task prediction application stored in the memory 1005, and specifically perform the following operations:

[0233] Determine the parameter weight of the target feature in the attention module of the mth sub-model; where m is a positive integer less than M;

[0234] Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the target features include target user features and target product features;

[0235] Based on the target features and the parameters in the attention module of the m+1th sub-model, the prediction result of the corresponding task of the m+1th sub-model is obtained.

[0236] The embodiment of the present specification also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes the above-mentioned Figure 3 or Figure 6 One or more steps in the illustrated embodiment. If the components of the electronic device described above are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0237] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of this specification is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0238] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0239] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Without departing from the design spirit of this specification, various modifications and improvements made to the technical solutions of this specification by ordinary technicians in this field should fall within the scope of protection determined by the claims of this specification.

Claims

1. A multi-task model training method, wherein the multi-task model includes M sub-models, each of which corresponds to a task, and each of which includes an attention module. The multi-task model is applied to a product recommendation scenario, and the task is to recommend a product to a user and expect the user to perform a corresponding event in the product recommendation scenario. The result output by each sub-model is the occurrence probability of the task corresponding to itself, and there is a correlation between the tasks corresponding to each sub-model in the order of event occurrence. The method includes: Determine the parameter weight of the sample feature in the attention module of the mth sub-model; where m is a positive integer less than M; Determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the m-th sub-model; wherein the sample features include sample user features, sample product features, and sample results of users for the task corresponding to the m+1th sub-model; the sample user features represent sample feature information of users who want to perform events, and the sample product features represent sample feature information of products corresponding to the events; when the task has not occurred, the sample result of the user for the task is represented by character 0; when the task has occurred, the sample result of the user for the task is represented by character 1; Training the multi-task model based on the sample features, the parameter weights in the attention module of the m-th sub-model, and the parameters in the attention module of the m+1-th sub-model; A fully connected module is arranged between the attention module of the mth sub-model and the attention module of the m+1th sub-model; the parameters in the attention module of the m+1th sub-model are determined according to the parameter weights in the attention module of the mth sub-model, comprising: inputting the parameter weights in the attention module of the mth sub-model into the fully connected module to obtain connection information containing the parameter weights in the attention module of the mth sub-model; and inputting the connection information into the attention module of the m+1th sub-model to obtain the parameters in the attention module of the m+1th sub-model.

2. According to the method of claim 1, the m-th sub-model further comprises an embedding module, a combination module and a first conversion module; The determining of the parameter weight of the sample feature in the attention module of the mth sub-model includes: Inputting the sample user features and the sample product features into the embedding module to obtain feature vectors corresponding to the sample user features and the sample product features respectively; Inputting the feature vectors corresponding to the sample user features and the sample product features respectively into the combination module to obtain a combined feature vector; Inputting the combined feature vector into the first conversion module to obtain first conversion information; The first conversion information is input into the attention module of the m-th sub-model to obtain the parameter weight of the sample feature in the m-th sub-model.

3. According to the method of claim 2, the m+1th sub-model further comprises the embedding module, the combining module and a second conversion module; The training of the multi-task model based on the sample features, the parameter weights in the attention module of the m-th sub-model, and the parameters in the attention module of the m+1-th sub-model includes: Inputting the sample user features and the sample product features into the embedding module to obtain feature vectors corresponding to the sample user features and the sample product features respectively; Inputting the feature vectors corresponding to the sample user features and the sample product features respectively into the combination module to obtain the combined feature vector; Inputting the combined feature vector into the second conversion module to obtain second conversion information; Inputting the second conversion information into the attention module of the m+1th sub-model, and obtaining a prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters in the attention module of the m+1th sub-model; The multi-task model is trained according to the prediction result of the task corresponding to the m+1th sub-model and the sample result of the user for the task corresponding to the m+1th sub-model.

4. According to the method of claim 3, after obtaining the prediction result of the task corresponding to the m+1th sub-model according to the second conversion information and the parameters of the m+1th sub-model, the method further comprises: Obtaining a first loss function according to the prediction result of the task corresponding to the m+1th sub-model and the sample features; The m+1th sub-model is optimized based on the first loss function.

5. The method according to claim 4, after obtaining the first loss function according to the prediction result of the task corresponding to the m+1th sub-model and the sample features, and before optimizing the m+1th sub-model based on the first loss function, further comprising: Determine whether the prediction result of the task corresponding to the m+1th sub-model meets the preset conditions; When it is determined that the prediction result of the task corresponding to the m+1th sub-model does not meet the preset condition, obtaining the prediction result of the task corresponding to the mth sub-model according to the first conversion information and the parameters in the attention module of the mth sub-model; Obtaining a second loss function according to the prediction result of the task corresponding to the m-th sub-model, the prediction result of the task corresponding to the m+1-th sub-model, and the sample features; The optimizing the m+1th sub-model based on the first loss function includes: The m+1th sub-model is optimized based on the first loss function and the second loss function.

6. The method according to claim 5, after obtaining the prediction result of the task corresponding to the m-th sub-model according to the first conversion information and the weight parameter of the m-th sub-model, the method further comprises: Obtaining a third loss function according to the prediction result of the task corresponding to the m-th sub-model and the sample features; The m-th sub-model is optimized based on the third loss function.

7. A multi-task prediction method, the method is applied to a multi-task model, the multi-task model includes M sub-models, each of the sub-models corresponds to a task, each of the sub-models includes an attention module, the multi-task model is applied to a product recommendation scenario, the task is to recommend a product to a user in the product recommendation scenario and expect the user to perform a corresponding event, the result output by each sub-model is the occurrence probability of the corresponding task, and there is a correlation between the tasks corresponding to each sub-model in the order of event occurrence, the method includes: Determine the parameter weight of the target feature in the attention module of the mth sub-model; where m is a positive integer less than M; Determining the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the target features include target user features and target product features; the target user features represent the target feature information of the user who is to perform the event, and the target product features represent the target feature information of the product corresponding to the event; Based on the target feature and the parameters in the attention module of the m+1th sub-model, obtaining a prediction result of the task corresponding to the m+1th sub-model; The multi-task model is obtained based on the multi-task model training method according to any one of claims 1 to 6.

8. A multi-task model training device, the multi-task model includes M sub-models, each of the sub-models corresponds to a task, each of the sub-models includes an attention module, the multi-task model is applied to a product recommendation scenario, the task is to recommend a product to a user in the product recommendation scenario and expect the user to perform a corresponding event, the result output by each sub-model is the occurrence probability of the corresponding task, and there is a correlation between the tasks corresponding to each sub-model in the order of event occurrence, the multi-task model training device includes: The first processing module is used to determine the parameter weight of the sample feature in the attention module of the mth sub-model; wherein m is a positive integer less than M; The second processing module is used to determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the sample features include sample user features, sample product features, and sample results of users for the task corresponding to the m+1th sub-model; the sample user features represent sample feature information of users who want to perform events, and the sample product features represent sample feature information of products corresponding to the events; when the task has not occurred, the sample result of the user for the task is represented by character 0; when the task has occurred, the sample result of the user for the task is represented by character 1; A training module, used for training the multi-task model based on the sample features, the parameter weights in the attention module of the m-th sub-model, and the parameters in the attention module of the m+1-th sub-model; A fully connected module is arranged between the attention module of the mth sub-model and the attention module of the m+1th sub-model; the parameters in the attention module of the m+1th sub-model are determined according to the parameter weights in the attention module of the mth sub-model, comprising: inputting the parameter weights in the attention module of the mth sub-model into the fully connected module to obtain connection information containing the parameter weights in the attention module of the mth sub-model; and inputting the connection information into the attention module of the m+1th sub-model to obtain the parameters in the attention module of the m+1th sub-model.

9. A multi-task prediction device, the device is applied to a multi-task model, the multi-task model includes M sub-models, each of the sub-models corresponds to a task, each of the sub-models includes an attention module, the multi-task model is applied to a product recommendation scenario, the task is to recommend a product to a user in the product recommendation scenario and expect the user to perform a corresponding event, the result output by each sub-model is the occurrence probability of the corresponding task, and there is a correlation between the tasks corresponding to each sub-model in the order of event occurrence, the device includes: The third processing module is used to determine the parameter weight of the target feature in the attention module of the mth sub-model; wherein m is a positive integer less than M; a fourth processing module, configured to determine the parameters in the attention module of the m+1th sub-model according to the parameter weights in the attention module of the mth sub-model; wherein the target features include target user features and target product features; the target user features represent the target feature information of the user who is to perform the event, and the target product features represent the target feature information of the product corresponding to the event; A prediction module, used for obtaining a prediction result of a task corresponding to the m+1th sub-model based on the target feature and the parameters in the attention module of the m+1th sub-model; The multi-task model is obtained based on the multi-task model training method according to any one of claims 1 to 6.

10. A multi-task model training device, comprising a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method according to any one of claims 1 to 6.

11. A multi-task prediction device, comprising a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method according to claim 7.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.