Multi-task processing method, multi-task processing model training method, and related device

Abstract

Embodiments of the present application provide a multi-task processing method, a multi-task processing model training method and related equipment, which relate to the field of artificial intelligence; in the method, input data of a multi-task processing model is obtained, shared features of multiple tasks are extracted from the input data by using a shared module; and through a first processing module corresponding to a first task, task information of other tasks except the first task in the multiple related tasks is selected to obtain first information related to the first task. The input data is related to the multiple related tasks. Then, an output result of the first task can be obtained based on the first information and the shared features input into the first processing module. It can be seen that the embodiments of the present application can explicitly migrate information of other tasks, which is beneficial to improving the task performance of the first task, and then effectively balancing the task performance of the multiple tasks in the multi-task processing model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a multi-task processing method, a multi-task processing model training method, and related equipment. Background Technology

[0002] Multi-task learning (MTL) is a machine learning method that, unlike single-task model learning, improves the learning performance of each task by sharing some parameters of the network and leveraging the correlations between different tasks. However, this machine learning method cannot balance the performance of each task across multiple tasks. Summary of the Invention

[0003] This application provides a multi-task processing method, a multi-task processing model training method, and related equipment, which can effectively balance the task performance of multiple tasks in a multi-task processing model.

[0004] Firstly, a multitasking method is provided, which can be executed by a multitasking device or by a chip in the multitasking device.

[0005] The aforementioned multi-task processing method is applied to a multi-task processing model. This model includes a shared module and multiple processing modules that correspond one-to-one with several related tasks.

[0006] A multi-task processing model is used to execute multiple related tasks. The network structures used to implement these tasks can be parallel, serial, or a combination of both. The multiple related tasks include a first task.

[0007] Specifically, the above-mentioned multi-task processing method includes the following steps: acquiring input data, and extracting features from the input data using a sharing module to obtain shared features of multiple related tasks. A first processing module obtains first information related to the first task from the task information of tasks other than the first task among the multiple related tasks, and obtains the output result of the first task based on the first information and the shared features. The input data is related to multiple related tasks. The first processing module corresponds to the first task.

[0008] In this embodiment, a shared module is used to extract shared features of multiple tasks from the input data; and a first processing module corresponding to the first task processes the task information of other tasks besides the first task among multiple related tasks to obtain first information related to the first task. Then, based on the first information and the shared features input to the first processing module, the output result of the first task is obtained. It can be seen that this embodiment sets up a network structure (e.g., a first processing module) to obtain information related to its own task from other task information, which can explicitly transfer information from other tasks, thus improving the task performance of the first task and effectively balancing the task performance of multiple tasks in the multi-task processing model.

[0009] In one possible implementation of the first aspect, the aforementioned plurality of related tasks further include a second task, and the multi-task processing method further includes the following step: processing the aforementioned shared features through a second processing module to obtain the output result of the second task. The second processing module corresponds to the second task.

[0010] In this embodiment of the application, the second processing module processes the shared features input to the second processing module to obtain the output result of the second task. The second processing module can complete the second task without transferring information from other tasks.

[0011] In one possible implementation of the first aspect, the first processing module includes a first selector and a first task network.

[0012] The above-mentioned first processing module obtains first information related to the first task from task information of tasks other than the first task among multiple related tasks, and obtains the output result of the first task based on the first information and shared features. Specifically, this includes the following steps: obtaining integrated information from task information of tasks other than the first task among multiple related tasks using the first selector based on a multi-head attention mechanism; and obtaining the first information by selecting a projection vector based on the integrated information and the information. Then, the first task network processes the first information and shared features to obtain the output result of the first task.

[0013] In this embodiment, the first selector employs a multi-head attention mechanism to integrate information from other tasks besides the first task to obtain integrated information. This multi-head attention mechanism captures more information from other tasks, enabling rapid, comprehensive, and effective information integration. The first selector then uses an information selection projection vector to filter the integrated information, removing information irrelevant to the first task, thus obtaining the aforementioned first information. Furthermore, the first task network obtains the output result of the first task based on the transferred first information and shared features, effectively utilizing the transferred information to improve the accuracy of the first task's output result.

[0014] In one possible implementation of the first aspect, the first task network includes a first processing tower and a first prediction module.

[0015] The above-mentioned processing of the first information and shared features by the first task network to obtain the output result of the first task specifically includes the following steps: processing the shared features and the first information through the first processing tower to obtain the first output data and the second output data respectively; and processing the first output data and the second output data through the first prediction module to obtain the output result of the first task.

[0016] In this embodiment, a first processing tower is used to process the shared features and the first information in relation to the first task to obtain first output data and second output data; then, a first prediction module is used to obtain the output result of the first task based on the first output data and the second output data; the first processing tower and the first prediction module focus on the first task, which can improve the performance of the first task.

[0017] In one possible implementation of the first aspect, the task information of the tasks other than the first task among the plurality of related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the plurality of related tasks.

[0018] In this embodiment, for each processing tower, the output vector of any network layer can be used as task information for information transfer. For example, multiple related tasks include a first task, task A, task B, and task C. Assuming that the processing towers corresponding to tasks A, B, and C each contain 5 network layers, the first processing module can obtain the first information based on the output vector of the first network layer in the processing tower of task A, the first network layer in the processing tower of task B, and the first network layer in the processing tower of task C. Alternatively, the first processing module can obtain the first information based on the output vector of the first network layer in the processing tower of task A, the second network layer in the processing tower of task B, and the last network layer in the processing tower of task C.

[0019] In one possible implementation of the first aspect, the above-described multitasking model further includes an aggregation module.

[0020] The aforementioned multi-task processing method further includes the following steps: Firstly, the task information of multiple related tasks, excluding the first task, is aggregated using an aggregation module to obtain aggregated information. Secondly, the first processing module obtains first information based on the aggregated information.

[0021] In this embodiment of the application, information aggregation can reduce the data dimension of information for tasks other than the first task, improve computational efficiency and accuracy, and quickly and accurately select the first information.

[0022] In one possible implementation of the first aspect, the aforementioned multiple related tasks include a click-through rate prediction task, a conversion rate prediction task, a payment rate prediction task, or an activation rate prediction task.

[0023] In one possible implementation of the first aspect, the aforementioned sharing module includes a shared underlying network or an expert network equipped with a gating mechanism. In this embodiment, the sharing module can be multiple related tasks sharing an identical network, which is the shared underlying network. Therefore, the shared features are the same for the multiple related tasks. Alternatively, the sharing module can be multiple related tasks sharing a partially identical network, such as an expert network equipped with a gating mechanism. Different tasks can selectively use features of the shared underlying network and the expert network according to their own task needs. Thus, the shared features can be the same or different for the multiple related tasks.

[0024] In one possible implementation of the first aspect, the shared module further includes a feature transformation module to perform feature transformation on the input data, thereby reducing the dimensionality of the input data, saving computing resources, and improving the computational efficiency of the model.

[0025] Secondly, this application also provides a method for training a multi-task processing model, used to train the aforementioned multi-task processing model. This training method can be executed by a model training device or by a chip within the model training device.

[0026] Specifically, the above training method includes the following steps: acquiring training data; extracting shared features from the training data using a shared module to obtain shared features for multiple related tasks; obtaining information related to the first task from the task information of tasks other than the first task among the multiple related tasks using a first processing module; processing the shared features and the information related to the first task input to the first processing module using the first processing module to obtain third output data and fourth output data; obtaining a first loss value for the first task based on the third output data and fourth output data; and adjusting the model parameters of the multi-task processing model based on the loss values ​​of the multiple related tasks, including the first loss value. The training data is related to the multiple related tasks. The first processing module corresponds to the first task.

[0027] In this embodiment, a shared module is used to extract shared features of multiple tasks from the training data. A first processing module corresponding to the first task processes the task information of other tasks besides the first task to obtain information related to the first task. The first processing module then processes the shared features and the information related to the first task to obtain third and fourth output data, and finally, a first loss value for the first task is obtained based on the third and fourth output data. Finally, the model parameters are adjusted based on the loss values ​​of multiple tasks (including the first loss value). Therefore, this embodiment sets up a network structure (e.g., a first processing module) to obtain information related to its own task from other task information. This allows the task information of other tasks to be incorporated into the task's loss, helping the model better learn the current task. After training, the task information of other tasks can be explicitly transferred, which is beneficial for improving the task performance of the first task and effectively balancing the task performance of multiple tasks in the multi-task processing model.

[0028] In one possible implementation of the second aspect, the first processing module includes a first selector. The process of obtaining information related to the first task from task information of tasks other than the first task among multiple related tasks via the first processing module specifically includes the following steps: obtaining integrated information from the task information of tasks other than the first task among multiple related tasks based on a multi-head attention mechanism using the first selector; and processing the integrated information and the information selection projection vector using the first selector to obtain information related to the first task.

[0029] In this embodiment, the first selector employs a multi-head attention mechanism to integrate task information from tasks other than the first task to obtain integrated information. This multi-head attention mechanism captures more information from other tasks, enabling rapid, comprehensive, and effective information integration. The first selector then uses an information selection projection vector to filter the integrated information, removing information irrelevant to the first task to obtain the aforementioned information related to the first task.

[0030] In one possible implementation of the second aspect, the first processing module further includes a first task network, which includes a first processing tower. The process of processing shared features and information related to the first task through the first processing module to obtain third output data and fourth output data specifically includes the following steps: processing the shared features through the first processing tower to obtain third output data; and processing the information related to the first task through the first processing tower to obtain fourth output data.

[0031] In this embodiment of the application, the first processing tower is used to process the shared features and information related to the first task to obtain the first output data and the second output data.

[0032] In one possible implementation of the second aspect, obtaining the first loss value of the first task based on the third output data and the fourth output data specifically includes the following steps: obtaining the task loss value of the first task based on the third output data and the first expected value. The first expected value is the expected value of the third output data. Obtaining the information migration loss value of the first task based on the fourth output data and the second expected value. The second expected value is the expected value of the fourth output data. Determining the first loss value based on the information migration loss value and the task loss value.

[0033] In this embodiment, the first loss value is determined based on the information migration loss value and the task loss value. Since the information migration loss is taken into account, the loss expression of the first task is more accurate, which is conducive to improving the task performance of the first task.

[0034] In one possible implementation of the second aspect, the aforementioned plurality of related tasks further include a second task, and the training method further includes the following steps: processing the shared features through a second processing module to obtain fifth output data; obtaining a second loss value for the second task based on the fifth output data and a third expected value; the loss value of the plurality of related tasks includes the second loss value; wherein, the second processing module corresponds to the second task; and the third expected value is the expected value of the fifth output data.

[0035] In this embodiment, the shared features input to the second processing module are processed by the second processing module to obtain the fifth output data, and then the second loss value is obtained based on the fifth output data and the third expected value. It is evident that the second processing module can complete the second task without migrating task information from other tasks; therefore, information migration loss does not need to be considered when determining the second loss value.

[0036] In one possible implementation of the second aspect, the task information of the tasks other than the first task among the aforementioned plurality of related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the plurality of related tasks.

[0037] In one possible implementation of the second aspect, the multi-task processing model further includes an aggregation module, and the training method further includes the following steps: The aggregation module aggregates task information of tasks other than the first task among multiple related tasks to obtain aggregated information. The first processing module then obtains information related to the first task based on the aggregated information.

[0038] In this embodiment of the application, information aggregation can reduce the data dimension of task information for tasks other than the first task, improve computational efficiency and accuracy, and quickly and accurately select information related to the first task.

[0039] In one possible implementation of the second aspect, the aforementioned multiple related tasks include a click-through rate prediction task, a conversion rate prediction task, a payment rate prediction task, or an activation rate prediction task.

[0040] In one possible implementation of the second aspect, the aforementioned shared module includes a shared underlying network or an expert network equipped with gating mechanisms.

[0041] In one possible implementation of the second aspect, the shared module further includes a feature transformation module to perform feature transformation on the training data, thereby reducing the dimensionality of the training data, saving computing resources, and improving the training efficiency of the model.

[0042] Thirdly, this application also provides a multitasking device, the device including a unit or module for performing the multitasking method described in the first aspect.

[0043] Fourthly, this application also provides a model training device, the device including a unit or module for performing the multi-task processing model training method described in the second aspect.

[0044] Fifthly, this application also provides an electronic device, including a processor and a memory, wherein the processor and the memory are connected together, wherein the memory is used to store program code, and the processor is used to call the program code to perform the method as described in the first or second aspect.

[0045] Sixthly, this application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the method as described in the first or second aspect.

[0046] In a seventh aspect, this application also provides a computer program product containing instructions that, when the computer program product is run on a computer, cause the computer to perform the method described in the first aspect or the second aspect.

[0047] Eighthly, this application also provides a chip, the chip including a processor and a data interface, the processor reading instructions stored in a memory through the data interface to execute the method described in the first aspect or the second aspect.

[0048] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is configured to execute the instructions stored in the memory. When the instructions are executed, the processor is configured to perform the method described in the first aspect or the second aspect. Attached Figure Description

[0049] The accompanying drawings used in the embodiments of this application are described below.

[0050] Figure 1A A schematic diagram of an attention layer provided for an embodiment of this application;

[0051] Figure 1B A schematic diagram of a multi-gated multi-expert model provided for an embodiment of this application;

[0052] Figure 2A A schematic diagram of a system architecture provided for an embodiment of this application;

[0053] Figure 2B A schematic diagram of a network structure for a multi-task processing model provided in an embodiment of this application;

[0054] Figure 2C A schematic diagram of the network structure of another multi-task processing model provided for an embodiment of this application;

[0055] Figure 2D A schematic diagram of the network structure of another multi-task processing model provided for an embodiment of this application;

[0056] Figure 2E A schematic diagram of the network structure of another multi-task processing model provided for an embodiment of this application;

[0057] Figure 3 A flowchart illustrating a multitasking method provided for an embodiment of this application;

[0058] Figure 4 A flowchart illustrating a multi-task processing model training method provided for an embodiment of this application;

[0059] Figure 5 A schematic diagram of the structure of a multitasking device provided for an embodiment of this application;

[0060] Figure 6 A schematic diagram of the structure of a model training device provided for an embodiment of this application;

[0061] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an embodiment of this application. Detailed Implementation

[0062] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0063] In this application, the terms "exemplarily" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplarily" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0064] In this application, "at least one" in the embodiments refers to one or more items, and "more than one" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or multiple. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The step numbers in the embodiments of this application (such as step S1, step S21, etc.) are only for distinguishing different steps and do not limit the order of execution between steps.

[0065] Furthermore, unless otherwise stated, the use of ordinal numbers such as "first" and "second" in the embodiments of this application is for distinguishing multiple objects and is not for limiting the order, sequence, priority, or importance of multiple objects. For example, "first device" and "second device" are only for ease of description and do not indicate that the first device and the second device are different in structure, importance, etc. In some embodiments, the first device and the second device may also be the same device.

[0066] In the above embodiments, the term "when..." can be interpreted, depending on the context, as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". The above descriptions are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of this application should be included within the protection scope of this application.

[0067] To facilitate understanding, the relevant terms and concepts involved in the embodiments of this application will be introduced below.

[0068] (1) Neural Networks

[0069] Neural networks can be composed of neural units, which can refer to units represented by x. s The arithmetic unit that takes an intercept of 1 as input can output the following:

[0070]

[0071] Where s = 1, 2, ..., d, d is a natural number greater than 1, W s For x s The weights are denoted by b, where b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer. The activation function can be the sigmoid function. A neural network is a network formed by connecting many of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0072] (2) Deep Neural Networks

[0073] Deep Neural Networks (DNNs), also known as multilayer neural networks, can be understood as neural networks with many hidden layers, though there's no specific metric for "many." DNNs can be categorized into three layers based on their position: input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. All layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer. Although DNNs appear complex, the operation of each layer is actually quite simple, resembling a linear relationship as follows: in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is therefore quite large. The definitions of these parameters in a DNN are as follows: Taking the coefficient W as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W is located, while the subscript corresponds to the third layer index 2 of the output and the second layer index 4 of the input.

[0074] In summary: the coefficient between the nth neuron in layer L-1 and the jth neuron in layer L is defined as... It's important to note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).

[0075] (3) Loss Function

[0076] In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (Loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0077] (4) Backpropagation algorithm

[0078] Convolutional neural networks can employ backpropagation (BP) to correct the parameters in the initial super-resolution model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates error loss; this error loss information is then propagated back to update the parameters in the initial super-resolution model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the super-resolution model, such as the weight matrix.

[0079] (5) Attention mechanism

[0080] The attention mechanism originates from research on human vision. In cognitive science, due to the bottleneck of information processing, humans selectively focus on a portion of all information while ignoring other visible information. This mechanism is commonly referred to as the attention mechanism.

[0081] Attention operations extract weights from the vast amount of information contained in the data to describe the importance of features, and combine these weights with the features, allowing the network to focus on information with higher weights. The network layer that performs attention operations is called an attention layer; specifically, refer to... Figure 1A , Figure 1A This is a schematic diagram of the attention layer; the attention layer includes the query feature transformation module (i.e. Figure 1A The Q module), the key feature transformation module (i.e. Figure 1A The K module), the Value feature transformation module (i.e. Figure 1A The V module in the attention layer, x is the input to the attention layer. Q x K x V After feature transformations by the Q module, K module, and V module respectively, it becomes K. out Q out V out According to K out and Q out Generate relevant weights, and then apply them to V based on the weight coefficients. out The weighted summation is performed, and the weighted feature `out` is output. Here, self-attention is a special type of attention operation, where self-attention occurs when the x-axis of the attention layer... Q =x K =x V When x = , the attention layer at this time is the self-attention layer.

[0082] refer to Figure 1A The basic process is described below. The input to the self-attention layer is x. First, three feature transformations are performed on x to obtain Q. out K out and V out :

[0083] Q out =xW Q K out =xW K V out =xW V (1)

[0084] In equation (1), x is the input of the self-attention layer, W QW K W v W is the characteristic transformation matrix. Q W K W v In a convolutional neural network, these are fully connected layers, and their weights are learnable parameters. Let x be correlated with W... Q W K W v Multiply to get Q out K out and V out The goal is to map the x features to another dimension, so that the mapped features satisfy the Q-axis in the Attention operation. out K out and V out Requirements, W Q W K W v The functionality will gradually become apparent as the deep neural network training process progresses. Generally, after W... K After transformation, K is obtained out It can describe the content features of the input, representing what the input features are like; after W Q Q after feature transformation out It can contain guiding features about the input, representing what information the model needs to process; and after W v V after feature transformation out It is a vector that represents the input features.

[0085] Then Q out With K out Matrix multiplication (MatMul) is performed, followed by numerical scaling using a scaling layer. Optionally, in text processing, a mask layer is also used to process the scaled data during attention operations. Finally, after Softmax normalization, an Attention Map describing the autocorrelation between dimensions x is obtained, i.e., the Attention Map is:

[0086]

[0087] In equation (2), d is a constant, which is a custom value and is set according to the different needs of the model. Matrix multiplication is equivalent to finding guiding features within content features.

[0088] Finally, an Attention Map is used to apply V. out Weighted, i.e., V out The matrix multiplication with the Attention Map, followed by feature transformation, is used as the output of the attention layer. W in equation (3)out This is the characteristic transformation matrix.

[0089] out = AV out W out (3)

[0090] Based on the above introduction to the Attention operation, we can denote the Attention operation as:

[0091] Attention(x Q x K x V )

[0092] The Self-Attention (SA) operation, based on the Attention operation, sets the inputs of the Q module, K module, and V module to the same input x. The Self-Attention operation is denoted as:

[0093] SA(x) = Attention(x,x,x)

[0094] In multi-head attention, the input features (typically queries, keys, and values) are processed by multiple independent, parallel-running attention modules (or "heads"). Each head independently computes an attention score and generates an attention-weighted output. These outputs are then combined to form a final, more complex representation—the final multi-head attention output.

[0095] (6) Multi-task processing model

[0096] A multi-task processing model is a model learned through multi-task learning, capable of handling multiple related tasks. Each task has its own processing tower and prediction module. The processing tower is used for feature extraction and representation learning for a specific task, providing effective input to the prediction module. The prediction module is responsible for making task predictions and outputs based on these feature representations.

[0097] The Multi-gate Mixture-of-Experts (MMoE) model is a multi-task learning architecture designed to effectively capture the relationships between multiple tasks and optimize learning performance.

[0098] refer to Figure 1BThe diagram illustrates the network architecture of a multi-gated, multi-expert (MMoE) model. The foundation of the MMoE model is a shared underlying network responsible for extracting general features from the input data. This design allows multiple tasks to share the same feature representations, thereby reducing the number of parameters and improving the model's generalization ability. Each task has an independent gating network that selectively utilizes the features extracted by the underlying network. The gating network assigns weights to the features output by the underlying network according to the needs of each task, enabling each task to focus on its relevant features. This mechanism significantly enhances the model's ability to capture differences between tasks. MMoE also introduces multiple expert networks, which focus on handling specific tasks. Figure 1B Taking expert networks 0, 1, and 2 as examples, gating networks achieve more efficient learning and better performance by dynamically selecting experts suitable for the current task. Figure 1B Taking Task A and Task B as examples, Task A is equipped with a gating network A, a processing tower A, and a prediction module A. Task B is equipped with a gating network B, a processing tower B, and a prediction module B.

[0099] However, Figure 1B The network structure shown has the following shortcomings:

[0100] First, modeling strongly correlated tasks is difficult. MMoE relies on a shared underlying network and multiple gating mechanisms to handle different tasks. When strong correlations exist between tasks, the model needs to accurately capture these complex relationships. However, since information transfer occurs implicitly through shared parameters, the model may not be able to effectively identify and utilize these strongly correlated features. This means that MMoE may not reach its full potential when faced with highly correlated tasks, resulting in poor model performance.

[0101] Second, negative transfer can occur. In multi-task learning, the correlation between different tasks is crucial to model performance. If the correlation between some tasks is low, MMoE may exhibit negative transfer, meaning that learning from one task can negatively impact another. When multiple tasks share underlying features, some features may be beneficial to one task but irrelevant or harmful to another. This is particularly evident in strongly correlated tasks because the model cannot flexibly adjust shared features to suit the unique needs of each task.

[0102] Third, information conflicts arise from parameter sharing. Since all expert networks in an MMoE model are shared, parameter conflicts may occur during training for different tasks. This conflict makes it difficult for the model to find the optimal solution, especially when dealing with tasks that differ significantly. For example, if one task needs to emphasize certain features while another task needs to suppress them, the model will struggle to balance these two aspects, thus affecting the final model performance.

[0103] The following example, using a multi-task processing model applied to a recommendation system, further describes the technical problems addressed by the embodiments of this application.

[0104] When multi-task processing models are applied to recommender systems, optimizing multiple recommendation-related tasks simultaneously (such as click-through rate prediction) can improve the generalization ability and recommendation quality of the model. Multi-task learning methods typically train by jointly optimizing multiple objective functions corresponding to different recommendation-related tasks (such as different types of recommendations or predictions). By sharing learned feature representations (such as user features, item features, etc.), these tasks can improve model performance through mutual influence and collaboration. Multiple tasks in a recommender system are often interconnected. For example, a user's behavioral history can be used not only to predict their ratings but also to infer their click-through rate or purchase tendency. By simultaneously optimizing these recommendation-related tasks in a single model, the model can learn shared information between tasks, thereby improving recommendation quality. Multi-task learning utilizes shared representations between tasks for information propagation and knowledge transfer. Through this implicit sharing, the model can acquire valuable knowledge from other tasks, especially when some tasks have limited data; the knowledge provided by other tasks can effectively supplement this, thereby improving the overall recommendation performance. For example, a user's preference for a certain type of item (such as movie recommendations) can provide useful prior knowledge for predicting preferences for other types of items (such as music recommendations). However, this learning method cannot fully realize the potential of multi-task processing models, resulting in poor model performance.

[0105] In addition, recommendation systems include Click-Through Rate (CTR) prediction tasks and Conversion Rate (CVR) prediction tasks. CTR refers to the ratio of the number of times a user clicks on an ad or link to the number of times that ad or link is displayed. Conversion rate refers to the ratio of the number of times a user completes a specific goal (such as purchasing, registering, or downloading) to the total number of times they visit that goal page or ad. Conversion rates can include those corresponding to different scenarios and purposes, such as basic conversion rate (e.g., the ratio of conversions to clicks), payment conversion rate (the ratio of total payments to total orders), service conversion rate, or Click-ThroughConversionRate (CTCVR), etc. CTCVR is calculated as CTR * CVR.

[0106] A challenge in recommender systems is the relatively sparse training samples for CTR prediction tasks, primarily due to infrequent user interactions. For example, in short video recommendation scenarios, user interactions with the platform can be categorized into two types of feedback signals. The first is a binary signal indicating whether the user performed an action, such as clicking, liking, or sharing. The second is a numerical signal, such as viewing time, profile dwell time, and comment section dwell time. Typically, clicking is a prerequisite for other user actions, resulting in significantly more effective data for CTR prediction tasks compared to relatively less for CVR prediction tasks. Therefore, recommender systems often struggle with sparse data, which weakens the performance of recommendation models that rely on historical interactions.

[0107] In summary, existing multi-task processing models cannot balance the performance of multiple tasks.

[0108] Therefore, this application proposes a multi-task processing method that can effectively balance the task performance of multiple tasks in a multi-task processing model.

[0109] The multi-task processing method of this application is applied to a multi-task processing model. This method can be applied to data mining (e.g., recommendation, search, and advertising scenarios) or other various machine learning fields, without particular limitation. The multi-task processing model is used to execute multiple related tasks. Multiple related tasks refer to multiple processing tasks that share some model parameters or structures. When the multi-task processing model is applied to a recommendation scenario, the multiple related tasks are recommendation-related tasks (e.g., click-through rate prediction, conversion rate prediction, rating prediction, user preference modeling, etc.). Using the method of this application, the accuracy and diversity of personalized user recommendations can be effectively improved. When the multi-task processing model is applied to a search scenario, the multiple related tasks are search-related tasks (e.g., page analysis, retrieval, payment rate prediction, activation rate prediction, etc.). Using the method of this application, the relevance of search results can be optimized, ultimately enhancing user experience and improving conversion rates. When the multi-task processing model is applied to an advertising scenario, the multiple related tasks are advertising-related tasks (e.g., click-through rate prediction, conversion rate prediction, purchase rate prediction, etc.). Using the method of this application, the advertising effect can be optimized, ultimately enhancing user experience and improving conversion rates.

[0110] The multitasking method of this application embodiment can be executed by a multitasking device or by a chip in the multitasking device.

[0111] The following describes a system architecture provided by an embodiment of this application.

[0112] See Figure 2A , Figure 2A This is a schematic diagram of a system architecture provided in an embodiment of this application; an embodiment of this application provides a system architecture 200. As shown in the system architecture 200, the data acquisition device 260 is used to collect training data for a multi-task processing model. In this embodiment, the training data consists of information related to multiple related tasks. The data acquisition device 260 stores the training data in a database 230.

[0113] Training device 220 can obtain multi-task processing model 201 based on training data maintained in database 230. The specific training process is described below. Figure 4 The relevant records are as follows. For example, in this embodiment, after model training is completed, the multi-task processing model 201 can acquire the ability to process multiple related tasks. The training device 220 can be a server or cloud service device, or it can be a mobile terminal, desktop computer, tablet computer, laptop computer, or other devices.

[0114] It should be noted that in practical applications, the training data maintained in the database 230 may not all come from the data acquisition device 260; it may also be received from other devices. Furthermore, it should be noted that the training device 220 may not necessarily train the model entirely based on the training data maintained in the database 230; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0115] The multi-task processing model 201 obtained after processing by training device 220 can be applied to systems or devices with the same hardware configuration as training device 220, such as in... Figure 2A The multitasking device 210 shown can complete multiple related tasks based on the multitasking model 201. The multitasking device 210 can be a terminal, such as a mobile terminal, desktop computer, tablet computer, laptop computer, etc., or it can be a server or cloud platform. The training device 220 and the multitasking device 210 can be the same device.

[0116] In the appendix Figure 2A In this embodiment, the multitasking device 210 is equipped with an I / O interface 212 for data interaction with external devices. Users can input data to the I / O interface 212 through the client device 240. In this embodiment, the input data can be data related to multiple related tasks (such as user behavior data). It can be input by the user or it can come from the database 230.

[0117] Optionally, the preprocessing module 213 is used to preprocess the input data received by the I / O interface 212. In this embodiment, the preprocessing module 213 is used to preprocess the input data received by the I / O interface 212, and the preprocessed data enters the calculation module 211.

[0118] During the preprocessing of input data by the multitasking device 210, or during the calculation process performed by the computing module 211 of the multitasking device 210, the multitasking device 210 can call data, code, etc. in the data storage system 250 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 250.

[0119] Finally, exemplarily, I / O interface 212 returns the model result of multitasking model 201 to client device 240, thereby providing it to the user. In this case, client device 240 can be a display. In this embodiment, multitasking device 210 executes the multitasking method of this embodiment.

[0120] In the appendix Figure 2AIn the scenario shown, the user can manually provide input data, which can be done through the interface provided by I / O interface 212. Alternatively, the client device 240 can automatically send input data to I / O interface 212. If user authorization is required for the client device 240 to automatically send input data, the user can set the corresponding permissions in the client device 240. The user can view the output results of the multitasking device 210 on the client device 240, which can be presented in various ways such as display, sound, or animation. The client device 240 can also act as a data acquisition terminal, collecting data such as... Figure 2A The input data and output results of the input I / O interface 212 are used as new sample data and stored in the database 230. Alternatively, data can be collected directly from the I / O interface 212 without going through the client device 240. Figure 2A The input data of the input I / O interface 212 and the output results of the output I / O interface 212 are stored in the database 230 as new sample data.

[0121] It is worth noting that, attached Figure 2A This is merely a schematic diagram of a system architecture provided in an embodiment of this application. Figure 2A The positional relationships between the devices, components, modules, etc. shown do not constitute any limitation. For example, in the attached... Figure 2A In this context, the data storage system 250 is an external storage device relative to the multitasking device 210. In other cases, the data storage system 250 may also be placed within the multitasking device 210.

[0122] The multi-task processing model 201 in this embodiment can be implemented using a neural network model, and the specific model structure of the multi-task processing model 201 is not limited. For example, for multiple related tasks in the multi-task processing model 201, some tasks can transfer information from other tasks to assist in task implementation, while the remaining tasks do not transfer information from other tasks to assist in task implementation. Alternatively, all tasks can transfer information from other tasks to assist in task implementation.

[0123] For example, the network structures used to implement multiple related tasks can be parallel network structures, serial network structures, or network structures where parallel and serial connections coexist.

[0124] For example, the multi-task processing model 201 includes a sharing module and multiple processing modules corresponding one-to-one with multiple related tasks. To obtain information from other tasks, each processing module needs to establish connections with the processing modules of those other tasks. In this case, the number of connections in the network structure is large, resulting in high computational complexity. Further exemplarily, the multi-task processing model 201 also includes an aggregation module. Setting up an aggregation module for information aggregation can simplify the model, reduce the number of connections, lower computational complexity, and accelerate information filtering. For a detailed description of the sharing module, processing module, and aggregation module, please refer to the following text. Figures 2B to 3 Related records.

[0125] refer to Figure 2B , Figure 2B This is a schematic diagram of a network structure for a multi-task processing model provided in an embodiment of this application. In this embodiment, multiple related tasks are taken as five tasks, namely, the first task, the second task, the third task, the fourth task, and the fifth task. The multi-task processing model includes a sharing module, an aggregation module, a first processing module corresponding to the first task, a second processing module corresponding to the second task, a third processing module corresponding to the third task, a fourth processing module corresponding to the fourth task, and a fifth processing module corresponding to the fifth task. Figure 2B In this system, each of the five tasks migrates task information from other tasks to assist in achieving its own task. The network structure used to achieve the five tasks is a parallel network structure.

[0126] refer to Figure 2C , Figure 2C This document provides a schematic diagram of the network structure of a multi-task processing model for an embodiment of this application. In this embodiment, taking a first processing module as an example, the first processing module includes a first selector and a first processing network, and the first processing network includes a first processing tower and a first prediction module. Exemplarily, the processing tower for each task performs deep learning label fitting for that task, and the processing tower can be a multilayer perceptron (MLP). The prediction module for each task can supervise the learning process of the neural network; that is, the output of the prediction module participates in loss calculation, while the labels in the training samples supervise model training. For a detailed description of the first selector, the first processing tower, and the first prediction module, please refer to the following text. Figure 2D and Figure 3 The relevant records. The structures of the second, third, fourth, and fifth processing modules are the same as those of the first processing module.

[0127] refer to Figure 2D , Figure 2DThis is a schematic diagram of the network structure of another multi-task processing model provided in an embodiment of this application. In this embodiment, the output of the first prediction module is used as the input of the fifth processing tower. Therefore, the network structure used to implement the first task and the network structure used to implement the fifth task are connected in series. The network structures used to implement the second task, the third task, and the fourth task are connected in parallel.

[0128] In one possible implementation, the aforementioned sharing module can facilitate feature interaction between tasks in a multi-task processing model. Further, the sharing module includes a shared bottom network or a Mixture of Experts (MOE) network equipped with a gating mechanism. In this embodiment, the sharing module can be multiple related tasks sharing an identical network, which is the shared bottom network. In this case, the shared features input to each of the multiple related tasks are the same. Alternatively, the sharing module can be multiple related tasks sharing a partially identical network, such as an expert network equipped with a gating mechanism. Different tasks can selectively use features from the shared bottom network and the expert network according to their own task needs. Thus, the shared features input to multiple related tasks can be the same or different.

[0129] In one possible implementation, the shared module further includes a feature transformation module to transform the features of the input data of the multi-task processing model, thereby reducing the dimensionality of the input data, saving computational resources, and improving the computational efficiency of the model. For example, the feature transformation module can be implemented using an embedding module, which maps the input data to a continuous vector space. This mapping preserves the key features of the data, enabling the data to better capture its inherent semantic, structural, or relational information in the new vector space.

[0130] refer to Figure 2E , Figure 2EThis is a schematic diagram of the network structure of another multi-task processing model provided in an embodiment of this application. In this embodiment, the shared module includes a feature conversion module and a shared underlying network as an example. Multiple related tasks include a CTR prediction task and four CVR prediction tasks, namely CVR prediction task 1, CVR prediction task 2, CVR prediction task 3, and CVR prediction task 4. These four CVR prediction tasks can be of different types, such as shopping cart CVR prediction tasks, website conversion rate prediction tasks, advertising conversion rate prediction tasks, or sales CVR prediction tasks. The CTR prediction task does not migrate task information from multiple CVR prediction tasks to assist in completing the prediction. Each CVR prediction task utilizes the method of this embodiment to migrate task information from other tasks to assist in completing its own task. For example, for CVR prediction task 1, the aggregation module aggregates information from CTR prediction task 2, CVR prediction task 3, and CVR prediction task 4, and selector 1 selects information from the aggregated information to filter out information related to CVR prediction task 1, so that the processing tower 2 and CVR prediction module 1 corresponding to CVR prediction task 1 can obtain the output result of CVR prediction task 1 based on shared features and information related to CVR prediction task 1.

[0131] Figure 2B , Figure 2C , Figure 2D and Figure 2E In the model structure shown, the dashed lines represent stopping gradients, indicating the position where the gradient stops during backpropagation. In addition to the multi-task processing model... Figure 2B , Figure 2C , Figure 2D and Figure 2E Besides the model structure shown, other model structures can also be used, as long as they can achieve multiple related task processing; there are no restrictions on this.

[0132] The following section provides a detailed explanation of the multitasking methods.

[0133] refer to Figure 3 , Figure 3 This is a flowchart illustrating a multitasking method provided in an embodiment of this application; the execution entity of the multitasking method is a multitasking device, and the multitasking method includes some or all of the following steps:

[0134] 301. Multitasking devices acquire input data.

[0135] Specifically, the input data described above is related to multiple related tasks. For example, multiple related tasks refer to multiple processing tasks that have model correlation. Here, model correlation means that the tasks share some model parameters or structures.

[0136] The specific type of input data depends on the task type. For example, when the task is advertising-related, the input data may include at least one of advertising data, user data, or user context data. As another example, when the task is recommendation-related, taking item recommendation as an example, the input data may include at least one of user data (e.g., including basic user information and user behavior data), user context data, or item data.

[0137] 302. The multi-task processing device extracts features from the input data through a shared module to obtain shared features of multiple related tasks.

[0138] Specifically, the shared features of multiple related tasks include the shared features of each task. The shared features of each task can be the same or different features, and can be set according to actual needs.

[0139] 303. The multi-task processing device obtains first information related to the first task from the task information of tasks other than the first task among multiple related tasks through the first processing module, and obtains the output result of the first task based on the first information and shared features.

[0140] Specifically, the aforementioned first processing module corresponds to the first task. The aforementioned plurality of processing modules include the first processing module. The first processing module obtains first information related to the first task from the task information of other tasks, and obtains the output result of the first task based on this first information and the shared features input to the first processing module. The network structure of the first processing module is not limited, as long as it can achieve the above functions.

[0141] Optionally, the first information refers to information that helps improve the performance of the first task. It can also be understood as information that has a positive effect on the task processing of the first task.

[0142] In this embodiment, a shared module is used to extract shared features of multiple tasks from the model's input data. A first processing module corresponding to the first task processes the task information of other tasks besides the first task among multiple related tasks to obtain first information related to the first task. Then, based on the first information and the shared features input to the first processing module, the output result of the first task is obtained. It is evident that this embodiment sets up a network structure (e.g., a first processing module) to obtain information related to its own task from other task information. This allows for explicit transfer of task information from other tasks, which is beneficial for improving the task performance of the first task and effectively balancing the task performance of multiple tasks in the multi-task processing model.

[0143] Furthermore, since multi-task processing models allow for information transfer between tasks to improve their performance, they can effectively balance the performance of each task, even when faced with strongly correlated tasks. Simultaneously, because the method in this embodiment effectively balances the performance of each task, negative transfer does not occur. When multi-task processing models are applied to recommendation systems, the method in this embodiment can fully leverage the potential of the multi-task processing model and improve recommendation effectiveness. Moreover, it can effectively balance the performance of each task even when training samples are sparse.

[0144] As one implementation, the task information of the tasks other than the first task among the aforementioned related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the related tasks.

[0145] In this embodiment, for each processing tower, the output vector of any network layer can be used as task information for information transfer. For example, multiple related tasks include a first task, task A, task B, and task C. Assuming that the processing towers corresponding to tasks A, B, and C each contain 5 network layers, the first processing module can obtain the first information based on the output vector of the first network layer in the processing tower of task A, the first network layer in the processing tower of task B, and the first network layer in the processing tower of task C. Alternatively, the first processing module can obtain the first information based on the output vector of the first network layer in the processing tower of task A, the second network layer in the processing tower of task B, and the last network layer in the processing tower of task C.

[0146] In this embodiment, information transfer of the task is achieved through the output vector of the network layer, which can achieve information transfer with less resource consumption.

[0147] As one implementation method, the above multitasking method also includes the following steps:

[0148] The multitasking device aggregates task information from multiple related tasks, excluding the first task, through an aggregation module to obtain aggregated information.

[0149] Specifically, this application embodiment designs a central hub for information integration and distribution, namely an aggregation module. Task information from all tasks is aggregated into this aggregation module. Taking a recommendation system as an example, refer to... Figure 2E By using an aggregation module, each CVR prediction task can receive task information not only from the CTR prediction task but also from all other CVR prediction tasks, effectively mitigating the data sparsity problem. Let f ctr f represents a representation vector containing information about the CTR prediction task.cvri Let represent the vector for CVR prediction task i. Therefore, the information imported from the aggregation module into CVR prediction task i (i.e., the information used for information selection) is defined as follows:

[0150] F i =[f ctr f cvr1 , ..., f cvri-1 f cvri+1 , ..., f cvrk ]

[0151] As can be seen, the information imported into a task by the aggregation module is task information from other tasks besides the task itself.

[0152] The specific method for aggregating task information is not limited. For example, when the task information is the output vector of the network layer, various vector aggregation methods can be used, such as vector concatenation, element-wise addition or multiplication, tensor product, etc.

[0153] The multi-tasking device selects the first information from the aggregated information through the first processing module.

[0154] In this embodiment of the application, information aggregation can reduce the data dimension of task information for tasks other than the first task, improve computational efficiency and accuracy, and thus enable the first information to be selected quickly and accurately.

[0155] As one implementation method, step 303 above specifically includes the following steps:

[0156] 331. The multitasking device obtains integrated information from task information of multiple related tasks other than the first task based on a multi-head attention mechanism through the first selector, and obtains the first information based on the integrated information and the selected projection vector.

[0157] Specifically, the specific value of the information selection projection vector can be determined through model training, and then the information selection projection vector is used to select information from the integrated information to obtain the first information.

[0158] For example, in step 331, the first selector can directly integrate information from task information other than the first task, or it can integrate information from aggregated information corresponding to the first task imported from the aggregation module, and then select the first information from the integrated information. Taking the integration of aggregated information as an example, the specific process of information selection is as follows:

[0159] Given a list F of input representations for CVR prediction task i i First, the multi-head self-attention mechanism is used to target F. iA comprehensive integration is performed. Specifically, the self-attention calculation is as follows:

[0160]

[0161] Q, K, and V are obtained by using different learnable projection matrices W. Q W K and W V ∈Rd×d from F i These matrices are derived from [the original text]. They correspond to the query, key, and value, respectively. The attention score can be viewed as a weighted sum of values ​​from one aspect. The calculation of the attention score can be performed in parallel, leading to the implementation of Multi-Head Self-Attention (MHSA):

[0162]

[0163] Among them, W O It is a projection matrix used to ensure dimensional consistency, while head h This indicates the output of the h-th header:

[0164]

[0165] Finally, the output of multi-head attention is processed for information selection, aiming to filter out irrelevant information in order to obtain the knowledge (or information) transferred from other tasks to CVR prediction task i:

[0166]

[0167] Here, σ is the sigmoid function, also called the logistic function, used for the output of hidden layer neurons. Its value range is (0,1), and it maps a real number to the interval (0,1), which can be used for binary classification. f and b fThe first selector defines the information selection projection vector, which is a hyperparameter. An initial value for the information selection projection vector is set before model training, and its specific value is determined after training. In this embodiment, the multi-head attention mechanism captures more information from other tasks, enabling rapid, comprehensive, and effective information integration. The first selector then uses the information selection projection vector to filter the integrated information, removing information irrelevant to the first task to obtain the aforementioned first information. Information transfer loss can be derived based on transfer knowledge. The task loss (loss without information transfer) and the information transfer loss are used as the final task loss. Model learning based on the task loss ultimately determines the specific value of the information selection projection vector. Information selection is achieved by supervising the learning process of the information selection projection vector through vector projection transformation. Furthermore, the aggregation module and the first selector filter task information from tasks other than the first task, rapidly achieving effective information transfer between multiple tasks and helping the current task improve its performance by utilizing task information from other tasks.

[0168] It is understandable that after directly integrating the task information of multiple related tasks other than the first task to obtain integrated information, the specific process of using information selection projection vector to obtain the first information from the integrated information can be referred to the above-mentioned specific process of integrating aggregated information and then selecting information, which will not be repeated here.

[0169] 332. The multi-task processing device processes the first information and shared features through the first task network to obtain the output result of the first task.

[0170] In this embodiment, the first task network obtains the output result of the first task based on the first information of the migration and the shared features, effectively utilizing the migration information to improve the accuracy of the output result of the first task.

[0171] As one implementation method, step 332 above specifically includes the following steps:

[0172] The multi-task processing device processes the shared features and the first information through the first processing tower to obtain the first output data and the second output data.

[0173] Specifically, the shared features of the input are processed by a first processing tower to obtain first output data. The first information of the input is processed by a second processing tower to obtain second output data.

[0174] The multitasking device processes the first output data and the second output data through the first prediction module to obtain the output result of the first task.

[0175] In this embodiment, a first processing tower is used to process the shared features and the first information in relation to the first task to obtain first output data and second output data; then, a first prediction module is used to obtain the output result of the first task based on the first output data and the second output data; the first processing tower and the first prediction module focus on the first task, which can improve the performance of the first task.

[0176] As one implementation method, the aforementioned multiple related tasks also include a second task, and the aforementioned multi-task processing method further includes the following steps:

[0177] The multi-task processing device processes the shared features of the corresponding second task through a second processing module to obtain the output result of the second task. The second processing module corresponds to the second task. The plurality of processing modules includes the second processing module.

[0178] In this embodiment of the application, the second processing module processes the shared features input to the second processing module to obtain the output result of the second task. The second processing module can complete the second task without transferring the task information of other tasks.

[0179] In this embodiment, experiments on two tasks are conducted using the AliExpress dataset, with the CVR prediction task exemplified by the CTCVR prediction task. This dataset consists of real user logs from the AliExpress e-commerce platform. Data from four countries are used: the Netherlands (NL), Spain (ES), France (FR), and the United States of America (US). The dataset includes two binary classification tasks: click-through rate (CTR) prediction and click-to-conversion rate (CTCVR) prediction. For the AliExpress dataset, Single Task Learning (STL), MMOE, Entire Space Multi-task Model (ESMM), and Effective Knowledge Transfer for Multi-Task Recommendation Models (EKTM) based on MMOE are used for training. The MMOE model consists of eight shared single-layer fully connected networks as expert networks. The learning rate is set to 1e-3, the weight decay to 1e-6, and the batch size to 4096. The loss weight α is used in a grid search within the range [0.01, 0.1, 1.0], and the number of heads is used in a grid search within the range [1, 2, 4, 8]. The area under the curve (AUC) is used as the evaluation metric, which measures the probability that a positive sample is ranked higher than a randomly selected negative sample. Table 1 below shows the training methods for the four models described above.

[0180] Table 1 Comparison of Experimental Results for Various Methods

[0181]

[0182]

[0183] Analysis of Table 1 leads to the following conclusions: EKTM can enhance multi-task learning backbone networks, improve task performance compared to single-task learning methods and other multi-task learning methods, and effectively solve the negative transfer problem.

[0184] The above combination Figures 2B to 3This application describes the multitasking method provided in the embodiments of this application. In the various embodiments of this application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0185] The following is a detailed description of the multi-task processing model training method provided in the embodiments of this application.

[0186] The model training method described above is used to train the multi-task processing model mentioned earlier. This training method can be executed by a model training device, or by a chip or module within the model training device.

[0187] refer to Figure 4 , Figure 4 The present application provides a flowchart illustrating a multi-task processing model training method. Taking a model training device as an example, the execution entity of the multi-task processing model training method includes some or all of the following steps:

[0188] 401. Model training equipment acquires training data.

[0189] The training data is related to several relevant tasks.

[0190] 402. The model training device extracts features from the training data through a shared module to obtain shared features for multiple related tasks.

[0191] Specifically, the shared features of multiple related tasks include the shared features of each task. The shared features of each task can be the same or different features, and can be set according to actual needs.

[0192] 403. The model training device obtains information related to the first task from the task information of tasks other than the first task among multiple related tasks through the first processing module.

[0193] The first processing module mentioned above corresponds to the first task.

[0194] 404. The model training device processes the shared features and information related to the first task input into the first processing module through the first processing module to obtain the third output data and the fourth output data.

[0195] 405. The model training device obtains the first loss value for the first task based on the third and fourth output data.

[0196] Specifically, steps 404 and 405 allow the incorporation of information related to the first task, transferred from other tasks, into the loss calculation stage. This ensures the positive impact of information transfer on the current model training. Positive effects from the introduced information are amplified, while negative effects are ignored. Furthermore, in this embodiment, various loss mechanisms can be employed to guarantee improved training accuracy after transfer.

[0197] 406. The model training device adjusts the model parameters of the multi-task processing model based on the loss values ​​of multiple related tasks, including the first loss value.

[0198] Specifically, after obtaining the loss values ​​of each task in multiple related tasks, the final loss value of the model is obtained based on the loss values ​​of multiple tasks (the specific organization of the loss values ​​is not limited). The model parameters are adjusted based on the final loss value, and a multi-task processing model can be obtained when the training stopping condition is met.

[0199] In this embodiment, the model training device utilizes a shared module to extract shared features of multiple tasks from the training data. A first processing module corresponding to the first task processes the task information of other tasks besides the first task among multiple related tasks to obtain information related to the first task. The first processing module then processes the shared features and the information related to the first task to obtain third and fourth output data, and based on these, obtains the first loss value for the first task. Finally, the model parameters are adjusted based on the loss values ​​of multiple tasks (including the first loss value). It is evident that this embodiment sets up a network structure (e.g., the first processing module) to obtain information related to its own task from other task information. This allows the task information of other tasks to be incorporated into the task's loss, helping the model better learn the current task. After training, information from other tasks can be explicitly transferred, which is beneficial for improving the task performance of the first task and effectively balancing the task performance of multiple tasks in the multi-task processing model. Furthermore, due to the aforementioned network structure, parameter conflicts do not occur during model training, effectively improving the model performance of the multi-task processing model.

[0200] As one implementation, the task information of the tasks other than the first task among the multiple related tasks in step 403 above is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the multiple related tasks.

[0201] As one implementation method, the above training method also includes the following steps:

[0202] The model training device uses an aggregation module to aggregate task information from multiple related tasks, excluding the first task, to obtain aggregated information.

[0203] The model training device obtains information related to the first task based on aggregated information through the first processing module.

[0204] In this embodiment of the application, information aggregation can reduce the data dimension of task information for tasks other than the first task, improve computational efficiency and accuracy, and quickly and accurately select information related to the first task.

[0205] As one implementation method, step 403 above specifically includes the following steps:

[0206] 431. The model training device obtains integrated information from task information of multiple related tasks other than the first task through a multi-head attention mechanism using a first selector.

[0207] Specifically, information integration can be performed by integrating task information from multiple related tasks other than the first task, or by integrating aggregated information.

[0208] 432. The model training device processes the integrated information and the information selection projection vector through the first selector to obtain information related to the first task.

[0209] In this embodiment, the model training device uses a multi-head attention mechanism via a first selector to integrate task information from tasks other than the first task to obtain integrated information. The multi-head attention mechanism captures more information from other tasks, enabling rapid, comprehensive, and effective information integration. The model training device then uses the first selector to filter the integrated information using an information selection projection vector, removing information irrelevant to the first task to obtain the aforementioned information related to the first task.

[0210] As one implementation method, step 404 above specifically includes the following steps:

[0211] 441. The model training device processes the shared features through the first processing tower to obtain the third output data.

[0212] 442. The model training device processes the information related to the first task through the first processing tower to obtain the fourth output data.

[0213] In this embodiment of the application, the model training device uses a first processing tower to process shared features and information related to the first task to obtain first output data and second output data.

[0214] As one implementation method, step 405 above specifically includes the following steps:

[0215] 451. The model training device obtains the task loss value for the first task based on the third output data and the first expected value. The first expected value is the expected value of the third output data.

[0216] 452. The model training device obtains the information transfer loss value for the first task based on the fourth output data and the second expected value. The second expected value is the expected value of the fourth output data.

[0217] For example, the first and second expected values ​​can be used as labels for the training data to supervise the training process.

[0218] 453. The model training device determines the first loss value based on the information transfer loss value and the task loss value.

[0219] In this embodiment, the model training device determines the first loss value based on the information transfer loss value and the task loss value. Since the information transfer loss is taken into account, the loss expression of the first task is more accurate, which is beneficial to improving the task performance of the first task.

[0220] The loss calculation process is explained below:

[0221] Incorporating transfer information into loss calculation ensures that this information guides parameter learning in a favorable direction. (Reference) Figure 2E , Figure 2E There are two types of arrows originating from the aggregation module and the sharing module, leading to different task processing modules. The thicker arrows originating from the aggregation module represent transferred knowledge, from which the information transfer loss Lt can be derived. The thinner arrows originating from the sharing module represent shared features input to different processing modules, from which the task loss Lo (i.e., the task loss without information transfer) can be derived. Taking a recommender system as an example, the overall loss for all tasks is formulated as follows:

[0222]

[0223] Among them, L ctr is the loss of the CTR prediction task, k is the total number of CVR prediction tasks, and the hyperparameter α∈(0,1) is used to balance the information transfer loss and the task loss.

[0224] The objective function for CTR prediction can be binary cross-entropy loss:

[0225] L ctr =-ylog(P ctr )-(1-y)log(1-P ctr )

[0226] For CVR prediction tasks, training data can, for example, be metrics such as viewing time or time spent on a profile, and the information transfer loss Lt and task loss Lo can be defined using mean squared error (MSE) or mean absolute error (MAE). As another example, if the training data is behavioral data such as "likes" or "shares," the information transfer loss Lt and task loss Lo can be defined using the view-to-click conversion rate loss Lctcvr. Specifically:

[0227] L ctcvr =-yzlog(P ctcvr )-(1-yz)log(1-P ctcvr )

[0228] Where y represents the label indicating the location where the item was clicked, and z represents the label indicating the location where the item converted. pctcvr is the product of the CTR prediction and the CVR prediction. The pctcvr differs for Lt and Lo; the Pctcvr for Lo comes from the data stream without information migration, i.e. Figure 2E The thinner arrows originate from the shared underlying network. Lo's Pctcvr originates from data streams with information migration, that is... Figure 2E The thicker arrow originating from the aggregation module.

[0229] As one implementation method, the aforementioned related tasks also include a second task, and the aforementioned training method further includes the following steps:

[0230] The model training device processes the shared features through the second processing module to obtain the fifth output data.

[0231] The model training device derives a second loss value for the second task based on the fifth output data and the third expected value. The second loss value is included in the loss values ​​for multiple related tasks.

[0232] The second processing module corresponds to the second task. The third expected value is the expected value of the fifth output data.

[0233] In this embodiment, the shared features input to the second processing module are processed by the second processing module to obtain the fifth output data, and then the second loss value is obtained based on the fifth output data and the third expected value. It is evident that the second processing module can complete the second task without transferring information from other tasks; therefore, information transfer loss does not need to be considered when determining the second loss value.

[0234] For a detailed description of each step in the training method, please refer to the relevant records of the multi-task processing method mentioned above, which will not be repeated here.

[0235] The above combination Figure 2E and Figure 4 This application describes the model training method provided in the embodiments. In the various embodiments of this application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0236] The methods of the embodiments of this application have been described in detail above. The following is a combination of... Figures 5 to 7 The apparatus provided in the embodiments of this application is described in detail. It should be understood that the description of the apparatus embodiments corresponds to the description of the method embodiments. Therefore, for content not described in detail, please refer to the method embodiments above. For the sake of brevity, it will not be repeated here.

[0237] Figure 5 A schematic diagram of a multitasking device provided for embodiments of this application, wherein, Figure 5 The multitasking device shown can be used to achieve the above. Figure 3 The multitasking method embodiments shown herein can achieve the beneficial effects of the aforementioned multitasking method embodiments. A multitasking device can be understood as a carrier running a multitasking model; a detailed description of the multitasking model can be found above. In the embodiments of this application, the multitasking device can be an electronic device or a module (such as a chip) applied in an electronic device.

[0238] like Figure 5 As shown, the multitasking device 500 includes a first acquisition module 510, a first extraction module 520, and a first determination module 530. The multitasking device 500 is used to implement the functions of the above-described multitasking method embodiments. Alternatively, the multitasking device 500 may include modules for implementing any function or operation of the above-described multitasking method embodiments, and these modules may be implemented wholly or partially through software, hardware, firmware, or any combination thereof.

[0239] When the multitasking device 500 is used to implement the functions of the above-described multitasking method embodiment, the first acquisition module 510 is used to acquire input data. The first extraction module 520 is used to extract features from the input data through the sharing module to obtain shared features of multiple related tasks. The first determination module 530 is used to obtain first information related to the first task from the task information of tasks other than the first task among the multiple related tasks through the first processing module, and to obtain the output result of the first task based on the first information and the shared features. The input data is related to multiple related tasks. The first processing module corresponds to the first task.

[0240] In this embodiment, the multi-task processing device 500 utilizes a shared module to extract shared features of multiple tasks from input data; and through a first processing module corresponding to the first task, processes the task information of other tasks besides the first task among multiple related tasks to obtain first information related to the first task. Then, based on the first information and the shared features input to the first processing module, the output result of the first task is obtained. It can be seen that this embodiment sets up a network structure (e.g., a first processing module) to obtain information related to its own task from information of other tasks, which can explicitly transfer information from other tasks, thus improving the task performance of the first task and effectively balancing the task performance of multiple tasks in the multi-task processing model.

[0241] As one implementation, the aforementioned multiple related tasks also include a second task. The first determining module 530 is further used to process the shared features through a second processing module to obtain the output result of the second task. The second processing module corresponds to the second task.

[0242] In one implementation, the first processing module includes a first selector and a first task network. Specifically, the first determining module 530 is used to obtain integrated information from task information of multiple related tasks (excluding the first task) based on a multi-head attention mechanism using the first selector, and to obtain the first information based on the integrated information and the selected projection vector. The first information and shared features are then processed by the first task network to obtain the output result of the first task.

[0243] In one implementation, the aforementioned first task network includes a first processing tower and a first prediction module. Specifically, the first determining module 530, in processing the first information and shared features through the first task network to obtain the output result of the first task, processes the shared features and the first information through the first processing tower to obtain first output data and second output data respectively. The first prediction module then processes the first output data and the second output data to obtain the output result of the first task.

[0244] As one implementation, the task information of the tasks other than the first task among the aforementioned related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the related tasks.

[0245] As one implementation, the aforementioned multi-task processing model further includes an aggregation module. The aforementioned multi-task processing device 500 also includes a first aggregation module, which is used to aggregate task information from multiple related tasks (excluding the first task) through the aggregation module to obtain aggregated information. The first determination module 530 is used to obtain first information based on the aggregated information through the first processing module.

[0246] As one implementation method, the aforementioned related tasks include click-through rate prediction tasks, conversion rate prediction tasks, payment rate prediction tasks, or activation rate prediction tasks.

[0247] As one implementation, the aforementioned shared modules include a shared underlying network or an expert network equipped with gating mechanisms.

[0248] As one implementation method, the aforementioned shared module also includes a feature transformation module, which performs feature transformation on the input data to reduce the dimensionality of the input data, save computing resources, and improve the computational efficiency of the model.

[0249] Figure 6 A schematic diagram of the structure of a model training device provided for an embodiment of this application, wherein, Figure 6 The model training device shown can be used to achieve the above. Figure 4 The model training method embodiment shown herein also achieves the beneficial effects of the model training method embodiment described above. The model training device is used to train a multi-task processing model; a detailed description of the multi-task processing model can be found above. In the embodiments of this application, the model training device can be an electronic device or a module (such as a chip) applied in an electronic device.

[0250] like Figure 6 As shown, the model training device 600 includes a second acquisition module 610, a second extraction module 620, a second determination module 630, and an adjustment module 640. The model training device 600 is used to implement the functions of the above-described model training method embodiments. Alternatively, the model training device 600 may include modules for implementing any function or operation of the above-described model training method embodiments, and these modules may be implemented wholly or partially through software, hardware, firmware, or any combination thereof.

[0251] When the model training device 600 is used to implement the functions of the above-described model training method embodiment, the second acquisition module 610 is used to acquire training data. The second extraction module 620 is used to extract features from the training data through the sharing module to obtain shared features of multiple related tasks. The second determination module 630 is used to obtain information related to the first task from the task information of tasks other than the first task among the multiple related tasks through the first processing module; and to process the shared features and information related to the first task input to the first processing module through the first processing module to obtain third output data and fourth output data respectively; and to obtain a first loss value of the first task based on the third output data and fourth output data. The adjustment module 640 is used to adjust the model parameters of the multi-task processing model based on the loss values ​​of the multiple related tasks, where the loss values ​​of the multiple related tasks include the first loss value. The training data is related to the multiple related tasks. The above-described first processing module corresponds to the first task.

[0252] In this embodiment, the model training device 600 utilizes a shared module to extract shared features of multiple tasks from the training data; and through a first processing module corresponding to the first task, processes the task information of other tasks besides the first task among multiple related tasks to obtain information related to the first task. Then, the first processing module processes the shared features and the information related to the first task respectively to obtain third and fourth output data, and then obtains the first loss value of the first task based on the third and fourth output data. Finally, the model parameters are adjusted based on the loss values ​​of multiple tasks (including the first loss value). It can be seen that this embodiment sets up a network structure (e.g., the first processing module) to obtain information related to its own task from information of other tasks. This allows the task information of other tasks to be introduced into the task's loss, helping the model better learn the current task. After training, information from other tasks can be explicitly transferred, which is beneficial to improving the task performance of the first task and thus effectively balancing the task performance of multiple tasks in the multi-task processing model.

[0253] In one implementation, the first processing module includes a first selector. Specifically, the second determining module 630, in obtaining information related to the first task from task information of tasks other than the first task among multiple related tasks through the first processing module, is used to: obtain integrated information from task information of tasks other than the first task among multiple related tasks based on a multi-head attention mechanism using the first selector; and process the integrated information and the information selection projection vector using the first selector to obtain information related to the first task.

[0254] In one implementation, the first processing module further includes a first task network, which includes a first processing tower. Specifically, the second determining module 630, in processing the shared features and information related to the first task through the first processing module to obtain third output data and fourth output data, is used to: process the shared features through the first processing tower to obtain third output data; and process the information related to the first task through the first processing tower to obtain fourth output data.

[0255] As one implementation, the second determining module 630, in obtaining the first loss value of the first task based on the third output data and the fourth output data, is specifically used for: obtaining the task loss value of the first task based on the third output data and the first expected value. The first expected value is the expected value of the third output data. Obtaining the information migration loss value of the first task based on the fourth output data and the second expected value. The second expected value is the expected value of the fourth output data. Determining the first loss value based on the information migration loss value and the task loss value.

[0256] As one implementation, the aforementioned multiple related tasks also include a second task, and the second determining module 630 is further configured to: process the shared features through the second processing module to obtain fifth output data; obtain a second loss value for the second task based on the fifth output data and a third expected value; the loss value of the multiple related tasks includes the second loss value; wherein, the second processing module corresponds to the second task; and the third expected value is the expected value of the fifth output data.

[0257] As one implementation, the task information of the tasks other than the first task among the aforementioned related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the related tasks.

[0258] As one implementation, the aforementioned multi-task processing model further includes an aggregation module, and the aforementioned model training device further includes a second aggregation module. The second aggregation module is used to aggregate task information of multiple related tasks other than the first task through the aggregation module to obtain aggregated information. The second determination module 630 is used to obtain information related to the first task based on the aggregated information through the first processing module.

[0259] As one implementation method, the aforementioned related tasks include click-through rate prediction tasks, conversion rate prediction tasks, payment rate prediction tasks, or activation rate prediction tasks.

[0260] As one implementation, the aforementioned shared modules include a shared underlying network or an expert network equipped with gating mechanisms.

[0261] As one implementation method, the aforementioned shared module also includes a feature transformation module, which performs feature transformation on the training data to reduce the dimensionality of the training data, save computing resources, and improve the training efficiency of the model.

[0262] refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; the electronic device 700 includes a memory 701, a processor 702, a communication interface 704, and a bus 703. The memory 701, processor 702, and communication interface 704 are interconnected via the bus 703. It should be understood that this application does not limit the number of processors and memories in the electronic device 700.

[0263] Optionally, the aforementioned electronic device 700 also includes a display screen (not shown), which is connected to the memory 701, processor 702, and communication interface 704 via a bus 703. The display screen is used to output information and interact with the user, such as through voice output or display output.

[0264] The memory 701 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 701 may store programs. When the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 704 are used to execute various steps of the multitasking processing method of any embodiment of this application or various steps of the model training method of any embodiment of this application. For example, the communication interface 704 is used to implement the functions of the first acquisition module 510 or the second acquisition module 610 described above, and the processor 702 is used to implement the functions of the first determination module 530 or the second determination module 630 described above.

[0265] The processor 702 may be a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute related programs to achieve the functions required by the units in the multi-task processing device or model training device of any embodiment of this application, or to execute the multi-task processing method or model training method of any embodiment of this application.

[0266] The processor 702 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the multi-task processing method or model training method in any embodiment of this application can be completed by the integrated logic circuits in the hardware of the processor 702 or by instructions in software form. The processor 702 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the multi-task processing method or model training method, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the multi-task processing method or model training method in conjunction with any embodiment of this application can be directly embodied in the execution by the hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 701. The processor 702 reads the information in the memory 701 and, in conjunction with its hardware, performs the functions required by the units included in the multi-task processing device or model training device of any embodiment of this application, or executes the multi-task processing method or model training method of any embodiment of this application.

[0267] The communication interface 704 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the electronic device 700 and other devices or communication networks. For example, input data or training data can be acquired through the communication interface 704.

[0268] Bus 703 may include a pathway for transmitting information between various components of electronic device 700 (e.g., memory 701, processor 702, communication interface 704).

[0269] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0270] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0271] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0272] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be read-only memory (ROM), random access memory (RAM), or magnetic media, such as floppy disks, hard disks, magnetic tapes, magnetic disks, or optical media, such as digital versatile discs (DVDs), or semiconductor media, such as solid state disks (SSDs).

[0273] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-task processing method, characterized in that, The method is applied to a multi-task processing model, which includes a shared module and multiple processing modules. The multi-task processing model is used to execute multiple related tasks, and the multiple processing modules correspond to the multiple related tasks, including a first task. The method includes: Obtain input data, which is related to the plurality of related tasks; The shared module extracts features from the input data to obtain shared features for the multiple related tasks. The first processing module obtains first information related to the first task from the task information of the tasks other than the first task among the multiple related tasks, and obtains the output result of the first task based on the first information and the shared features; the first processing module corresponds to the first task.

2. The method according to claim 1, characterized in that, The plurality of related tasks also include a second task, and the method further includes: The shared features are processed by a second processing module to obtain the output result of the second task. The second processing module corresponds to the second task.

3. The method according to claim 1 or 2, characterized in that, The first processing module includes a first selector and a first task network. The step of obtaining first information related to the first task from task information of tasks other than the first task among the plurality of related tasks, and obtaining the output result of the first task based on the first information and the shared features, includes: The first selector obtains integrated information from task information of tasks other than the first task among the multiple related tasks based on a multi-head attention mechanism, and obtains the first information based on the integrated information and the information selection projection vector; The first task output is obtained by processing the first information and the shared features through the first task network.

4. The method according to claim 3, characterized in that, The first task network includes a first processing tower and a first prediction module. The step of processing the first information and the shared features through the first task network to obtain the output result of the first task includes: The shared features are processed by the first processing tower to obtain the first output data; The first information is processed by the first processing tower to obtain the second output data; The first prediction module processes the first output data and the second output data to obtain the output result of the first task.

5. The method according to claim 4, characterized in that, The task information of the tasks other than the first task among the multiple related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the multiple related tasks.

6. The method according to any one of claims 1-5, characterized in that, The multi-task processing model further includes an aggregation module, and the method further includes: The aggregation module aggregates the task information of tasks other than the first task from the plurality of related tasks to obtain aggregated information. The first information is obtained by the first processing module based on the aggregated information.

7. A method for training a multi-task processing model, characterized in that, The multi-task processing model is used to execute multiple related tasks, the multiple related tasks including a first task; the multi-task processing model includes a shared module and multiple processing modules, the multiple processing modules corresponding to the multiple related tasks; the method includes: Acquire training data, which is related to the plurality of related tasks; The shared module extracts features from the training data to obtain shared features for the multiple related tasks; The first processing module obtains information related to the first task from the task information of the tasks other than the first task among the plurality of related tasks; the first processing module corresponds to the first task. The first processing module processes the shared features and the information related to the first task to obtain the third output data and the fourth output data, respectively. The first loss value of the first task is obtained based on the third output data and the fourth output data; The model parameters of the multi-task processing model are adjusted based on the loss values ​​of the multiple related tasks, wherein the loss values ​​of the multiple related tasks include the first loss value.

8. The method according to claim 7, characterized in that, The first processing module includes a first selector. The step of obtaining information related to the first task from task information of tasks other than the first task among the plurality of related tasks via the first processing module includes: The first selector obtains integrated information from the task information of tasks other than the first task among the multiple related tasks based on a multi-head attention mechanism. The integrated information and the information selection projection vector are processed by the first selector to obtain the information related to the first task.

9. The method according to claim 8, characterized in that, The first processing module further includes a first task network, which includes a first processing tower. The step of processing the shared features and the information related to the first task through the first processing module to obtain third output data and fourth output data includes: The third output data is obtained by processing the shared features through the first processing tower; The fourth output data is obtained by processing the information related to the first task through the first processing tower.

10. The method according to any one of claims 7-9, characterized in that, The process of obtaining the first loss value for the first task based on the third output data and the fourth output data includes: Based on the third output data and the first expected value, the task loss value of the first task is obtained; Based on the fourth output data and the second expected value, the information migration loss value of the first task is obtained; The first loss value is determined based on the information migration loss value and the task loss value.

11. The method according to any one of claims 7-10, characterized in that, The plurality of related tasks also include a second task, and the method further includes: The shared features are processed by the second processing module to obtain the fifth output data. The second processing module corresponds to the second task. The second loss value of the second task is obtained based on the fifth output data and the third expected value; the loss values ​​of the multiple related tasks include the second loss value.

12. The method according to any one of claims 7-11, characterized in that, The task information of the tasks other than the first task among the multiple related tasks is the output vector of any network layer of the processing tower corresponding to the tasks other than the first task among the multiple related tasks.

13. The method according to any one of claims 7-12, characterized in that, The multi-task processing model further includes an aggregation module, and the method further includes: The aggregation module aggregates the task information of tasks other than the first task from the plurality of related tasks to obtain aggregated information. The first processing module obtains the information related to the first task based on the aggregated information.

14. A multitasking device, characterized in that, The device includes a unit or module for performing the multitasking method according to any one of claims 1-6.

15. A model training device, characterized in that, The device includes a unit or module for performing the multi-task processing model training method according to any one of claims 7-13.

16. An electronic device, characterized in that, The method includes a processor and a memory, wherein the processor and the memory are connected together, wherein the memory is used to store program code, and the processor is used to call the program code to perform the method as described in any one of claims 1-13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method as described in any one of claims 1-13.

18. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-13.

Images