Multi-task learning method and device, computer equipment and program product

Abstract

The invention is suitable for the technical field of machine learning, and provides a multi-task learning method and device, computer equipment and a program product, and the method comprises the steps: collecting original data needed by an initial task set, and generating a training sample to train a multi-task learning model; the initial task set comprises at least one core task and at least one auxiliary task; the multi-task learning model is trained by the following steps: extracting shared features from input data; evaluating the correlation among the tasks in the initial task set; taking at least one auxiliary task of which the correlation with the core task is lower than a threshold value as an irrelevant task to form a target task set with the core task; determining a sharing proportion of each task according to the correlation; and dynamically adjusting the loss weight or sharing proportion of the irrelevant tasks according to the verification result, or adjusting the irrelevant tasks in the target task set. According to the method, the task correlation is dynamically evaluated, the sharing proportion is adjusted, the irrelevant tasks are introduced, and the multi-task processing accuracy is improved.

Description

Technical Field

[0001] The present application belongs to the field of machine learning technology, and in particular, relates to a multi-task learning method, apparatus, computer equipment and program product. Background Art

[0002] In many fields such as image processing and natural language processing, multi-task learning (MTL) has been widely used as a technology that jointly trains multiple related tasks by sharing features and knowledge. Taking image processing as an example, by building a multi-task learning model, related tasks such as image classification, object detection, and image segmentation can be jointly trained, which can effectively improve the efficiency of model training and the processing performance of each task.

[0003] Traditional multi-task learning methods usually adopt fixed task combination design and rigid parameter sharing strategy, and build multi-task learning models only based on prior task correlation. This method is prone to model overfitting when the correlation between tasks is weak or there is negative transfer, resulting in poor processing accuracy of each task in actual application scenarios. For example, when a multi-task learning model processes machine translation and text classification tasks at the same time, it may overfit the features of the machine translation task, resulting in a decrease in the performance of the text classification task; for another example, when a multi-task learning model processes image classification and object detection tasks at the same time, due to the different requirements for features of the image classification task and the object detection task, if a fixed ratio of features is used to allocate features, the image classification task may interfere with the object detection task, resulting in a decrease in the processing accuracy of the object detection task.

[0004] In view of the above problems, there is an urgent need for a method that can effectively improve the accuracy of multi-task processing. Summary of the invention

[0005] The embodiments of the present application provide a multi-task learning method, apparatus, computer equipment, and program product, which can effectively improve the accuracy of multi-task processing.

[0006] In a first aspect, an embodiment of the present application provides a multi-task learning method, including: Collecting raw data required for an initial task set and generating training samples; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, which is used to process at least the core task in the initial task set; The multi-task learning model is trained in the following way: Extract shared features from input data; Evaluate the correlation between tasks in the initial task set; use at least one auxiliary task with a correlation lower than the threshold with the core task as an irrelevant task, and form a target task set with the irrelevant task and the core task; Determine the sharing ratio of features obtained by each task from the shared features according to the correlation between tasks in the target task set; Periodically verify the output results of tasks in the target task set. If the verification result does not meet the training end condition, dynamically adjust the loss weight or sharing ratio of the irrelevant task, or adjust the irrelevant tasks in the target task set.

[0007] In a second aspect, an embodiment of the present application provides a multi-task learning device, including: A sample generation module, configured to collect the original data required for the initial task set and generate training samples; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, and the multi-task learning model is at least used to process the core tasks in the initial task set; A model training module, configured to train a multi-task learning model, including: A shared feature extraction sub-module, configured to extract shared features from input data; A task association evaluation sub-module, configured to evaluate the correlation between tasks in the initial task set; use at least one auxiliary task with a correlation lower than the threshold with the core task as an irrelevant task, and form a target task set with the irrelevant task and the core task; A parameter sharing sub-module, configured to determine the sharing ratio of features obtained by each task from the shared features according to the correlation between tasks in the target task set; A dynamic adjustment sub-module, configured to periodically verify the output results of tasks in the target task set. If the verification result does not meet the training end condition, dynamically adjust the loss weight or sharing ratio of the irrelevant task, or adjust the irrelevant tasks in the target task set.

[0008] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in any item of the first aspect is implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method described in any item of the first aspect can be implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer device, the computer device is caused to execute the method described in any one of the above first aspects.

[0011] It can be understood that for the beneficial effects of the above second aspect to fifth aspect, reference can be made to the relevant descriptions in the above first aspect, and details are not repeated here.

[0012] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: By dynamically evaluating the relevance of multiple tasks, actively introducing auxiliary tasks with low or no relevance, and further determining the feature sharing ratio based on task relevance, the present application can achieve effective information sharing and regularization constraints between multiple tasks, reduce the overfitting risk of a single task, and improve the accuracy of multi-task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0014] Figure 1 is a flowchart of a multi-task learning method provided by an embodiment of the present application; Figure 2 is a structural diagram of a multi-task learning model provided by an embodiment of the present application; Figure 3 is a structural diagram of a multi-task learning device provided by an embodiment of the present application; Figure 4 is a structural diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0016] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0017] It should also be understood that the term "and / or" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0018] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0019] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0020] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0021] Most existing multi-task learning solutions adopt a fixed parameter sharing strategy, that is, using the same feature extraction network at the bottom layer, and then separating tasks by simple hard branches at the high layer. This method cannot dynamically adjust the ratio of shared features to specific features according to the actual relevance between tasks, which easily leads to negative transfer and model overfitting, and further results in poor processing effects of each task in practical applications. For example, in the field of natural language processing, a multi-task learning model can simultaneously process two common tasks, machine translation and text classification. If the data volume of the machine translation task is much larger than that of the text classification task, the model may overfit the features of the machine translation task, resulting in a decline in the performance of the text classification task. Another example is in image processing. A multi-task learning model can simultaneously process image classification and object detection tasks. The image classification task and the object detection task have different requirements for features. The image classification task pays more attention to global features, while the object detection task requires more refined local features. If a fixed ratio is used to allocate features, the image classification task may interfere with the object detection task, resulting in a decline in the processing accuracy of the object detection task.

[0022] In response to this, the present application proposes a multi-task learning method, which automatically calculates the sharing ratio of each task through dynamic task association evaluation, and at the same time introduces irrelevant tasks as regularization to effectively extract shared features and task-specific features, thereby effectively improving the accuracy of multi-task processing.

[0023] The technical solutions in the embodiments of the present application will be described in detail below.

[0024] Figure 1 The schematic flowchart of the multi-task learning method provided by the present application is shown. By way of example and not limitation, this method can be applied to a computer device. As Figure 1 shown, the method includes the following steps: S101, collect the original data required for the initial task set, and generate training samples based on the original data; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, and the multi-task learning model is at least used to process the core tasks in the initial task set.

[0025] It should be noted that the core tasks in the initial task set are important tasks in multi-task learning and are the main work of the model. The auxiliary tasks are tasks introduced to help the core tasks better complete, and can improve the performance of the core tasks by providing a regularization effect. For example, in image analysis, the core task can be image object classification, and the auxiliary tasks can be image segmentation, image texture classification, etc.; in natural language processing, the core task can be text classification, and the auxiliary tasks can be named entity recognition, etc.

[0026] Different initial task sets can be used for different application scenarios. For example, in medical image analysis, the core task may be the classification of tumor areas, while the auxiliary task may be the location of organ boundaries. By introducing organ location tasks, the model can learn richer anatomical structure information, thereby improving the accuracy of tumor classification. In autonomous driving scenarios, the core task may be vehicle detection, while the auxiliary task may be lane line detection. Through joint training, the model can better understand the road environment and reduce the false detection rate. Through flexible task settings, multi-task learning models can achieve efficient knowledge sharing and performance improvement in different application scenarios.

[0027] There are many ways to implement the setting of core tasks and auxiliary tasks in the initial task set. In some embodiments, they can be manually designed based on prior knowledge or domain experience between tasks, or they can be dynamically generated through data-driven automation methods. In one example, core tasks and / or auxiliary tasks can be dynamically generated through a machine learning model. For example, related auxiliary tasks can be generated based on core tasks to provide regularization and improve the performance of core tasks; for example, core tasks and auxiliary tasks can be automatically split according to user needs.

[0028] The multi-task learning method provided in this application is applicable to various technical fields and supports various types of raw data. In one embodiment, the raw data in S101 can be images, text, audio, video or multimodal data, which is specifically determined according to the data type required by the initial task set. Accordingly, when collecting raw data, the corresponding collection method can be selected according to the data type, and the acquisition of raw data from multiple sources is supported. Exemplarily, if the core task is tumor segmentation in medical images and the auxiliary task is organ positioning, the raw data can be CT images and pathological slice images, which can be collected by medical imaging equipment (such as CT scanners, defect scanners); if the core task is text sentiment analysis and the auxiliary task is named entity recognition, the raw data can be text data, which can be directly obtained or obtained by voice-to-text conversion. In another embodiment, when obtaining the raw data used to generate training samples, it can also be directly obtained from channels such as databases and public websites.

[0029] In one embodiment, during the process of generating training samples based on the original data, preprocessing such as data cleaning, data normalization, data standardization, and data augmentation can be performed on the original data. Among them, through the data cleaning stage, low-quality or redundant data can be removed according to different task types. For example, in image analysis, blurred and low-resolution image samples can be removed. The data normalization and standardization operations can be flexibly designed according to task requirements. For example, image data usually normalizes pixel values to the range [0, 1] and adopts channel-level standardization (such as ImageNet mean-variance normalization). The data augmentation strategy can be adapted to the task characteristics. For example, in the image classification task, operations such as random rotation, cropping, and color jitter are adopted, while in the object detection task, the coordinates of the image and the annotation box need to be adjusted synchronously. For multi-modal data (such as image-text pairs), cross-modal alignment processing can also be performed. For example, the image region is spatially associated with the corresponding text description, or the video and audio data are synchronized through timestamps. Through the above preprocessing, while eliminating noise interference, the training samples can effectively adapt to the heterogeneous input requirements of the multi-task model, and improve the generalization ability and robustness of the model in complex scenarios.

[0030] S102. Train a multi-task learning model.

[0031] Specifically, the multi-task learning model is trained in the following way: S1021. Extract shared features from the input data.

[0032] In multi-task learning, shared features refer to the feature representations commonly used in multiple tasks. These features can capture the common information between different tasks and provide a unified feature basis for all tasks. By learning the shared features of multiple tasks, on the one hand, the model can better adapt to different tasks and reduce overfitting. On the other hand, the model can more efficiently utilize the common information in the input data, thereby accelerating the training process. This step S1021 aims to extract the underlying and middle-level features of the input data and provide a unified feature basis for all tasks. In contrast to shared features, task-specific features refer to the features extracted and learned specifically for a certain task. These features are usually used to capture the unique information and patterns of the task.

[0033] In this step S1021, general features can be extracted from the input data to form a shared feature representation. The specific feature extraction method can be selected according to the task characteristics and data types in different fields. For example, in computer vision tasks, general features are usually extracted through a convolutional neural network (CNN). For instance, the first few convolutional blocks of ResNet or EfficientNet are used to capture low-level features such as edges, textures, and shapes of images. These features can provide a shared basic representation for tasks such as image classification, object detection, and image segmentation. In natural language processing tasks, general features can be extracted through a Transformer encoder (such as BERT or RoBERTa), using the self-attention mechanism to model the context dependencies between words. The generated semantic representation can support tasks such as text classification, named entity recognition, and sentiment analysis. For multimodal tasks (such as image-text joint analysis), the extraction of general features requires the combination of cross-modal fusion techniques. For example, the dual-encoder structure of CLIP is used to extract the features of images and texts respectively, and the shared representation space of the two is aligned through contrastive learning. In speech processing tasks, general features can be extracted through Mel spectrograms or WaveNet to capture the time-frequency characteristics of acoustic signals and provide a shared basis for tasks such as speech recognition and speech emotion classification. In some embodiments, for the requirements of specific fields, the extraction of general features can also be optimized by combining domain knowledge. For example, in medical image analysis, a 3D convolutional network is used to extract the spatial features of CT or MRI volume data, and deformable convolutions are combined to adapt to organ deformations. In industrial defect detection, traditional image processing algorithms (such as Canny edge detection) are combined with a deep feature extraction network to enhance the representation ability of surface textures and defect areas. The shared feature representation extracted in the above manner can not only effectively support the joint training of multiple tasks but also significantly reduce the consumption of computing resources during the inference phase and improve the deployment efficiency of the model. Moreover, by switching different shared feature extraction methods (specifically, different shared feature extraction modules can be switched), the model can be flexibly migrated to different application fields to improve the generalization ability of the model.

[0034] Specifically, the shared feature representation can be denoted as . Among them represents the feature transformation obtained through any one of the feature extraction methods. For example, the shared feature representation extracted by using a convolutional layer is as follows:

[0035] where L is the number of convolutional layers, is the activation function (such as ReLU) and are the weight and bias of the l-th layer respectively.

[0036] S1022. Evaluate the correlation between each task in the initial task set; take at least one auxiliary task with a correlation lower than the threshold with the core task as an irrelevant task, and form a target task set by combining the irrelevant task and the core task; This step S1022 aims to evaluate the correlation between each task and actively introduce auxiliary tasks with weak or no correlation with the core task. By using these auxiliary tasks to provide a regularization effect, it forces the extraction of features from different angles during shared feature extraction, enhancing the universality and robustness of the shared features.

[0037] Existing correlation calculation methods can be used to evaluate the correlation between tasks, such as gradient cosine similarity, Pearson correlation coefficient, Euclidean distance, mutual information, etc. In some embodiments, the collaborative or conflict relationship between tasks can be intuitively reflected through a task correlation matrix.

[0038] The following details the correlation evaluation method based on gradient cosine similarity: During each backpropagation process, calculate the gradient of the key features of the shared layer for each task i , and calculate the correlation between task i and task j through gradient cosine similarity. The specific formula is as follows:

[0039] where is the dot product (inner product) of the gradient vectors and , which represents the consistency of the gradient directions. The specific calculation method is:

[0040] and are the Euclidean norms (L2 norms) of the gradient vectors and respectively, that is:

[0041] Integrate all into the task correlation matrix R as shown below.

[0042]

[0043] According to the correlation between each task, tasks with a correlation lower than the threshold with the core task can be selected from the initial task set (such as The auxiliary tasks (in parentheses) are regarded as irrelevant tasks and together with the core tasks form the target task set. In one embodiment, the number of specifically selected irrelevant tasks can be set in various ways and can be dynamically adjusted. In one example, if the number of irrelevant tasks that can be selected in the initial task set is greater than or equal to the number of irrelevant tasks that need to be selected (hereinafter referred to as the preset number), then the preset number of irrelevant tasks can be selected and added to the target task set. In one example, if the number of irrelevant tasks that can be selected in the initial task set is less than the preset number, various methods can be used for processing. For example: only the irrelevant tasks that can be selected in the initial task set can be selected, or a certain number of auxiliary tasks can be adaptively generated and added to the initial task set to re-evaluate the relevance between tasks, or irrelevant tasks can be generated and added to the target task set to make the number of irrelevant tasks reach the preset number, or perturbations can be added to the auxiliary tasks in the initial task set to reduce the relevance between tasks, and so on. In one embodiment, for irrelevant tasks, further perturbations can be added to play a better regularization role, such as adding noise to the task-independent data, adding noise to the task-independent labels, etc.

[0044] In one embodiment, when determining the relevance to the core task, if there are more than two core tasks, auxiliary tasks with a relevance lower than the threshold to all core tasks can be selected as irrelevant tasks, or, according to actual needs, auxiliary tasks with a relevance lower than the threshold to at least one core task can be selected as irrelevant tasks.

[0045] S1023. Determine the sharing ratio of each task to obtain features from the shared features according to the relevance between the tasks in the target task set.

[0046] This step S1023 aims to achieve adaptive parameter sharing. Specifically, the core tasks and the tasks with high relevance to other tasks obtain a higher sharing ratio, and the irrelevant tasks obtain a lower sharing ratio. Exemplarily, the sharing ratio of the core tasks can be higher than that of the irrelevant tasks. To achieve this goal, various possible sharing ratio allocation methods can be adopted. Specifically, it includes strategies such as dynamic weight allocation based on task relevance, feature channel selection based on the attention mechanism, and sharing ratio adjustment based on gradient conflict.

[0047] Exemplarily, in the case of generating the task relevance matrix R in step S1022, a sharing ratio factor α can be generated for each task according to the task relevance matrix R, where . Specifically, a fully connected network and an activation function (sigmoid or softmax) are used to implement the calculation, and the formula is as follows:

[0048] Among them, and is a parameter, and is a sigmoid function or a softmax function.

[0049] In one embodiment, each task fuses the shared features and task-specific features through a dedicated branch layer (such as a fully connected layer, a convolutional layer, or a Transformer Decoder layer) to achieve a task-specific output feature representation.

[0050] After generating the shared scale factor, for task i, during feature partitioning, the shared feature part is:

[0051] And the task-specific features are obtained through the transformation function as follows:

[0052] Fusing the above two parts to obtain the task-specific representation of task i, and the fusion formula is:

[0053] where , and are the learnable parameters of the branch layer of task i.

[0054] Through the shared scale allocation in S1023, on the one hand, the core task and the tasks with high relevance to the core task obtain a higher shared scale, which is beneficial for the core task to give priority to using the global features, enhancing the performance of the core task, and optimizing knowledge transfer; on the other hand, the irrelevant tasks obtain a lower shared scale, and through the weak constraint of the shared features, the robustness of the model to noise data or distribution shift is improved.

[0055] S1024, periodically verify the output results of each task in the target task set. If the verification result does not meet the training end condition, dynamically adjust the loss weight or shared scale of the irrelevant tasks, or adjust the irrelevant tasks in the target task set.

[0056] Through this step S1024, the loss weight, shared scale of the irrelevant tasks, and the irrelevant tasks in the target task set can be dynamically adjusted.

[0057] In one embodiment, during the training process of the multi-task model, the total loss of the model can be obtained by weighted summation of the losses of each task in the target task set. Among them, different tasks can adopt different loss functions and loss weights, and usually the loss weight of the core task is greater than that of the irrelevant tasks. Exemplarily, the total loss can be calculated by the following formula:

[0058] Among them, L total is the total loss, L i and L j are the losses of task i and task j respectively, and λ i and μ j are the loss weights of task i and task j respectively.

[0059] The loss function of each task can be determined according to the task type. For example, for a classification task, cross-entropy loss can be adopted, and the specific loss function is as follows:

[0060] Another example is that for a regression task, mean squared error loss can be adopted, and the specific loss function is as follows:

[0061] In one embodiment, the network parameters can be updated according to the total loss of the model. Exemplarily, this process may include: Using the total loss to calculate the gradient by using the standard backpropagation algorithm, and updating the network parameters through gradient descent:

[0062] where represents all learnable parameters, is the learning rate.

[0063] In one example, when the gradient cosine similarity is used in S1022 to evaluate the correlation between tasks, the gradient information recorded during the backpropagation process can be fed back to the similarity calculation, so as to update the calculation strategies of the task correlation matrix R and the sharing ratio factor α in real time.

[0064] In one embodiment, the output results of each task can be periodically verified by using a validation set. Among them, the validation set can be generated based on the original data collected in step S101, or can be generated by additionally collecting data. In one example, when performing periodic verification, multiple verification metrics can be set for verification, such as accuracy, IoU, etc. According to the evaluation results, the loss weights and the sharing ratio of irrelevant tasks and the irrelevant tasks in the target task set can be dynamically adjusted. Specifically, if according to the verification results, it is found that any irrelevant task interferes with the core task (which can be determined by specific verification metrics), then its influence is reduced by dynamic adjustment, or measures such as removing the irrelevant task from the target task set, replacing the irrelevant task with other irrelevant tasks, or introducing at least one new irrelevant task into the target task set are adopted.

[0065] After completing the model training, input the actual data required for each task processing into the multi-task learning model, and the processing results of each task can be obtained.

[0066] So far, the description of Figure 1 the process shown above is completed.

[0067] Through Figure 1 the process shown above, the relevance of multi-tasks can be dynamically evaluated, auxiliary tasks with low or no relevance can be actively introduced, and the feature sharing ratio can be further determined based on task relevance, realizing effective information sharing and regularization constraints between tasks, reducing the risk of overfitting for a single task, and improving the accuracy of multi-task processing.

[0068] For the convenience of understanding Figure 1 the process shown above, the following takes multi-task learning based on image data as an example for detailed description: Input data: RGB image with a size of 224*224 Initial task set, which can be used for image analysis, image processing, and / or computer vision. Specifically, it can include: Task 1 (core task): Object classification (e.g., 10-class classification); Task 2 (core task): Image segmentation (pixel-level annotation); Task 3 (auxiliary task): Texture classification (such as rough, smooth, stripe classification), used to provide regularization constraints.

[0069] Multi-task learning model structure (see Figure 2 shown above): Data preprocessing module: 1 layer; Shared feature extraction module: Use a 6-layer convolutional neural network to extract shared features ; Dynamic task association evaluation module: Calculate the gradient cosine similarity to form a task relevance matrix R; Unrelated task injection module: Used to introduce unrelated tasks and perform perturbation design; Gating mechanism layer: 1 layer, calculate the sharing ratio ; Dynamic task branch module: Used to fuse shared features and task-specific features, design 2 layers of dedicated branches for each task, a total of 3×2 = 6 layers; Task output layer: Used to obtain the output of each task, 1 layer of output for each task, a total of 3 layers; Dynamic loss reconciliation module: Used to calculate the model loss, 1 layer; Backpropagation and parameter update module: Used for backpropagation and updating the parameters of the entire network; Verification and adjustment module: Used to implement model verification and dynamic adjustment of unrelated tasks.

[0070] The training process is summarized as follows: The preprocessed input image obtains shared features through the shared feature extraction module ; The dynamic task association evaluation module calculates the task correlation matrix R using gradient cosine similarity and introduces irrelevant tasks into the target task set; The gating mechanism layer calculates the sharing ratio of each task according to the task correlation matrix R ; For each task, the dynamic task branch module fuses the shared features and task-specific features to generate the final features ; The output layer of each task generates predictions based on the final features and calculates the losses respectively ; The dynamic loss reconciliation module calculates the total loss according to the loss weights of each task ; Backpropagation updates the parameters of the entire network, and at the same time feeds back to update R and the sharing ratio ; After regular verification, the loss weights, sharing ratios of irrelevant tasks or irrelevant tasks in the target task set are dynamically adjusted according to the verification metrics to achieve closed-loop feedback.

[0071] Next, in combination with Figure 3 the specific implementation steps of multi-task learning based on image data will be described.

[0072] S301, Collect the original image data required for the initial task set and generate training samples.

[0073] As the task settings given above, the initial task set includes two core tasks (Task 1 (core object classification) and Task 2 (image segmentation)) and one auxiliary task (Task 3 (texture classification)).

[0074] Exemplarily, this step can collect RGB images of 224×224 from a public image dataset, and the object classification, segmentation labels and texture information are annotated by experts. Through preprocessing such as unifying the image format and normalizing (using the formula ), the preprocessed image data together with the labels of each task form a training sample set: .

[0075] S302, Extract shared features from the preprocessed image data.

[0076] This step extracts general visual features from the preprocessed image data as shared features, providing the basic information common to all tasks. Specifically, the image can be input into a 6-layer convolutional network to extract features layer by layer: Low layer (layers 1-2): Extract edge, corner, and basic texture information. Middle layer (layers 3-4): Capture local structures and shapes. High layer (layers 5-6): Gradually extract some global semantic information while still retaining general features. Finally, the shared features are obtained. Among them Among them represents the forward propagation operation of the 6-layer convolutional network, and the output contains information such as edges, textures, and local shapes.

[0077] S303, evaluate the correlation between tasks in the initial task set, regard at least one auxiliary task with a correlation lower than the threshold with the core task as an irrelevant task, and form a target task set by combining the irrelevant task and the core task.

[0078] Exemplarily, this step S303 calculates the correlation between tasks in the initial task set based on the gradient cosine similarity. Specifically, it includes: during backpropagation, record the gradients of the object classification, image segmentation, and texture classification tasks in the shared layer. For example, for task 1 (object classification), task 2 (image segmentation), and task 3 (texture classification), record the corresponding gradients , , , calculate the gradient cosine similarity to form the correlation matrix R. Among them, the gradient calculation includes the following steps: 1) Perform forward propagation for each task to obtain the predicted output of the task; 2) According to the loss function of each task, calculate the loss of each task (for example, the classification task uses cross-entropy loss, and the regression task uses mean square error, etc.); 3) Through backpropagation, calculate the gradient of each task with respect to the shared feature layer. For task i, the gradient of the shared layer is . The formula for the gradient cosine similarity is as follows:

[0079] According to the gradient cosine similarity , the task correlation matrix R can be obtained. The element of this matrix represents the similarity between task i and task j. The task correlation matrix R is as follows:

[0080] Judge that if the texture classification task has a low correlation with other core tasks, for example, r 13 and r 23 are both less than the threshold τ, then regard the texture classification task as an irrelevant task and add it to the target task set.

[0081] S304. Determine the sharing ratio of each task to obtain features from the shared features according to the relevance between tasks in the target task set.

[0082] Exemplarily, the average relevance of task i can be calculated first according to the task relevance matrix R, and the calculation formula is as follows:

[0083] Then calculate the sharing ratio factor based on the average relevance. Specifically, it can be generated through a fully connected layer and the Sigmoid activation function to generate the sharing ratio, and the specific formula is as follows:

[0084] The calculated ranges from (0,1), indicating how many shared features task i needs. The larger the value of, the more shared features task i needs.

[0085] Exemplarily, after calculating the sharing ratio, the shared features and task-specific features of each task can be calculated respectively, and then the two are fused to obtain the final features of the task.

[0086] Specifically, taking task 1 (object classification) as an example, calculate the shared features and task-specific features respectively; among them, the shared features are mainly used to capture global semantic information, which helps to improve the classification accuracy; among them is a small convolutional network for extracting specific information of task 1. The shared features and task-specific features are fused through the ReLU activation function to obtain the final features of task 1. The final features of task 1 contain both shared information and retain task-specific features, which are suitable for classification tasks.

[0087] The same applies to task 2 and task 3. Finally, through ReLU fusion, the final features of each task are obtained .

[0088] The following is illustrated with a more specific example: For task 1 (object classification): If according to the calculation, the average relevance c of task 1 with other tasks 1 is very high, then the calculated value of α 1 will also be relatively high, for example, it may be 0.8. Based on this, when dividing features: Shared features of task 1:

[0089] Task-specific features of task 1:

[0090] After fusing the shared features and task-specific features, Task 1 mainly relies on the shared features, which is beneficial to capturing global semantic information.

[0091] For Task 3 (texture classification): If Task 3, which is an auxiliary task, is added to the target task set, then the correlation between this task and other tasks is relatively low, and the calculated value of α 3 will also be relatively low. For example, it may be 0.3. Based on this, when dividing the features: Shared features of Task 3:

[0092] Task-specific part:

[0093] Thus, after fusing the shared features and task-specific features, Task 3 mainly retains its own specific texture information, and at the same time, a small amount of shared features can also play a regularization role.

[0094] S305. Periodically verify the output results of each task in the target task set. If the verification result does not meet the training end condition, dynamically adjust the loss weight or sharing ratio of the irrelevant tasks, or adjust the irrelevant tasks in the target task set.

[0095] Exemplarily, the output layer of each task is designed as follows: The output layer of Task 1 uses softmax for 10-class classification; the output layer of Task 2 uses upsampling and softmax for pixel-level segmentation; the output layer of Task 3 uses softmax for texture classification. Specifically, Task 1 uses Softmax for 10-class classification, and the specific formula is as follows:

[0096] Task 2 uses upsampling and Softmax for pixel-level classification, and the specific formula is as follows:

[0097] Task 3 uses Softmax for texture classification, and the specific formula is as follows:

[0098] Exemplarily, the total loss of the multi-task learning model is calculated in the following way: Calculate the cross-entropy loss of each task separately, and use learnable weights σ 1 , σ 2 , σ 3 to weight and form the total loss . Specifically: Calculate the loss of Task 1 through the following loss function:

[0099] Calculate the loss of Task 2 through the following loss function:

[0100] Calculate the loss of Task 3 through the following loss function:

[0101] Use uncertainty weighting to calculate the total loss:

[0102] Exemplarily, according to the total loss Perform backpropagation to update the entire network parameters, and feedback new gradient information to update the task correlation matrix R and the sharing ratio.

[0103] Exemplarily, taking every 10 epochs (training rounds) as a cycle, evaluate the output performance of each task through the validation set. If the auxiliary task 3 (texture classification task) interferes greatly with the core tasks (Task 1, Task 2), then reduce the loss weight of Task 3 or adjust the sharing ratio ; if the regularization effect is not obvious, then consider adjusting the auxiliary task data acquisition strategy or introducing new irrelevant tasks.

[0104] This embodiment proves that the present application can effectively improve the robustness of the shared features, reduce the overfitting risk of the core tasks, and ultimately improve the processing performance of each task by introducing irrelevant task regularization.

[0105] In an alternative embodiment, the solution can be described in terms of a multi-task medical image analysis task, and the specific implementation process of integrating quantum computing in the present application is elaborated in detail. In this scenario, the initial task set is set as follows: Task 1 (core task) is to classify tumors in medical image data, that is, the tumor classification task; Task 2 (core task) is the image segmentation of the tumor region, that is, the tumor segmentation task; Task 3 (auxiliary task) is to identify the texture features in the image, that is, the tumor segmentation task, so as to assist in improving the accuracy of the core tasks. The steps include: Step (1) Perform standardized preprocessing on the medical image data, map and encode the gray values of the image pixels with the amplitude and phase of the quantum state to generate medical image data represented by quantum states; Step (2) Input the medical image data represented by quantum states generated in step (1) into a quantum convolutional neural network (QCNN). The QCNN includes a quantum convolutional layer, a quantum pooling layer, and a quantum fully connected layer, and perform convolutional operations on the medical image data represented by quantum states through quantum gate operations to extract quantum features of the tumor edge, texture, and shape; Step (3): Based on the quantum features extracted in step (2), calculate the quantum state entanglement measure between the tumor classification task, the tumor segmentation task, and the texture recognition task, quantify the correlation between tasks through the quantum mutual information value, and obtain the task correlation result; Step (4): According to the task correlation result obtained in step (3), dynamically adjust the shared feature layer ratio and loss function weight of multi-task training, and use quantum parallel computing to optimize the model parameters to complete the joint output of tumor classification, segmentation, and texture recognition.

[0106] Among them, the mapping encoding in step (1) specifically includes: normalizing the gray value of the image pixel to the amplitude of the quantum state, and representing the spatial position relationship of the pixels through phase encoding; using the superposition state of quantum bits to perform parallel representation on the entire image, and the state vector of each quantum bit corresponds to the quantum state information of the local area of the image.

[0107] Among them, the implementation of the quantum convolution layer in step (2) includes: applying a combination of Hadamard gates, CNOT gates, and parameterized rotation gates to the input quantum state medical image data in the quantum circuit to generate a quantum convolution kernel; through the superposition and entanglement characteristics of the quantum state, parallelly extract the edge and texture features of the tumor area in the image, and output a multi-channel quantum feature map.

[0108] Among them, the calculation of the quantum state entanglement measure in step (3) includes: constructing density matrices for the quantum feature states of the tumor classification task, the tumor segmentation task, and the texture recognition task respectively; calculating the joint entropy and conditional entropy between tasks, and determining the task correlation based on the quantum mutual information formula; when the mutual information value exceeds the preset threshold, it is determined that there is a strong quantum correlation between tasks, and the feature sharing weight needs to be increased.

[0109] Among them, the dynamic adjustment strategy in step (4) includes: for the first task pair (tumor classification and texture recognition) with a mutual information value higher than the threshold, increase the number of quantum bits of its shared quantum fully connected layer; for the second task pair (tumor segmentation and texture recognition) with a mutual information value lower than the threshold, introduce a quantum decoherence noise suppression module in the shared feature layer to reduce the interference between tasks.

[0110] Among them, the quantum parallel computing optimization in step (4) includes: in the backpropagation process, use the superposition state of quantum bits to calculate the multi-task gradients simultaneously, and accelerate the gradient update through the quantum amplitude amplification algorithm; adopt the quantum approximate optimization algorithm to search for the global optimal solution of the model parameters and reduce the number of training iterations.

[0111] Among them, the implementation method of the quantum pooling layer of QCNN is: perform a quantum measurement operation on the multi-channel quantum feature map output by the quantum convolution layer to compress the feature dimension; retain the key quantum state amplitude distribution information of the tumor area and discard the redundant phase components.

[0112] In the embodiments of the present application, by mapping and encoding the gray values of medical image pixels to the amplitudes and phases of quantum states, and utilizing the superposition state characteristics of qubits, the high-dimensional pixel space of traditional images can be compressed into a logarithmic number of qubits, significantly reducing the overhead of data storage and transmission. The quantum convolutional layer combines quantum gates such as Hadamard gates and CNOT gates, and uses the quantum superposition state to perform parallel processing on multi-region features of images (such as tumor edges and textures). A single operation can cover the feature extraction that traditional CNNs need to complete through multiple convolutional operations. When the operations of quantum gates act on the superposition state, all possible input states can be processed simultaneously. For example, the convolutional kernel for tumor edge detection needs to slide pixel by pixel in traditional computing, while quantum convolution completes the full-image convolution operation at once through the superposition state, and the reduction in time complexity comes from quantum parallelism.

[0113] In the embodiments of the present application, through the measurement of quantum state entanglement, the non-linear dependence relationship between tasks in the quantum feature space can be quantified. Based on the determination of task correlation using quantum mutual information, dynamically adjusting the shared feature ratio and loss weights can accelerate the convergence speed of model training and improve the final classification accuracy. In a quantum system, the entanglement degree of task feature states directly reflects their information sharing potential. For example, if there is strong entanglement between the feature states of the tumor classification task and the texture recognition task, it indicates that the two share quantum features at the microscopic level (such as sub-pixel level texture patterns), which are difficult to capture by traditional methods due to the limitation of the feature space dimension. Introducing a quantum decoherence suppression module for low-correlation tasks (such as tumor segmentation and texture recognition) can reduce the interference of irrelevant features.

[0114] In the embodiments of the present application, by using the quantum superposition state to calculate multi-task gradients simultaneously and combining with the quantum amplitude amplification algorithm, the gradient update speed of backpropagation can be improved. Traditional multi-task gradient calculation requires taking derivatives for each task and weighted summation, while the superposition state of qubits allows encoding the gradient directions of multiple tasks simultaneously, and completing the multi-gradient superposition calculation through a single quantum operation, with the time consumption being independent of the number of tasks.

[0115] The magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0116] Corresponding to the multi-task processing method described in the above embodiments, Figure 3 The structural block diagram of the multi-task processing device provided by the embodiments of the present application is shown. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown.

[0117] Referring to Figure 3 , the device includes: A sample generation module, configured to collect the raw data required for an initial task set and generate training samples; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, and the multi-task learning model is at least used to process the core tasks in the initial task set; A model training module, configured to train a multi-task learning model, including: A shared feature extraction sub-module, configured to extract shared features from input data; A task correlation evaluation sub-module, configured to evaluate the correlation between tasks in the initial task set; at least one auxiliary task with a correlation lower than a threshold with the core task is used as an irrelevant task, and the irrelevant task and the core task form a target task set; A parameter sharing sub-module, configured to determine the sharing ratio of features obtained by each task from the shared features according to the correlation between tasks in the target task set; A dynamic adjustment sub-module, configured to periodically verify the output results of tasks in the target task set. If the verification results do not meet the training end condition, dynamically adjust the loss weight or sharing ratio of the irrelevant tasks, or adjust the irrelevant tasks in the target task set.

[0118] It should be noted that for the information interaction, execution process, etc. between the above modules, since they are based on the same concept as the method embodiment of the present application, the specific functions and the technical effects brought by them can be specifically referred to the method embodiment part, and will not be elaborated here.

[0119] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiment and will not be elaborated here.

[0120] An embodiment of the present application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0121] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments can be implemented.

[0122] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal is enabled to execute the steps in the above method embodiments when executed.

[0123] Figure 4 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. As Figure 4 shown, the computer device in this embodiment includes: at least one processor 40 ( Figure 4 only one is shown in the figure), a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40. When the processor 40 executes the computer program 42, the steps in any of the above visual programming method embodiments are implemented.

[0124] The computer device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art can understand that Figure 4 this is only an example of a computer device and does not constitute a limitation on the computer device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0125] The so-called processor 40 may be a central processing unit (CPU), and the processor 40 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0126] In some embodiments, the memory 41 may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In some other embodiments, the memory 41 may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the computer device. Further, the memory 41 may also include both the internal storage unit and the external storage device of the computer device. The memory 41 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-described embodiment methods of the present application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the device / computer device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0128] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0129] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0130] In the embodiments provided in this application, it should be understood that the disclosed device / computer equipment and method can be implemented in other ways. For example, the device / computer equipment embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0131] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of this application, and should all be included in the protection scope of this application.

Claims

1. A multi-task learning method, characterized in that: include: Collect the raw data required for the initial task set and generate training samples; The initial task set includes at least one core task and at least one auxiliary task; The training samples are used to train a multi-task learning model, and the multi-task learning model is used to process at least the core tasks in the initial task set; The multi-task learning model is trained in the following way: Extract shared features from input data; Evaluate the correlation between the tasks in the initial task set; regard at least one auxiliary task whose correlation with the core task is lower than a threshold as an irrelevant task, and form a target task set with the irrelevant task and the core task; Determining, according to the correlation between the tasks in the target task set, a sharing ratio of features acquired by each task from the shared features; The output results of each task in the target task set are periodically verified. If the verification result does not meet the training end condition, the loss weight or sharing ratio of irrelevant tasks is dynamically adjusted, or the irrelevant tasks in the target task set are adjusted.

2. The method according to claim 1, characterized in that The evaluating the correlation between the tasks in the initial task set includes: The correlation between the tasks in the initial task set is calculated based on the gradient cosine similarity to generate a task correlation matrix.

3. The method according to claim 1, characterized in that The method further comprises: The total loss is obtained by weighted summing up the losses of each task in the target task set; wherein the loss weight of the core task is greater than the loss weight of the irrelevant task.

4. The method according to claim 1, characterized in that The adjusting of irrelevant tasks in the target task set includes: If any irrelevant task interferes with the core task, the irrelevant task is removed from the target task set; Alternatively, an irrelevant task is reselected from the initial task set to replace the irrelevant task.

5. The method according to claim 1, characterized in that The sharing ratio of core tasks in the target task set is higher than the sharing ratio of irrelevant tasks.

6. The method according to claim 1, characterized in that The step of treating at least one auxiliary task whose relevance to the core task is lower than a threshold as an irrelevant task comprises: If the number of auxiliary tasks in the initial task set whose relevance to the core task is lower than the threshold does not reach a preset number, all auxiliary tasks in the initial task set whose relevance to the core task is lower than the threshold are selected as irrelevant tasks; Alternatively, a certain number of auxiliary tasks are generated and added to the initial task set, and the correlation between the tasks is re-evaluated; Alternatively, a certain number of irrelevant tasks are generated so that the number of irrelevant tasks reaches a preset number; Alternatively, disturbances are added to other auxiliary tasks in the initial task set to reduce the correlation between tasks.

7. The method according to claim 1, characterized in that The method according to any one of claims 1 to 6, characterized in that the original data is image data, and the initial task set is used for image analysis, image processing and / or computer vision.

8. A multi-task learning device, characterized in that: include: The sample generation module is used to collect the original data required for the initial task set and generate training samples; The initial task set includes at least one core task and at least one auxiliary task; The training samples are used to train a multi-task learning model, and the multi-task learning model is used to process at least the core tasks in the initial task set; Model training module, used to train multi-task learning models, including: A shared feature extraction submodule, used to extract shared features from input data; The task correlation evaluation submodule is used to evaluate the correlation between the tasks in the initial task set; regard at least one auxiliary task whose correlation with the core task is lower than a threshold as an irrelevant task, and form a target task set with the irrelevant task and the core task; A parameter sharing submodule, used for determining a sharing ratio of features acquired by each task from the shared features according to the correlation between the tasks in the target task set; The dynamic adjustment submodule is used to periodically verify the output results of each task in the target task set. If the verification result does not meet the training end condition, the loss weight or sharing ratio of irrelevant tasks is dynamically adjusted, or the irrelevant tasks in the target task set are adjusted.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that When the computer program product is executed on a computer device, the computer device is caused to execute the method according to any one of claims 1 to 7.

Images