A multitask learning method, device, computer equipment and program product

Abstract

This application relates to the field of machine learning technology and provides a multi-task learning method, apparatus, computer device, and program product, including: collecting raw data required for an initial task set to generate training samples for training a multi-task learning model; the initial task set includes at least one core task and at least one auxiliary task; the multi-task learning model is trained by: extracting shared features from the input data; evaluating the correlation between tasks in the initial task set; including at least one auxiliary task whose correlation with the core task is below a threshold as an irrelevant task to form a target task set with the core task; determining the sharing ratio of each task based on the correlation; and dynamically adjusting the loss weight or sharing ratio of irrelevant tasks based on the validation results, or adjusting the irrelevant tasks in the target task set. This application improves the accuracy of multi-task processing by dynamically evaluating task correlation and adjusting the sharing ratio, and introducing irrelevant tasks.

Description

Technical Field

[0001] This application belongs to the field of machine learning technology, and in particular relates to a multi-task learning method, apparatus, computer equipment and program product. Background Technology

[0002] Multi-task learning (MTL), a technique that jointly trains multiple related tasks by sharing features and knowledge, has been widely applied in various fields such as image processing and natural language processing. Taking image processing as an example, by constructing a multi-task learning model, related tasks such as image classification, object detection, and image segmentation can be jointly trained, which can effectively improve the model training efficiency and the processing performance of each task.

[0003] Traditional multi-task learning methods typically employ fixed task combination designs and hard parameter sharing strategies, building multi-task learning models solely based on prior task relevance. This approach is prone to overfitting when the correlation between tasks is weak or negative transfer exists, leading to poor accuracy in real-world applications. For example, when simultaneously handling machine translation and text classification, a multi-task learning model may overfit the features of machine translation, resulting in decreased performance in text classification. Similarly, when simultaneously handling image classification and object detection, the different feature requirements of these two tasks mean that using a fixed feature allocation ratio can cause image classification to interfere with object detection, leading to decreased accuracy in object detection.

[0004] To address the above issues, there is an urgent need for a method that can effectively improve the accuracy of multitasking. Summary of the Invention

[0005] This application provides a multi-task learning method, apparatus, computer device, and program product, which can effectively improve the accuracy of multi-task processing.

[0006] Firstly, embodiments of this application provide a multi-task learning method, including:

[0007] Collect raw data required for an initial task set and generate training samples; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, which is used to process at least the core task in the initial task set.

[0008] The multi-task learning model is trained in the following manner:

[0009] Extract shared features from the input data;

[0010] Evaluate the correlation between tasks in the initial task set; identify at least one auxiliary task whose correlation with the core task is below a threshold as an irrelevant task, and combine the irrelevant task with the core task to form a target task set.

[0011] Based on the correlation between tasks in the target task set, determine the sharing ratio of features obtained by each task from the shared features;

[0012] The output results of each task in the target task set are periodically verified. If the verification results do not meet the training termination condition, the loss weight or sharing ratio of irrelevant tasks are dynamically adjusted, or the irrelevant tasks in the target task set are adjusted.

[0013] Secondly, embodiments of this application provide a multi-task learning device, including:

[0014] A sample generation module is used to collect the raw data required for the initial task set and generate training samples; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, which is used to process at least the core task in the initial task set.

[0015] The model training module, used to train multi-task learning models, includes:

[0016] The shared feature extraction submodule is used to extract shared features from the input data;

[0017] The task association evaluation submodule is used to evaluate the correlation between tasks in the initial task set; at least one auxiliary task whose correlation with the core task is lower than a threshold is regarded as an irrelevant task, and the irrelevant task and the core task are combined to form a target task set;

[0018] The parameter sharing submodule is used to determine the sharing ratio of features obtained by each task from the shared features based on the correlation between the tasks in the target task set.

[0019] The dynamic adjustment submodule is used to periodically verify the output results of each task in the target task set. If the verification result does not meet the training termination condition, the loss weight or sharing ratio of irrelevant tasks is dynamically adjusted, or the irrelevant tasks in the target task set are adjusted.

[0020] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the first aspects above.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the method described in any of the first aspects above.

[0022] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to perform the method described in any one of the first aspects above.

[0023] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0024] The beneficial effects of this application embodiment compared with the prior art are: by dynamically evaluating the correlation of multiple tasks, actively introducing auxiliary tasks with low or no correlation, and further determining the feature sharing ratio based on task correlation, this application can achieve effective information sharing and regularization constraints among multiple tasks, reduce the risk of overfitting of a single task, and improve the accuracy of multi-task processing. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating a multi-task learning method provided in an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of the structure of a multi-task learning model provided in one embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the structure of the multi-task learning device provided in the embodiments of this application;

[0029] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0032] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0033] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0034] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0036] Most existing multi-task learning schemes employ fixed parameter-sharing strategies, using the same feature extraction network at the bottom layer and then separating tasks using simple hard branches at the higher layers. This approach cannot dynamically adjust the ratio of shared and specific features based on the actual relevance between tasks, easily leading to negative transfer and model overfitting, resulting in poor performance for each task in practical applications. For example, in natural language processing, multi-task learning models can handle two common tasks simultaneously: machine translation and text classification. If the data volume for machine translation is much larger than that for text classification, the model may overfit the features of machine translation, leading to a decline in the performance of text classification. Similarly, in image processing, multi-task learning models can handle image classification and object detection simultaneously. However, image classification and object detection have different feature requirements; image classification focuses on global features, while object detection requires more refined local features. If a fixed ratio of feature allocation is used, image classification may interfere with object detection, leading to a decrease in the accuracy of object detection.

[0037] In response, this application proposes a multi-task learning method that automatically calculates the sharing ratio of each task through dynamic task association evaluation, while introducing unrelated tasks as regularization to effectively extract shared features and task-specific features, thereby effectively improving the accuracy of multi-task processing.

[0038] The technical solutions in the embodiments of this application will be described in detail below.

[0039] Figure 1 A schematic flowchart of the multi-task learning method provided in this application is shown as an example and not a limitation; this method can be applied to computer devices. Figure 1 As shown, the method includes the following steps:

[0040] S101, Collect the raw data required for the initial task set, and generate training samples based on the raw data; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, which is used to process at least the core task in the initial task set.

[0041] It's important to note that the core tasks in the initial task set are crucial in multi-task learning and constitute the model's primary work. Auxiliary tasks, introduced to assist the core tasks, can improve their performance through regularization. For example, in image analysis, the core task might be object classification, while auxiliary tasks could include image segmentation or texture classification. In natural language processing, the core task might be text classification, while auxiliary tasks could include named entity recognition.

[0042] Different initial task sets can be adopted for different application scenarios. For example, in medical image analysis, the core task could be tumor region classification, while the auxiliary task could be organ boundary localization. By introducing organ localization tasks, the model can learn richer anatomical structural information, thereby improving the accuracy of tumor classification. In autonomous driving scenarios, the core task could be vehicle detection, while the auxiliary task could be lane line detection. Through joint training, the model can better understand the road environment and reduce the false detection rate. Through flexible task settings, multi-task learning models can achieve efficient knowledge sharing and performance improvement in different application scenarios.

[0043] There are several ways to set up core and auxiliary tasks in the initial task set. In some embodiments, they can be manually designed based on prior knowledge or domain experience between tasks, or they can be dynamically generated using data-driven automated methods. In one example, core and / or auxiliary tasks can be dynamically generated using a machine learning model. For instance, related auxiliary tasks can be generated based on core tasks to provide regularization and improve the performance of core tasks; alternatively, core and auxiliary tasks can be automatically split according to user needs.

[0044] The multi-task learning method provided in this application is applicable to various technical fields and supports multiple types of raw data. In one embodiment, the raw data in S101 can be images, text, audio, video, or multimodal data, specifically determined according to the data type required by the initial task set. Accordingly, when collecting raw data, the appropriate collection method can be selected according to the data type, supporting the acquisition of raw data from multiple sources. For example, if the core task is tumor segmentation in medical imaging and the auxiliary task is organ localization, the raw data can be CT images and pathological slide images, specifically acquired through medical imaging equipment (such as CT scanners and defect scanners); if the core task is text sentiment analysis and the auxiliary task is named entity recognition, the raw data can be text data, specifically acquired directly or through speech-to-text conversion. In another embodiment, when acquiring raw data for generating training samples, it can also be directly acquired from databases, public websites, and other channels.

[0045] In one embodiment, during the generation of training samples based on raw data, the raw data can undergo preprocessing such as data cleaning, data normalization, data standardization, and data augmentation. The data cleaning stage removes low-quality or redundant data for different task types, such as removing blurry or low-resolution image samples in image analysis. Data normalization and standardization operations can be flexibly designed according to task requirements. For example, image data typically normalizes pixel values ​​to the [0,1] range and uses channel-level standardization (such as ImageNet mean-variance normalization). Data augmentation strategies can be adapted to task characteristics; for example, random rotation, cropping, and color dithering are used in image classification tasks, while the coordinates of the image and bounding boxes need to be adjusted simultaneously in object detection tasks. For multimodal data (such as image-text pairs), cross-modal alignment processing can also be performed, such as spatially associating image regions with corresponding text descriptions, or synchronizing video and audio data using timestamps. Through the above preprocessing, the training samples can effectively adapt to the heterogeneous input requirements of multi-task models while eliminating noise interference, improving the model's generalization ability and robustness in complex scenarios.

[0046] S102, training a multi-task learning model.

[0047] Specifically, the multi-task learning model is trained in the following way:

[0048] S1021, Extract shared features from input data.

[0049] In multi-task learning, shared features refer to feature representations used in multiple tasks. These features capture common information across different tasks, providing a unified feature foundation for all tasks. By learning shared features across multiple tasks, the model can better adapt to different tasks and reduce overfitting. Furthermore, the model can more efficiently utilize common information in the input data, thereby accelerating the training process. Step S1021 aims to extract the low-level and mid-level features of the input data, providing a unified feature foundation for all tasks. In contrast to shared features, task-specific features refer to features extracted and learned specifically for a particular task. These features are typically designed to capture information and patterns unique to that task.

[0050] In step S1021, common features can be extracted from the input data to form a shared feature representation. The specific feature extraction method can be selected based on the task characteristics and data types of different domains. For example, in computer vision tasks, common features are typically extracted using convolutional neural networks (CNNs), such as using the first few layers of convolutional blocks in ResNet or EfficientNet to capture low-level features like edges, textures, and shapes of images. These features can provide shared basic representations for tasks such as image classification, object detection, and image segmentation. In natural language processing tasks, common features can be extracted using Transformer encoders (such as BERT or RoBERTa), utilizing self-attention mechanisms to model contextual dependencies between words. The generated semantic representation can simultaneously support tasks such as text classification, named entity recognition, and sentiment analysis. For multimodal tasks (such as image-text joint analysis), the extraction of common features requires combining cross-modal fusion techniques, such as using a CLIP dual-encoder structure to extract features from images and text separately, and aligning their shared representation spaces through contrastive learning. In speech processing tasks, general features can be extracted using Mel spectrograms or WaveNet to capture the time-frequency characteristics of acoustic signals, providing a shared foundation for tasks such as speech recognition and speech emotion classification. In some embodiments, the extraction of general features can be optimized by combining domain knowledge to meet the needs of specific domains: for example, in medical image analysis, 3D convolutional networks are used to extract spatial features from CT or MRI volume data, combined with deformable convolutions to adapt to organ deformation; in industrial defect detection, traditional image processing algorithms (such as Canny edge detection) are combined with deep feature extraction networks to enhance the representation of surface textures and defect regions. The shared feature representations extracted in the above ways can not only effectively support joint training of multiple tasks, but also significantly reduce the consumption of computing resources during the inference stage, improving the deployment efficiency of the model. Furthermore, by switching different shared feature extraction methods (specifically, different shared feature extraction modules), the model can be flexibly transferred to different application domains, improving the model's generalization ability.

[0051] Specifically, the shared feature representation can be denoted as: .in This represents the feature transformation obtained through any feature extraction method. For example, the shared features extracted using convolutional layers are represented as follows:

[0052]

[0053] Where L is the number of convolutional layers. For activation functions (e.g., ReLU) and These are the weights and biases of the l-th layer, respectively.

[0054] S1022, Evaluate the correlation between tasks in the initial task set; identify at least one auxiliary task whose correlation with the core task is below a threshold as an irrelevant task, and combine the irrelevant task with the core task to form a target task set.

[0055] This step S1022 aims to evaluate the correlation between tasks and proactively introduce auxiliary tasks that are weakly or unrelated to the core tasks. These auxiliary tasks provide regularization, forcing the extraction of shared features from different perspectives, thereby enhancing the universality and robustness of the shared features.

[0056] Assessing the correlation between tasks can employ existing correlation calculation methods, such as gradient cosine similarity, Pearson correlation coefficient, Euclidean distance, and mutual information. In some implementations, a task correlation matrix can be used to visually reflect the cooperative or conflicting relationships between tasks.

[0057] The following details the correlation evaluation method based on gradient cosine similarity: In each backpropagation process, the gradient of the key features of the shared layer is calculated for each task i. The correlation between task i and task j is calculated using gradient cosine similarity, as shown in the following formula:

[0058]

[0059] in It is the gradient vector and The dot product (inner product) represents the consistency of the gradient direction. The specific calculation method is as follows:

[0060]

[0061] and These are the gradient vectors. and The Euclidean norm (L2 norm) is:

[0062]

[0063] All The results are integrated into a task relevance matrix R as shown below.

[0064]

[0065] Based on the correlation between tasks, tasks with a correlation below a threshold can be selected from the initial task set (e.g., tasks with a correlation below a threshold for the core task). The auxiliary tasks are treated as irrelevant tasks and together with the core tasks form the target task set. In one embodiment, the number of irrelevant tasks selected can be set in various ways and can be dynamically adjusted. In one example, if the number of irrelevant tasks that can be selected in the initial task set is greater than or equal to the number of irrelevant tasks to be selected (hereinafter referred to as the preset number), then the preset number of irrelevant tasks can be selected and added to the target task set. In another example, if the number of irrelevant tasks that can be selected in the initial task set is less than the preset number, various methods can be used to handle this, such as: only selecting the irrelevant tasks that can be selected in the initial task set; or, adaptively generating a certain number of auxiliary tasks and adding them to the initial task set to re-evaluate the correlation between tasks; or, generating irrelevant tasks and adding them to the target task set to bring the number of irrelevant tasks to the preset number; or, adding perturbations to the auxiliary tasks in the initial task set to reduce the correlation between tasks, etc. In one embodiment, for irrelevant tasks, further perturbations can be added to achieve better regularization, such as adding noise to the task-independent data or adding noise to the task-independent labels.

[0066] In one embodiment, when determining the relevance to a core task, if there are more than two core tasks, an auxiliary task whose relevance to all core tasks is below a threshold can be selected as an irrelevant task. Alternatively, depending on actual needs, an auxiliary task whose relevance to at least one core task is below a threshold can be selected as an irrelevant task.

[0067] S1023, Based on the correlation between tasks in the target task set, determine the sharing ratio of features obtained by each task from the shared features.

[0068] Step S1023 aims to achieve adaptive parameter sharing. Specifically, it aims to give core tasks and tasks highly correlated with other tasks a higher sharing ratio, while giving unrelated tasks a lower sharing ratio. For example, the sharing ratio of core tasks can be higher than that of unrelated tasks. To achieve this, various possible sharing ratio allocation methods can be adopted. These include strategies such as dynamic weight allocation based on task relevance, feature channel selection based on attention mechanisms, and sharing ratio adjustment based on gradient conflict.

[0069] For example, in step S1022, when generating the task relevance matrix R, a shared scaling factor α can be generated for each task based on the task relevance matrix R, wherein... Specifically, the calculation is performed using a fully connected network and activation functions (sigmoid or softmax), as shown in the following formula:

[0070]

[0071] in, and For parameters, This refers to either the sigmoid function or the softmax function.

[0072] In one embodiment, each task fuses shared features with task-specific features through a dedicated branch layer (such as a fully connected layer, a convolutional layer, or a TransformerDecoder layer) to achieve task-specific output feature representation.

[0073] After generating the shared scaling factor, for task i, the shared feature portion during feature partitioning is:

[0074]

[0075] Task-specific features are transformed by the function. get:

[0076]

[0077] The above two parts are fused to obtain the task-specific representation of task i, and the fusion formula is:

[0078]

[0079] in, , and These are the learnable parameters for the i-th branch layer of task.

[0080] By allocating the sharing ratio in S1023, on the one hand, core tasks and tasks highly related to core tasks receive a higher sharing ratio, which is conducive to core tasks making priority use of global features, enhancing the performance of core tasks, and optimizing knowledge transfer; on the other hand, unrelated tasks receive a lower sharing ratio, which improves the robustness of the model to noisy data or distribution shifts through the weak constraint of shared features.

[0081] S1024, periodically verify the output results of each task in the target task set. If the verification results do not meet the training termination condition, dynamically adjust the loss weight or sharing ratio of irrelevant tasks, or adjust the irrelevant tasks in the target task set.

[0082] Through this step S1024, the loss weights, sharing ratios, and unrelated tasks in the target task set can be dynamically adjusted.

[0083] In one embodiment, during the training of a multi-task model, the total loss of the model can be obtained by weighted summation of the losses of each task in the target task set. Different tasks may use different loss functions and weights, and typically the loss weight of core tasks is greater than that of irrelevant tasks. For example, the total loss can be calculated using the following formula:

[0084]

[0085] Among them, L total For the total loss, L i and L j The losses for task i and task j are λ, respectively. i and μ j These are the loss weights for task i and task j, respectively.

[0086] The loss function for each task can be determined based on the task type. For example, classification tasks can use cross-entropy loss, with the specific loss function as follows:

[0087]

[0088] For example, mean squared error loss can be used for regression tasks, with the specific loss function as follows:

[0089]

[0090] In one embodiment, network parameters can be updated based on the total model loss. Exemplarily, this process may include:

[0091] Utilizing total loss The gradient is calculated using the standard backpropagation algorithm, and the network parameters are updated using gradient descent.

[0092]

[0093] in Represents all learnable parameters. This is the learning rate.

[0094] In one example, when gradient cosine similarity is used in S1022 to evaluate the correlation between tasks, the gradient information recorded during backpropagation can be fed back to the similarity calculation, thereby updating the task correlation matrix R and the calculation strategy of the shared scaling factor α in real time.

[0095] In one embodiment, a validation set can be used to periodically validate the output results of each task. The validation set can be generated based on the raw data collected in step S101, or it can be generated by collecting additional data. In one example, multiple validation metrics can be set for periodic validation, such as accuracy and IoU. Based on the evaluation results, the loss weights of irrelevant tasks can be dynamically adjusted. and sharing ratio And irrelevant tasks in the target task set. Specifically, if the verification results show that any irrelevant task interferes with the core task (which can be determined by specific verification indicators), then its impact will be reduced through dynamic adjustments, or by means such as removing the irrelevant task from the target task set, replacing the irrelevant task with other irrelevant tasks, or introducing at least one new irrelevant task into the target task set.

[0096] After the model training is completed, the actual data required for each task processing is input into the multi-task learning model to obtain the processing results of each task.

[0097] This completes the work on... Figure 1 Description of the process shown.

[0098] pass Figure 1 The process shown can dynamically evaluate the correlation between multiple tasks, actively introduce auxiliary tasks with low or no correlation, and further determine the feature sharing ratio based on task correlation to achieve effective information sharing and regularization constraints between tasks, reduce the risk of overfitting to a single task, and improve the accuracy of multi-task processing.

[0099] For ease of understanding Figure 1 The process shown below will be explained in detail using multi-task learning based on image data as an example:

[0100] Input data: RGB image with dimensions 224*224

[0101] An initial set of tasks that can be used for image analysis, image processing, and / or computer vision. Specifically, these may include:

[0102] Task 1 (Core Task): Object Classification (e.g., 10 categories);

[0103] Task 2 (Core Task): Image Segmentation (Pixel-Level Annotation);

[0104] Task 3 (Auxiliary Task): Texture classification (e.g., roughness, smoothness, stripe classification) to provide regularization constraints.

[0105] Multi-task learning model structure (see [reference]) Figure 2 (as shown)

[0106] Data preprocessing module: 1 layer;

[0107] Shared feature extraction module: Employs a 6-layer convolutional neural network to extract shared features. ;

[0108] Dynamic task association evaluation module: Calculates gradient cosine similarity to form task relevance matrix R;

[0109] Irrelevant task injection module: used to introduce irrelevant tasks and perform perturbation design;

[0110] Gating mechanism layer: Layer 1, calculates the sharing ratio ;

[0111] Dynamic task branching module: used to integrate shared features and task-specific features, with 2 levels of dedicated branches designed for each task, for a total of 3×2=6 levels;

[0112] Task output layer: Used to obtain the output of each task, with one layer of output for each task, for a total of 3 layers;

[0113] Dynamic loss reconciliation module: used to calculate model loss, 1 layer;

[0114] Backpropagation and parameter update module: used for backpropagation and updating the parameters of the entire network;

[0115] Validation and Adjustment Module: Used to validate the model and dynamically adjust unrelated tasks.

[0116] The training process can be summarized as follows:

[0117] The preprocessed input image obtains shared features through the shared feature extraction module. ;

[0118] The dynamic task association evaluation module uses gradient cosine similarity to calculate the task relevance matrix R and introduces unrelated tasks into the target task set.

[0119] The gating mechanism layer calculates the sharing ratio of each task based on the task relevance matrix R. ;

[0120] For each task, the dynamic task branching module merges shared features and task-specific features to generate the final feature. ;

[0121] Each task's output layer generates predictions based on the final features and calculates the loss separately. ;

[0122] The dynamic loss reconciliation module calculates the total loss according to the loss weight of each task. ;

[0123] Backpropagation updates all network parameters, while also feeding back updates to R and the sharing ratio. ;

[0124] After periodic verification, the loss weights, sharing ratios, or irrelevant tasks in the target task set are dynamically adjusted based on the verification metrics to achieve closed-loop feedback.

[0125] The following is combined with Figure 3 The specific implementation steps of multi-task learning based on image data are explained.

[0126] S301, collect the raw image data required for the initial task set and generate training samples.

[0127] As given in the task settings above, the initial task set includes two core tasks (Task 1 (core object classification) and Task 2 (image segmentation)) and one auxiliary task (Task 3 (texture classification)).

[0128] For example, this step can acquire 224×224 RGB images from a publicly available image dataset, and have experts annotate them with object classification, segmentation labels, and texture information. This is achieved by formatting and normalizing the images (using a formula). Preprocessing, such as , is performed on the image data. The preprocessed image data, along with the labels for each task, constitutes the training sample set. .

[0129] S302, extract shared features from preprocessed image data.

[0130] This step extracts common visual features from the preprocessed image data as shared features, providing fundamental information common to all tasks. Specifically, the image can be input into a 6-layer convolutional network, with features extracted layer by layer: Low layers (layers 1-2): extracting edges, corners, and basic texture information. Middle layers (layers 3-4): capturing local structure and shape. High layers (layers 5-6): progressively extracting some global semantic information while still retaining common features. The final shared features are obtained. .in .in This represents the forward propagation operation of a 6-layer convolutional network, and the output is... It includes information such as edges, textures, and local shapes.

[0131] S303, evaluate the correlation between tasks in the initial task set, identify at least one auxiliary task whose correlation with the core task is below a threshold as an irrelevant task, and combine the irrelevant task with the core task to form a target task set.

[0132] For example, step S303 calculates the correlation between tasks in the initial task set based on gradient cosine similarity. Specifically, this includes: during backpropagation, recording the gradients of object classification, image segmentation, and texture classification tasks in the shared layer, such as recording the corresponding gradients for task 1 (object classification), task 2 (image segmentation), and task 3 (texture classification). , , The gradient cosine similarity is calculated to form the correlation matrix R. The gradient calculation includes the following steps: 1) Forward propagation is performed for each task to obtain the predicted output; 2) The loss for each task is calculated based on its loss function (e.g., cross-entropy loss for classification tasks, mean squared error for regression tasks, etc.); 3) The gradient of each task with respect to the shared feature layer is calculated through backpropagation. For task i, the gradient of the shared layer is... The formula for calculating gradient cosine similarity is as follows:

[0133]

[0134] Based on gradient cosine similarity This yields the task relevance matrix R. The elements of this matrix... This represents the similarity between task i and task j. The task relevance matrix R is as follows:

[0135]

[0136] If the texture classification task has low correlation with other core tasks, for example, r 13 and r 23 If all values ​​are less than the threshold τ, then the texture classification task will be added to the target task set as an irrelevant task.

[0137] S304, Based on the correlation between tasks in the target task set, determine the sharing ratio of features obtained by each task from the shared features.

[0138] For example, the average relevance of task i can be calculated first based on the task relevance matrix R, using the following formula:

[0139]

[0140] Then, the sharing ratio factor is calculated based on the average correlation. Specifically, this can be achieved through a fully connected layer. The sigmoid activation function generates the sharing ratio, and the specific formula is as follows:

[0141]

[0142] Calculated The value ranges from (0,1), indicating how many shared features task i needs. The larger the value, the more shared features task i needs.

[0143] For example, after calculating the sharing ratio, the shared features and task-specific features of each task can be calculated separately, and then the two can be fused to obtain the final features of the task.

[0144] Specifically, taking Task 1 (object classification) as an example, we calculate the shared features respectively. and task-specific characteristics Among them, shared features are mainly used to capture global semantic information, which helps to improve classification accuracy; This is a small convolutional network used to extract task-specific information. Shared features and task-specific features are fused using the ReLU activation function to obtain the final features for task 1. The final features of Task 1 contain both shared information and task-specific features, making them suitable for classification tasks.

[0145] The same principle applies to Tasks 2 and 3; ultimately, ReLU fusion is used to obtain the final features of each task. .

[0146] The following examples illustrate this further:

[0147] For Task 1 (object classification):

[0148] If, according to calculations, the average correlation c1 between Task 1 and other tasks is high, then the calculated value of α1 will also be high, for example, it might be 0.8. Based on this, in feature segmentation:

[0149] Shared features of Task 1:

[0150] Task-specific characteristics of Task 1:

[0151] After fusing shared features and task-specific features, Task 1 mainly relies on shared features, which is beneficial for capturing global semantic information.

[0152] For Task 3 (Texture Classification):

[0153] If task 3, as an auxiliary task, is added to the target task set, then this task has a low correlation with other tasks, and the calculated value of α3 will also be low, for example, it might be 0.3. Based on this, in feature segmentation:

[0154] Shared features of Task 3:

[0155] Task-specific parts:

[0156] Therefore, by fusing shared features and task-specific features, Task 3 mainly retains its own specific texture information, while a small amount of shared features can also play a regularization role.

[0157] S305 periodically validates the output results of each task in the target task set. If the validation results do not meet the training termination condition, the loss weight or sharing ratio of irrelevant tasks is dynamically adjusted, or the irrelevant tasks in the target task set are adjusted.

[0158] For example, the output layer design for each task is as follows: Task 1's output layer uses softmax for 10-class classification; Task 2's output layer uses upsampling and softmax for pixel-level segmentation; Task 3's output layer uses softmax for texture classification. Specifically, Task 1 uses softmax for 10-class classification, with the following formula:

[0159]

[0160] Task 2 uses upsampling and Softmax for pixel-level classification, with the specific formula as follows:

[0161]

[0162] Task 3 uses Softmax for texture classification, with the following formula:

[0163]

[0164] For example, the total loss of a multi-task learning model is calculated as follows: the cross-entropy loss of each task is calculated separately, and then weighted by learnable weights σ1, σ2, σ3 to form the total loss. . Specifically:

[0165] The loss for Task 1 is calculated using the following loss function:

[0166]

[0167] The loss for Task 2 is calculated using the following loss function:

[0168]

[0169] The loss for Task 3 is calculated using the following loss function:

[0170]

[0171] Calculate the total loss using uncertainty weighting:

[0172]

[0173] For example, based on total loss Backpropagation is performed to update the parameters of the entire network, and new gradient information is fed back to update the task relevance matrix R and the sharing ratio.

[0174] For example, each training cycle consists of 10 epochs. The output performance of each task is evaluated using a validation set. If the auxiliary task 3 (texture classification task) significantly interferes with the core tasks (task 1 and task 2), the loss weight of task 3 is reduced. Or adjust the sharing ratio If the regularization effect is not obvious, consider adjusting the auxiliary task data collection strategy or introducing new unrelated tasks.

[0175] This embodiment demonstrates that the present application can effectively improve the robustness of shared features, reduce the risk of overfitting in core tasks, and ultimately improve the processing performance of each task by introducing unrelated task regularization.

[0176] In an optional embodiment, the scheme can be described using a multi-task medical image analysis task, detailing the specific implementation process of this application integrating quantum computing. In this scenario, the initial task set is set as follows: Task 1 (core task) is to classify tumors in medical image data, i.e., tumor classification task; Task 2 (core task) is image segmentation of tumor regions, i.e., tumor segmentation task; Task 3 (auxiliary task) is to identify texture features in the image, i.e., tumor segmentation task, thereby assisting in improving the accuracy of the core task. The process includes the following steps:

[0177] Step (1) Standardize and preprocess the medical image data, and map and encode the gray values ​​of the image pixels with the amplitude and phase of the quantum state to generate medical image data represented by quantum state;

[0178] Step (2) Input the medical image data in quantum state representation generated in step (1) into a quantum convolutional neural network (QCNN). QCNN contains quantum convolutional layers, quantum pooling layers and quantum fully connected layers. The quantum state representation of the medical image data is convolved through quantum gate operations to extract the quantum features of tumor edge, texture and shape.

[0179] Step (3) Based on the quantum features extracted in step (2), calculate the quantum state entanglement measure between the tumor classification task, the tumor segmentation task and the texture recognition task, and quantify the correlation between tasks through the quantum mutual information value to obtain the task correlation result;

[0180] Step (4) Based on the task relevance results obtained in step (3), dynamically adjust the proportion of shared feature layers and the weight of loss function in multi-task training, and use quantum parallel computing to optimize model parameters to complete the joint output of tumor classification, segmentation and texture recognition.

[0181] Specifically, the mapping encoding in step (1) includes: normalizing the gray values ​​of image pixels to the amplitude of quantum states, representing the spatial position relationship of pixels through phase encoding; using the superposition state of qubits to represent the entire image in parallel, with the state vector of each qubit corresponding to the quantum state information of a local region of the image.

[0182] The implementation of the quantum convolutional layer in step (2) includes: applying a combination of Hadamard gate, CNOT gate and parameterized rotation gate to the input quantum state medical image data in the quantum circuit to generate a quantum convolutional kernel; and extracting the edge and texture features of the tumor region in the image in parallel through the superposition and entanglement characteristics of quantum states to output a multi-channel quantum feature map.

[0183] The calculation of quantum state entanglement measurement in step (3) includes: constructing density matrices for the quantum feature states of the tumor classification task, tumor segmentation task and texture recognition task respectively; calculating the joint entropy and conditional entropy between each task, and determining the task correlation based on the quantum mutual information formula; when the mutual information value exceeds the preset threshold, it is determined that there is a strong quantum correlation between the tasks, and the feature sharing weight needs to be increased.

[0184] The dynamic adjustment strategy described in step (4) includes: increasing the number of qubits in the shared quantum fully connected layer for the first task pair (tumor classification and texture recognition) with mutual information values ​​higher than the threshold; and introducing a quantum decoherence noise suppression module in the shared feature layer for the second task pair (tumor segmentation and texture recognition) with mutual information values ​​lower than the threshold to reduce inter-task interference.

[0185] The quantum parallel computing optimization in step (4) includes: during the backpropagation process, using the superposition state of qubits to simultaneously calculate the gradient of multiple tasks, accelerating the gradient update through the quantum amplitude amplification algorithm; and using the quantum approximation optimization algorithm to search for the global optimal solution of the model parameters, thereby reducing the number of training iterations.

[0186] The quantum pooling layer of QCNN is implemented by performing quantum measurement operations on the multi-channel quantum feature map output by the quantum convolution layer to compress the feature dimension; retaining the key quantum state amplitude distribution information of the tumor region and discarding redundant phase components.

[0187] In this embodiment, by mapping and encoding the grayscale values ​​of medical image pixels with the amplitude and phase of quantum states, and utilizing the superposition characteristics of qubits, the high-dimensional pixel space of traditional images can be compressed to a logarithmic number of qubits, significantly reducing the overhead of data storage and transmission. The quantum convolutional layer, through combinations of quantum gates such as Hadamard gates and CNOT gates, utilizes quantum superposition states to process multi-regional features of the image (such as tumor edges and textures) in parallel. A single operation can cover the feature extraction that traditional CNNs require multiple convolution operations to complete. When the quantum gate operation acts on the superposition state, it can simultaneously process all possible input states. For example, the convolution kernel for tumor edge detection needs to slide pixel by pixel in traditional computation, while quantum convolution completes the full-image convolution operation in one go through superposition states; the reduction in time complexity stems from quantum parallelism.

[0188] In this embodiment, quantum entanglement measurement quantifies the nonlinear dependencies between tasks in the quantum feature space. Based on task relevance determination using quantum mutual information, dynamically adjusting the proportion of shared features and loss weights accelerates model training convergence and improves final classification accuracy. In quantum systems, the degree of entanglement of task feature states directly reflects their information sharing potential. For example, if the feature states of a tumor classification task and a texture recognition task are strongly entangled, it indicates that they share microscopic quantum features (such as sub-pixel-level texture patterns), a correlation that traditional methods struggle to capture due to limitations in feature space dimensionality. Introducing a quantum decoherence suppression module for low-correlation tasks (such as tumor segmentation and texture recognition) reduces interference from irrelevant features.

[0189] In this embodiment, the gradient of multiple tasks is calculated simultaneously using quantum superposition states, and combined with a quantum amplitude amplification algorithm, the gradient update speed of backpropagation can be improved. Traditional multi-task gradient calculation requires differentiation and weighted summation for each task, while the superposition state of qubits allows the gradient directions of multiple tasks to be encoded simultaneously. The multi-gradient superposition calculation is completed in a single quantum operation, and the time consumption is independent of the number of tasks.

[0190] The order of the steps in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0191] Corresponding to the multi-task processing method described in the above embodiments, Figure 3 A structural block diagram of a multitasking processing device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0192] Reference Figure 3 The device includes:

[0193] The sample generation module is used to collect the raw data required for the initial task set and generate training samples; the initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, which is used to process at least the core task in the initial task set.

[0194] The model training module, used to train multi-task learning models, includes:

[0195] The shared feature extraction submodule is used to extract shared features from the input data;

[0196] The task association evaluation submodule is used to evaluate the correlation between tasks in the initial task set; at least one auxiliary task whose correlation with the core task is below a threshold is regarded as an irrelevant task, and the irrelevant tasks are combined with the core task to form a target task set.

[0197] The parameter sharing submodule is used to determine the sharing ratio of features obtained by each task from the shared features based on the correlation between the tasks in the target task set.

[0198] The dynamic adjustment submodule is used to periodically verify the output results of each task in the target task set. If the verification results do not meet the training termination condition, the loss weight or sharing ratio of irrelevant tasks is dynamically adjusted, or irrelevant tasks in the target task set are adjusted.

[0199] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0200] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0201] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor, when executing the computer program, implements the steps in any of the above method embodiments.

[0202] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0203] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0204] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 4 As shown, the computer device of this embodiment includes: at least one processor 40 ( Figure 4 (Only one is shown in the diagram), memory 41, and computer program 42 stored in the memory 41 and executable on the at least one processor 40, wherein the processor 40 executes the computer program 42 to implement the steps in any of the above-described visual programming method embodiments.

[0205] The computer device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0206] The processor 40 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0207] In some embodiments, the memory 41 may be an internal storage unit of the computer device, such as a hard disk or memory. In other embodiments, the memory 41 may be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 41 may include both internal and external storage units of the computer device. The memory 41 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0208] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / computer equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0209] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0210] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0211] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-task learning method, characterized in that, include: Collect the raw data required for the initial task set and generate training samples; The initial task set includes at least one core task and at least one auxiliary task; The training samples are used to train a multi-task learning model, which is used to process at least the core tasks in the initial task set; wherein the original data is any one of the following: image data, text data, audio data, and video data. The multi-task learning model is trained in the following manner: Extracting shared features from input data ; Assess the correlation between tasks in the initial task set; identify at least one auxiliary task whose correlation with the core task is below a threshold as an irrelevant task, and combine the irrelevant tasks with the core task to form a target task set; determine the sharing ratio of features acquired from the shared features by each task in the target task set in the following manner: generate a sharing ratio factor α based on the correlation between tasks in the initial task set, and the shared feature portion of task i... for: ; By transforming the function Obtain the task-specific characteristics of task i : ; Will and The final features of task i are obtained by fusion. The fusion formula is: ; in, , and For the learnable parameters of the i-th branch layer; , The total number of all tasks in the initial task set; The output results of each task in the target task set are periodically verified. If the verification result does not meet the training termination condition, the irrelevant tasks in the target task set are adjusted. The adjustment of the irrelevant tasks in the target task set includes: if any irrelevant task interferes with the core task, the irrelevant task is removed from the target task set; or, an irrelevant task is selected from the initial task set to replace the irrelevant task. The step of designating at least one auxiliary task whose relevance to the core task is below a threshold as an irrelevant task includes: if the number of auxiliary tasks in the initial task set whose relevance to the core task is below a threshold does not reach a preset number, then selecting all auxiliary tasks in the initial task set whose relevance to the core task is below a threshold as irrelevant tasks; or, generating a certain number of auxiliary tasks and adding them to the initial task set to reassess the relevance between tasks; or, generating a certain number of irrelevant tasks to bring the number of irrelevant tasks to a preset number; or, adding perturbations to other auxiliary tasks in the initial task set to reduce the relevance between tasks.

2. The method as described in claim 1, characterized in that, The evaluation of the correlation between tasks in the initial task set includes: The correlation between tasks in the initial task set is calculated based on gradient cosine similarity to generate a task correlation matrix.

3. The method as described in claim 1, characterized in that, The method further includes: The total loss is obtained by weighted summation of the losses of each task in the target task set; wherein the loss weight of the core task is greater than the loss weight of the irrelevant task.

4. The method as described in claim 1, characterized in that, The sharing ratio of core tasks in the target task set is higher than the sharing ratio of unrelated tasks.

5. The method according to any one of claims 1-4, characterized in that, If the original data is image data, then the initial task set is used for image analysis, image processing, and / or computer vision.

6. A multi-task learning device, characterized in that, include: The sample generation module is used to collect the raw data required for the initial task set and generate training samples. The initial task set includes at least one core task and at least one auxiliary task; the training samples are used to train a multi-task learning model, which is used to process at least the core task in the initial task set; wherein, the original data is any one of the following: image data, text data, audio data, and video data; The model training module, used to train multi-task learning models, includes: The shared feature extraction submodule is used to extract shared features from the input data; The task association evaluation submodule is used to evaluate the correlation between tasks in the initial task set; at least one auxiliary task whose correlation with the core task is lower than a threshold is regarded as an irrelevant task, and the irrelevant task and the core task are combined to form a target task set; The parameter sharing submodule is used to determine the sharing ratio of features obtained from the shared features by each task in the target task set in the following manner: a sharing ratio factor α is generated based on the correlation between tasks in the initial task set, and the shared feature portion of task i... for: ; By transforming the function Obtain the task-specific characteristics of task i : ; Will and The final features of task i are obtained by fusion. The fusion formula is: ; in, , and For the learnable parameters of the i-th branch layer; , The total number of all tasks in the initial task set; The dynamic adjustment submodule is used to periodically verify the output results of each task in the target task set. If the verification result does not meet the training termination condition, the irrelevant tasks in the target task set are adjusted. The adjustment of the irrelevant tasks in the target task set includes: if any irrelevant task interferes with the core task, the irrelevant task is removed from the target task set; or, an irrelevant task is selected from the initial task set to replace the irrelevant task. The step of designating at least one auxiliary task whose relevance to the core task is below a threshold as an irrelevant task includes: if the number of auxiliary tasks in the initial task set whose relevance to the core task is below a threshold does not reach a preset number, then selecting all auxiliary tasks in the initial task set whose relevance to the core task is below a threshold as irrelevant tasks; or, generating a certain number of auxiliary tasks and adding them to the initial task set to reassess the relevance between tasks; or, generating a certain number of irrelevant tasks to bring the number of irrelevant tasks to a preset number; or, adding perturbations to other auxiliary tasks in the initial task set to reduce the relevance between tasks.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, When the computer program product is run on a computer device, it causes the computer device to perform the method as described in any one of claims 1 to 5.

Images