Deep cooperative multi-task feature learning method based on mutual information regularization

Abstract

The invention provides a deep collaborative multi-task feature learning method based on mutual information regularization. Aiming at the technical problems of low feature sharing efficiency, serious inter-task interference, poor generalization ability and the like in the existing multi-task learning, the invention constructs a deep learning framework based on mutual information constraint, and designs a mutual information calculation module of variational inference and a KL divergence regularization optimization mechanism. And cross-task feature efficient sharing is realized through a multi-layer encoder structure and a feature adaptive selection mechanism. According to the scheme, mutual information calculation is optimized by adopting a batch estimation strategy and a parallel calculation mechanism, and a dynamic task weight distribution and cross-task knowledge migration mechanism is introduced to enhance the generalization ability of the model. Experiments show that compared with the prior art, the classification accuracy, convergence speed, calculation efficiency and the like of the method are improved by 31.2%, 65.3% and 44.8% respectively, and the method has higher cross-domain migration ability and can be widely applied to multi-task learning scenes such as computer vision and natural language processing.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a deep collaborative multi-task feature learning method based on mutual information regularization. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning has achieved significant breakthroughs in many fields such as computer vision and natural language processing. Traditional single-task deep learning models are usually optimized for specific tasks, making it difficult to effectively utilize common features across different tasks, resulting in low training efficiency and limited generalization ability. To address this issue, Multi-Task Learning (MTL) has emerged. Its core idea is to improve learning efficiency and model performance by simultaneously learning multiple related tasks, sharing feature representations and model parameters.

[0003] In recent years, multi-task learning has received widespread attention in academia and industry. Early multi-task learning methods mainly adopted hard parameter sharing strategies, achieving feature sharing by sharing the underlying network structure. For example, the DeepRelationship Network proposed in 2015 promoted feature sharing by establishing tensor regularization constraints between different tasks. However, this simple feature sharing approach is prone to the negative transfer problem, where feature interference between different tasks actually reduces model performance. To alleviate this problem, Cross-stitch Networks proposed in 2017 introduced an adaptive feature fusion mechanism, but its computational complexity is high and it is difficult to scale to large-scale task scenarios.

[0004] With the continuous development of deep learning models, multi-task learning methods based on attention mechanisms have begun to emerge. The Multi-gate Mixture-of-Experts (MMoE) model, proposed in 2019, achieves task-specific expert model selection through gating mechanisms, improving the flexibility of feature sharing to some extent. However, this type of method still has two main problems: first, the lack of explicit modeling of feature relationships between tasks leads to low feature sharing efficiency; second, the model optimization objective is too simplistic, making it difficult to effectively balance the learning of task-specific and common features.

[0005] To further improve the performance of multi-task learning, researchers have begun exploring information theory-based optimization methods. The Information Bottleneck Multi-Task Learning approach, proposed in 2020, reduces task interference by minimizing mutual information between task features; however, this method ignores potential beneficial feature associations between tasks. Research in 2021 showed that appropriately utilizing mutual information between tasks can improve the model's representation learning ability, but how to maximize the sharing of beneficial features while maintaining task independence remains an unsolved problem.

[0006] On the other hand, as model size continues to increase, multi-task learning faces challenges in computational efficiency and training stability. Existing training methods often employ simple loss weighting strategies, which struggle to effectively handle issues such as uneven task difficulty and gradient conflicts. Furthermore, due to the lack of effective feature selection mechanisms, models are prone to overfitting features of certain tasks, leading to decreased generalization performance. In practical applications, these problems result in unstable model training, slow convergence speed, and difficulty adapting to new task scenarios.

[0007] In summary, existing multi-task learning methods still have significant shortcomings in terms of feature sharing efficiency, inter-task interference control, and model optimization strategies. Especially when dealing with large-scale, heterogeneous tasks, how to design efficient feature extraction and sharing mechanisms, how to achieve effective collaborative learning between tasks, and how to ensure model training stability and generalization ability are all pressing technical problems that need to be solved. These challenges severely restrict the promotion and effectiveness of multi-task learning in practical applications. Therefore, there is an urgent need for a novel multi-task learning method that can effectively solve the above problems to improve the overall performance and practical value of the model. Summary of the Invention

[0008] In view of the technical problems existing in multi-task learning techniques, such as low feature sharing efficiency, serious interference between tasks, and insufficient model generalization ability, this invention is proposed.

[0009] Therefore, the technical problem solved by this invention is: how to optimize feature extraction and knowledge transfer in the multi-task learning process through mutual information constraints and variational inference mechanisms, so as to achieve efficient collaboration among multiple tasks.

[0010] To address the aforementioned technical problems, this invention provides the following technical solution: a deep collaborative multi-task feature learning method based on mutual information regularization, comprising: constructing a deep learning framework based on mutual information constraints; designing a mutual information calculation module for variational inference; implementing a KL divergence regularization optimization mechanism to obtain an optimized multi-task feature learning model.

[0011] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, the method includes: constructing a deep learning framework based on mutual information constraints, which includes constructing a mutual information objective function.

[0012]

[0013] Among them, L MI The mutual information objective function is the core objective of this invention; I(X;Y) represents the mutual information between the input feature X and the task output Y, used to measure the correlation between different task features; β is a weighting coefficient used to balance the importance of the mutual information term and the KL divergence regularization term; KL(·∣∣·) represents the KL divergence, used to measure the difference between two probability distributions; Here, φ represents the encoder distribution, z is the learned latent feature representation, and p(z) is a predefined prior distribution, usually a standard normal distribution. By optimizing this objective function, this invention aims to maximize task-related mutual information, while ensuring that the learned feature distribution is close to the prior distribution through KL divergence constraints, thereby achieving efficient multi-task feature learning and knowledge transfer.

[0014] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, the method includes: the mutual information calculation module for designing variational inference includes calculating the variational lower bound:

[0015]

[0016] Where I(X;Y) represents the mutual information between random variables X and Y, used to quantify the correlation between different task features; E represents the expectation operator, used to calculate the average value of the probability distribution; p(x,y) represents the joint probability distribution of input data x and label y; q ψ (y|z) represents a decoder network with parameter ψ that predicts label y based on latent representation z; p(y) represents the marginal probability distribution of label y. Through the combination of these symbols and variables, this invention achieves efficient estimation of mutual information, provides a theoretical basis and optimization objective for multi-task learning, and effectively improves the quality of feature extraction and the collaborative learning ability between tasks.

[0017] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, the method includes: implementing the KL divergence regularization optimization mechanism includes optimizing the KL divergence constraint.

[0018]

[0019] in, Indicates by parameters The defined conditional probability distribution represents the encoder distribution of the latent variable z given an input x, reflecting the probabilistic encoding mechanism of the deep learning model for the input data; p(z) represents the prior distribution of the latent variable z, usually chosen as the standard normal distribution as the ideal reference distribution in the model learning process; E represents the expectation operation, used to calculate the average feature of the random variable; The logarithmic probability of the encoder distribution is represented by logp(z), and the logarithmic probability of the prior distribution is represented by logp(z). The difference between the two constitutes the key term for measuring the difference in probability distributions. Essentially, it quantifies the information theoretical distance between the encoding distribution learned by the model and the preset prior distribution by the difference in log probabilities, thereby constraining and regularizing the latent space representation of the deep learning model. This mechanism has broad theoretical significance and practical value in deep learning optimization strategies such as variational inference and information bottleneck.

[0020] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, the method includes: introducing a multi-layer encoder structure; designing a feature adaptive selection mechanism; and implementing a cross-task feature sharing module.

[0021] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, the method includes: employing a batch estimation strategy; implementing a parallel computing mechanism; and introducing dynamic weight adjustment.

[0022] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, the method includes: designing adaptive learning rate adjustment; implementing gradient accumulation and updating; and constructing a multi-task scheduling mechanism.

[0023] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, it includes: dynamic task weight allocation; cross-task knowledge transfer; and adaptive feature fusion.

[0024] As a preferred embodiment of the deep collaborative multi-task feature learning method based on mutual information regularization described in this invention, it includes: a deep learning framework module for constructing a deep learning framework based on mutual information constraints; a mutual information calculation module for designing mutual information calculation for variational inference; and an optimization mechanism module for implementing the KL divergence regularization optimization mechanism.

[0025] A computer device includes: a memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, it implements the steps of the deep collaborative multi-task feature learning method based on mutual information regularization as described in any one of the present invention.

[0026] A computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the deep collaborative multi-task feature learning method based on mutual information regularization as described in any one of the present invention.

[0027] The beneficial effects of this invention: The deep collaborative multi-task feature learning method based on mutual information regularization provided by this invention solves the key problems in multi-task learning from both theoretical and practical perspectives. Through innovative mechanisms such as mutual information regularization, feature adaptation, and multi-task collaboration, it achieves a comprehensive improvement in multi-task learning performance, providing new research ideas and solutions for the development of related technical fields. Attached Figure Description

[0028] Figure 1 The overall architecture diagram of the deep collaborative multi-task feature learning system based on mutual information regularization of the present invention is shown. The diagram shows the hierarchical relationship and data flow between the three core components: the deep learning framework module, the mutual information calculation module, and the optimization mechanism module. It also presents the input data processing and output result feedback process of the system.

[0029] Figure 2 The diagram shows a detailed structure of the deep learning framework based on mutual information constraints in this invention, including the construction process of the mutual information objective function, the composition of the multi-layer encoder structure, and the implementation of the feature adaptive selection mechanism. It clearly expresses the connection relationship and interaction mode between the various functional units of the framework.

[0030] Figure 3 It is a flowchart of mutual information calculation for variational inference, which describes in detail the calculation process of variational lower bound, shows the specific steps of expectation calculation, distribution estimation and mutual information optimization, and improves computational efficiency through batch estimation strategy and parallel computing mechanism.

[0031] Figure 4 The flowchart illustrates the implementation of the KL divergence regularization optimization mechanism, showcasing the KL divergence constraint optimization process between the encoder distribution and the prior distribution, including key steps such as dynamic weight adjustment and gradient accumulation update.

[0032] Figure 5 The diagram illustrates the structure of the cross-task feature sharing module, demonstrating the composition and collaborative working mechanism of the three sub-modules: dynamic task weight allocation, cross-task knowledge transfer, and feature adaptive fusion.

[0033] Figure 6 It is an optimized flowchart of the multi-task learning and training process, which details the implementation of adaptive learning rate adjustment, gradient accumulation and update and multi-task scheduling mechanism, and shows the resource allocation and task coordination strategies in the training process. Detailed Implementation

[0034] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0035] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0036] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0037] Example 1 is an embodiment of the present invention, providing a deep collaborative multi-task feature learning method based on mutual information regularization, including:

[0038] S1: Data Preprocessing and Initialization

[0039] S1.1: The input dataset D = {(xi,yi)} ni = 1 is first standardized to scale the feature values ​​to the range [0,1], where xi represents the input sample and yi represents the corresponding task label. For each task t, a corresponding task-specific data subset Dt is constructed.

[0040] S1.2: Initialize the multilayer encoder network structure parameters It includes L encoding layers {E1,...,EL}. Each encoder layer El uses a fully connected neural network structure with an input dimension of dl-1 and an output dimension of dl, where d0 is the original feature dimension. Figure 1 As shown, each layer of the encoder also contains a batch normalization layer and a ReLU activation function.

[0041] S1.3: Initialize the task-specific decoder network parameters ψt. The decoder employs a symmetric multi-layer structure to map the shared feature space back to the task-specific output space. Simultaneously, initialize the mutual information estimation network parameters θ to evaluate the mutual information between features from different tasks.

[0042] S2: Construction of the mutual information objective function

[0043] S2.1: According to Figure 2The framework shown constructs a mutual information-based objective function L_MI. This function consists of two main parts: a mutual information term I(X;Y) and a KL divergence regularization term. The importance of the two is balanced by a weighting coefficient β.

[0044] S2.2: For the mutual information term I(X;Y), the variational lower bound is used for estimation: I(X;Y)≥ in, Let represent the feature distribution of the encoder network output, and q_ψ(y|z) represent the conditional distribution predicted by the decoder network.

[0045] S2.3: For the KL divergence regularization term, calculate the encoder distribution. This is achieved through the KL divergence between the standard normal prior distribution p(z) and the standard normal prior distribution:

[0046] S3: Variational Inference Mutual Information Calculation

[0047] S3.1: According to Figure 3 The computational process shown first involves sampling a mini-batch of samples {(xi,yi)}mi=1 from the dataset, where m is the batch size. For each sample, the feature representation zi=E(xi) is calculated through the encoder network.

[0048] S3.2: Based on the feature representations obtained from sampling, the Monte Carlo method is used to estimate the variational lower bound of the mutual information. Specifically, for each feature representation zi, from... Sample K times to obtain {zi,k}Kk=1, then calculate the expectation:

[0049] S3.3: Improve computational efficiency by simultaneously processing the mutual information estimation of multiple samples through parallel computing mechanisms. A batch estimation strategy is adopted, dividing the large-scale dataset into multiple small batches and processing them batch by batch, effectively reducing memory overhead.

[0050] S4: KL divergence regularization optimization

[0051] S4.1: As Figure 4 As shown, the KL divergence regularization optimization mechanism is implemented. First, the encoder output distribution is calculated. The mean μ and variance σ² are used to parameterize the distribution of the latent variable z.

[0052] S4.2: Based on the reparameterization technique, ε ~ N(0,1) is sampled from the standard normal distribution, and then latent variable samples are generated by z = μ + σ·ε. This method ensures that the gradient can be backpropagated through the sampling operation.

[0053] S4.3: Calculate the KL divergence loss: The strength of the KL divergence constraint is adaptively adjusted based on the performance during training through a dynamic weight adjustment mechanism.

[0054] S5: Cross-task feature sharing implementation

[0055] S5.1: According to Figure 5 The module structure shown implements a dynamic task weight allocation mechanism. For each task t, its weight wt = softmax(st) is calculated, where st is the task-specific importance score, which is optimized through learnable parameters.

[0056] S5.2: Achieve cross-task knowledge transfer in the feature space. For each pair of tasks (i,j), calculate its feature correlation matrix Cij = ZiZjT, where Zi and Zj are the feature matrices of tasks i and j, respectively. Based on the correlation matrix, construct a knowledge transfer channel to achieve feature-level information interaction.

[0057] S5.3: Through a feature adaptive fusion mechanism, the feature representations of different tasks are dynamically combined. Specifically, for each task t, its final feature representation is: zt=∑iwi·fi(zi), where fi is the feature transformation function and wi is the adaptive weight.

[0058] S6: Training Process Optimization

[0059] S6.1: According to Figure 6 The optimization process implements an adaptive learning rate adjustment mechanism. The initial learning rate is set to η0, and it is dynamically adjusted during training based on the changing trend of the loss function: ηt = η0·γt, where γt is the learning rate decay factor.

[0060] S6.2: A gradient accumulation update strategy is adopted, accumulating n mini-batch gradients before updating the parameters. This method achieves the effect of large-batch training while maintaining low memory overhead.

[0061] S6.3: Construct a multi-task scheduling mechanism to determine the training order based on task complexity and dependencies. For each training cycle, first train the basic tasks, and then gradually add complex tasks to ensure that knowledge can be effectively transferred from simple tasks to complex tasks.

[0062] S7: Feature Selection and Optimization

[0063] S7.1: Implement an adaptive feature selection mechanism by dynamically weighting features at different levels through an attention mechanism. For the l-th layer feature hl, calculate its attention weight αl = softmax(W·tanh(V·hl)), where W and V are learnable parameter matrices.

[0064] S7.2: Residual connections are introduced into the multilayer encoder structure to avoid the gradient vanishing problem in deep networks. For the features of the l-th layer, the output is: hl = F(hl-1) + hl-1, where F represents the nonlinear transformation function.

[0065] S7.3: Batch normalization standardizes the feature distribution of each layer, improving the training stability and convergence speed of the model. For feature h, normalization is performed: h'=γ·(h-μB) / σB+β, where μB and σB are the mean and standard deviation of the batch, respectively, and γ and β are learnable scaling and offset parameters.

[0066] Example 2 is an embodiment of the present invention, providing a deep collaborative multi-task feature learning method based on mutual information regularization, including:

[0067] 1. Experimental Environment

[0068] This experiment was conducted on a high-performance computing platform, using an Intel Xeon CPU E5-2680 v4 processor with a clock speed of 2.40GHz, equipped with an NVIDIA Tesla V100 GPU with 16GB of video memory. The software environment used Python 3.8 as the development language, PyTorch 1.9.0 as the deep learning framework, and CUDA 11.2 for GPU acceleration. The experimental datasets included standard computer vision datasets MNIST and CIFAR-10, as well as the cross-domain transfer learning dataset Office-31, to verify the model's performance and generalization ability in different scenarios.

[0069] Comparison of Option 1

[0070] Traditional multi-task learning (MTL) methods employ a hard parameter sharing mechanism for feature extraction, specifically including a shared encoder and multiple task-specific decoders. The shared encoder uses the ResNet-50 architecture, and the decoders employ a three-layer fully connected network. This approach shares the same feature extraction layer across all tasks, neglecting task specificity and easily leading to negative transfer problems.

[0071] Comparison of Scheme 2

[0072] Attention-based multi-task learning (Attention-MTL) introduces an attention mechanism for feature selection on top of traditional methods. This approach uses a self-attention module to weight feature maps, enabling dynamic feature selection. However, it lacks a mutual information constraint mechanism to optimize feature representation, resulting in low feature extraction efficiency and significant inter-task interference.

[0073] 2. Experimental Procedure

[0074] The experiment first preprocessed the input data, including standardization and data augmentation. Standardization used the Z-score method to transform the data into a distribution with a mean of 0 and a standard deviation of 1. Data augmentation techniques included random pruning, horizontal flipping, and random rotation to improve the model's robustness. The model parameters were initialized using the Xavier method to ensure the stability of network training.

[0075] During training, a mini-batch stochastic gradient descent method with a batch size of 128 was employed. The initial learning rate was set to 0.001 and dynamically adjusted using cosine annealing. The Adam algorithm was selected as the optimizer, with β1 set to 0.9 and β2 set to 0.999. The mutual information regularization parameter β was set to 0.01 to balance the mutual information term and the KL divergence constraint term. The total number of training epochs was set to 100, and the model performance was evaluated on the validation set every 10 epochs.

[0076] During feature extraction, the multi-layer encoder employs a five-layer convolutional neural network structure, with each layer followed by batch normalization and ReLU activation functions. The attention mechanism uses a multi-head self-attention structure with 8 heads. The task-specific decoder uses a two-layer fully connected network with an intermediate layer dimension of 512. To improve training efficiency, a gradient accumulation update mechanism is implemented, with a cumulative step count of 4.

[0077] 3. Test methods and standards

[0078] Performance evaluation employs a multi-dimensional metrics system. Classification accuracy is obtained by predicting on the test set and calculating the proportion of correctly classified samples. Model convergence speed is measured by recording the number of training epochs required to reach 90% optimal performance. Memory usage is recorded as the peak GPU memory usage during training. Training time is recorded as the average time spent per training epoch.

[0079] To evaluate the model's generalization ability, cross-dataset transfer learning experiments were conducted. After the model was trained on the source domain dataset, it was directly tested on the target domain, and the classification accuracy after transfer was calculated. Each experiment was repeated 5 times, and the average value was taken as the final result. The standard deviation was also calculated to evaluate the stability of the results.

[0080] 4. Experimental Results Table

[0081]

[0082] 5. Interpretation of Experimental Results

[0083] Experimental results show that the proposed solution significantly outperforms the comparative solution in all performance indicators. In terms of classification accuracy, the proposed solution achieves 93.4%, an improvement of 11.1 percentage points compared to comparative solution 1 and 7.7 percentage points compared to comparative solution 2, representing a performance improvement of 31.2%. This result directly verifies the effectiveness of the mutual information regularization mechanism in improving the quality of feature representation.

[0084] In terms of convergence speed, the proposed solution requires only 30 rounds to reach the convergence criterion, which is 55 rounds less than the comparative solution 1 and 35 rounds less than the comparative solution 2, resulting in a 65.3% improvement in convergence speed. This significant improvement is attributed to the introduction of the feature adaptive selection mechanism, which effectively reduces interference between tasks and accelerates the model's convergence process.

[0085] Regarding memory usage, the peak memory usage of this invention is 7.1GB, which is a reduction of 44.5% and 38.2% compared to the comparative solutions' 12.8GB and 11.5GB, respectively. This optimization is attributed to the implementation of a batch estimation strategy and an efficient parallel computing mechanism. Simultaneously, training time is also significantly improved, with each training round reduced from 245 seconds in comparative solution 1 to 110 seconds, representing a 44.8% improvement in time efficiency.

[0086] In the generalization ability evaluation, the proposed solution demonstrates excellent cross-domain transfer capability. The cross-domain classification accuracy reaches 92.8%, which is 27.4 and 21.6 percentage points higher than the comparative solutions 1 and 2, respectively, representing an improvement of 41.9%. This result fully demonstrates the advantages of the proposed solution in feature representation learning and knowledge transfer, and verifies the effectiveness of the dynamic task weight allocation and cross-task knowledge transfer mechanism.

[0087] Example 3 is an embodiment of the present invention, providing a deep collaborative multi-task feature learning method based on mutual information regularization, including:

[0088] This embodiment demonstrates an implementation scheme for applying a deep collaborative multi-task feature learning method based on mutual information regularization to a large-scale visual-language pre-training scenario. In this embodiment, the system needs to simultaneously handle three core tasks: image understanding, text understanding, and cross-modal alignment, which places higher demands on the algorithm's scalability and performance.

[0089] Regarding the encoder architecture, this embodiment employs a dual-stream encoder structure. The visual encoder consists of an improved Vision Transformer, comprising 12 encoder layers, each containing 8 attention heads, with a hidden layer dimension of 768. The text encoder uses the BERT architecture, also containing a 12-layer encoder structure. To enhance feature extraction capabilities, residual connections and layer normalization modules are added to each encoder layer, and relative position encoding is introduced into the attention mechanism to better capture sequence information.

[0090] The mutual information calculation module has been specifically optimized in this embodiment. Considering the significant differences in the distribution of visual and linguistic features, a modality-specific mutual information estimator was designed. For visual features, pixel-level local mutual information estimation is used, calculating the mutual information contribution of local regions by dividing the image into patches; for text features, mutual information is calculated at the token level, while also considering the contextual dependencies between adjacent tokens. The mutual information of the two modalities is fused using adaptive weights, and the weight coefficients are dynamically adjusted based on the validation performance during training.

[0091] To improve the large-scale data processing capability, this embodiment improves the KL divergence regularization mechanism. First, a hierarchical KL divergence calculation strategy is adopted, applying constraints to different levels of the encoder separately to avoid over-compression of low-level features. Second, a soft constraint term based on Wasserstein distance is introduced to retain more mode-specific information while maintaining distribution alignment. Furthermore, a temperature-based KL divergence annealing mechanism is implemented, reducing constraint strength in the early stages of training and gradually increasing it as training progresses.

[0092] Regarding feature sharing, this embodiment implements a multi-level feature fusion network. Low-level features primarily retain modality-specific information, mid-level features are selectively shared through an attention mechanism, and high-level features are deeply fused through a cross-modal transformation module. During feature fusion, a gating mechanism dynamically adjusts the importance of features at different levels, and a contrastive learning loss is introduced to enhance feature discriminativity. Simultaneously, a feature distillation module is designed to transfer knowledge from a single-modality pre-trained model to a multi-task framework.

[0093] The training optimization strategy was also adjusted accordingly. Massive training based on gradient accumulation was implemented, with each mini-batch containing 256 pairs of image-text samples. Parameters were updated after accumulating four gradients. The learning rate adopted a cosine annealing strategy, with an initial value of 2e-4 and a minimum value of 2e-5. To handle the differences in learning difficulty among different tasks, a course learning mechanism was designed, focusing on relatively simple single-modal tasks in the early stages of training, and then gradually increasing the proportion of cross-modal tasks.

[0094] This embodiment also features deep optimizations for computational efficiency. Mixed-precision training is implemented on GPUs, using FP16 or FP32 precision for different types of computational operations. A training pipeline is implemented through overlapping computation and data loading, while simultaneously achieving a distributed training strategy combining cross-GPU model parallelism and data parallelism. Furthermore, gradient checkpointing and selective gradient accumulation are used to balance memory usage and training efficiency.

[0095] To enhance the model's generalization ability, this embodiment employs various data augmentation strategies. For visual data, operations including random cropping, color perturbation, and occlusion are implemented; for text data, methods such as back-translation augmentation and synonym replacement are used. Simultaneously, adversarial training techniques are employed to improve the model's robustness by adding perturbations to both the feature space and input space, and optimizing model parameters using adversarial example loss.

Claims

1. A deep collaborative multi-task feature learning method based on mutual information regularization, characterized in that, include: Construct a deep learning framework based on mutual information constraints; Design a mutual information calculation module for variational inference; Implement the KL divergence regularization optimization mechanism to obtain the optimized multi-task feature learning model.

2. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 1, characterized in that: The construction of the deep learning framework based on mutual information constraints includes constructing a mutual information objective function: Among them, L MI The objective function is defined as mutual information; I(X;Y) represents the mutual information between the input feature X and the task output Y; β is a weighting coefficient; KL(·||·) represents the KL divergence. φ is the encoder distribution, where φ represents the encoder parameters, z is the learned latent feature representation, and p(z) is the predefined prior distribution.

3. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 2, characterized in that: The mutual information calculation module for the design variational inference includes calculating the variational lower bound: Where I(X;Y) represents the mutual information between random variables X and Y; E represents the expectation operator; p(x,y) represents the joint probability distribution of input data x and label y; q ψ (y|z) represents a decoder network with parameter ψ that predicts label y based on latent representation z; p(y) represents the marginal probability distribution of label y.

4. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 3, characterized in that: The implementation of the KL divergence regularization optimization mechanism includes optimizing the KL divergence constraint: in, Indicates by parameters A defined conditional probability distribution, representing the encoder distribution of the latent variable z given an input x; p(z) represents the prior distribution of the latent variable z; E represents the expectation operation; (z|x) represents the log probability of the encoder distribution, and logp(z) represents the log probability of the prior distribution.

5. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 4, characterized in that: It also includes feature extraction optimization, which includes: Introducing a multi-layer encoder structure; Design an adaptive feature selection mechanism; Implement a cross-task feature sharing module.

6. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 5, characterized in that: It also includes mutual information calculation optimization, which includes: A batch estimation strategy is adopted; Implement parallel computing mechanisms; Introduce dynamic weight adjustment.

7. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 6, characterized in that: It also includes training process optimization, which includes: Design adaptive learning rate adjustment; Implement gradient accumulation update; Construct a multi-task scheduling mechanism.

8. The deep collaborative multi-task feature learning method based on mutual information regularization as described in claim 7, characterized in that: The module for enabling cross-task feature sharing includes: Dynamic task weight allocation; Cross-task knowledge transfer; Feature adaptive fusion.

9. A deep collaborative multi-task feature learning method based on mutual information regularization using the method described in any one of claims 1-7, characterized in that, include: The deep learning framework module is used to build deep learning frameworks based on mutual information constraints. The mutual information calculation module is used to design mutual information calculation for variational inference; The optimization mechanism module is used to implement the KL divergence regularization optimization mechanism.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the deep collaborative multi-task feature learning method based on mutual information regularization as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the deep collaborative multi-task feature learning method based on mutual information regularization as described in any one of claims 1 to 8.

Images