Parallel continual learning method and device, storage medium, electronic device, computer program product

Abstract

The application discloses a parallel continual learning method and device, a storage medium, an electronic device and a computer program product, and aims to solve the training instability and task interference caused by the asynchrony and diversity of data flow in PCL. The method comprises the following steps: constructing a parallel continual learning model comprising a shared feature extraction network and a task-specific classifier; constructing a dynamic gradient optimization module through gradient matrix singular value decomposition to balance the gradient contribution between tasks; introducing feature orthogonal constraint in the shared convolution layer to optimize the shared feature space; and updating the parameters by combining the two types of optimization results, so that the related weights can be adaptively adjusted. The application improves the stability and coordination of multi-task training, enhances the generalization ability and robustness of the model, and is suitable for complex systems such as automatic driving and intelligent monitoring.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a parallel continuous learning method and apparatus, storage medium, electronic device, and computer program product. Background Technology

[0002] In the fields of artificial intelligence and machine learning, Parallel Continual Learning (PCL) aims to enable models to process dynamic data streams from multiple sources simultaneously, making it suitable for complex systems such as autonomous driving and intelligent monitoring. PCL improves training efficiency by learning multiple tasks in parallel and can better adapt to complex scenarios in real-world applications. However, in real-world applications, these data streams exhibit high dynamism and uncertainty. Specifically, the start and end times of different data streams are often asynchronous, and the data content in each stream may be completely different. This asynchronicity and diversity of data streams pose significant challenges to PCL, particularly in maintaining model performance and training stability.

[0003] Orthogonalization techniques have been proven in deep learning to improve model stability and generalization ability. By forcing network parameters or feature representations to maintain orthogonality, redundant information can be reduced, improving the model's expressive efficiency. Orthogonalization methods have shown significant effects in single-task learning; for example, orthogonal weight initialization can accelerate network convergence, and orthogonal constraints can prevent gradient vanishing or exploding problems. In multi-task learning, orthogonalization helps reduce interference between tasks and improve the learning performance of each task. However, existing orthogonalization methods are mainly designed for single-task learning or traditional multi-task learning and are difficult to apply directly to dynamically changing PCL scenarios. The special characteristic of PCL is that it needs to handle multiple asynchronous and diverse data streams, which poses new challenges to orthogonalization techniques. Traditional orthogonalization methods may not be effective in handling the task dynamism and data diversity in PCL, requiring new strategies to adapt to this complex learning environment. Summary of the Invention

[0004] The purpose of this invention is to provide a parallel continuous learning method and apparatus, storage medium, electronic device, and computer program product to effectively solve the training instability problem in PCL, while considering the stability during forward and backward propagation. In addition, it can flexibly adapt to the asynchronicity and diversity of data flow in PCL, and ensure that the model can effectively learn and adapt to new tasks while maintaining the independence between tasks, thereby improving the performance and reliability of PCL in complex real-world application scenarios.

[0005] To achieve the objectives of this invention, the technical solution provided by this invention is as follows: First aspect This invention provides a parallel continuous learning method, comprising the following: A parallel continuous learning model is constructed based on a unified backbone network structure and multi-task source data. A dynamic gradient optimization module is constructed based on the parallel continuous learning model to dynamically adjust the gradient relationship between tasks during multi-task training. The shared feature space is optimized based on the feature orthogonal constraint mechanism to reduce feature interference between tasks and enhance task-specific representation capabilities. The model parameters are updated jointly based on the gradient optimization results and the feature optimization results.

[0006] Furthermore, the construction of the parallel continuous learning model based on a unified backbone network structure and multi-task source data includes: Determine general feature representations based on shared feature extraction networks; Establish task-specific classifiers based on the target data of different tasks; A multi-task joint training framework is formed based on the shared feature representation and the task-specific classifier.

[0007] Furthermore, the inter-task gradient adjustment based on the dynamic gradient optimization module includes: Obtain gradient information for each task during the parallel training phase; The parameters are adjusted based on the gradient differences between tasks, and the gradient contributions are dynamically balanced. The shared network parameters are updated based on the balanced gradient to alleviate gradient conflicts between tasks.

[0008] Furthermore, the step of updating the shared network parameters based on the balanced gradient includes: The overall update direction is determined based on the consistency of gradient direction and the balance of magnitude among tasks; The backbone network parameters are optimized and updated based on the comprehensive update direction.

[0009] Furthermore, the optimization of the shared feature space based on the feature orthogonal constraint mechanism includes: Introduce feature orthogonality constraints in shared convolutional feature layers; Feature separation is achieved by limiting the correlation between convolutional kernels; The parameters of the shared feature extraction module are updated based on the optimization results of orthogonal constraints.

[0010] Furthermore, the method also includes: The gradient optimization strength and feature constraint weights are adaptively adjusted based on the differences in tasks during training. The parameters of the backbone network and the task-specific classifier are jointly updated based on the dynamic adjustment results.

[0011] Second aspect This invention provides a parallel continuous learning apparatus for executing the aforementioned parallel continuous learning method, comprising: The first building module is used to construct a parallel continuous learning model based on a unified backbone network structure and multi-task source data. The second building module is used to establish a dynamic gradient optimization module and perform dynamic balancing adjustments based on the gradient relationships between tasks. The first optimization module is used to optimize the shared feature space based on the feature orthogonality constraint mechanism. The update module is used to jointly update model parameters based on gradient optimization results and feature optimization results.

[0012] Third aspect The present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the parallel continuous learning method described above.

[0013] Fourth aspect The present invention provides a storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the parallel continuous learning method described above.

[0014] Fifth aspect The present invention provides a computer program product, including a computer program that, when executed by a processor, implements the parallel continuous learning method.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The technical solution provided by this invention effectively mitigates gradient conflicts and optimizes the feature space in multi-task parallel training by introducing a dynamic gradient optimization mechanism and a feature orthogonality constraint mechanism. This method not only improves the stability and coordination of multi-task learning but also enhances the model's generalization ability and robustness in complex dynamic scenarios, demonstrating promising application prospects. Attached Figure Description

[0016] To more intuitively present the technical solution of the present invention and its advantages over the prior art, the illustrations involved in the specification will be briefly introduced below. These illustrations show some specific embodiments of the present invention, but do not constitute a limitation on the scope of the present invention. Those skilled in the art, based on an understanding of these illustrations, can derive other related graphic representations without creative effort. Next, the present invention will provide a brief description of each illustration to facilitate a more comprehensive understanding of the technical content and innovative points of the present invention.

[0017] Figure 1 This is a flowchart illustrating the parallel continuous learning method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the changes in the average accuracy of multiple tasks based on the CIFAR-100 dataset under different task scenarios, provided by an embodiment of the present invention. Figure 3 This is a schematic diagram showing the comparison of model performance under different gradient orthogonalization intensities provided in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the degree of conflict between tasks before and after feature orthogonal constraints provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the parallel continuous learning method provided in this embodiment includes the following steps: Step 1: Construct a parallel continuous learning model based on a unified backbone network structure and multi-task source data; specifically including the following: Determine general feature representations based on shared feature extraction networks; This invention first constructs a continuous learning model based on multi-source dynamic data streams. The multi-source data includes data distributions from different task domains, used to simulate task switching and data changes in a real-world continuous learning environment. In terms of model structure, it adopts a combination of a shared backbone network and multiple task-specific branches. The shared backbone network is used to extract general feature representations of the input data to achieve cross-task knowledge sharing.

[0020] Establish task-specific classifiers based on the target data of different tasks; To address the asynchronous and diverse nature of data streams, the model continuously receives dynamic data from multiple sources and establishes specific classifier branches for different tasks. Let the input data stream be... The output labels are Define the task loss function at each time step. as follows: ; In the formula: This represents the task loss value at time t; N represents the total length of the data stream or the total number of samples; Indicates input data and tags Mathematical expectation on the joint distribution; Represents the input sample data at time t; Indicates the true category label at time t; This represents the set of model parameters that share a backbone network and a task-specific classifier. This represents the forward prediction function of a parallel continuous learning model. This represents a task-specific loss function (such as cross-entropy loss).

[0021] A multi-task joint training framework is formed based on the shared feature representation and the task-specific classifier.

[0022] Based on the shared feature representation and independently optimized task-specific classifier defined above, combined with the loss function at time t... This forms a complete multi-task joint training framework. This framework can utilize model parameters... The update process enables incremental learning of the task sequence, thereby adapting to new tasks while maintaining knowledge of old tasks.

[0023] Step 2: Construct a dynamic gradient optimization module based on the parallel continuous learning model, used to optimize gradients based on multiple tasks. The gradient relationship between tasks is dynamically adjusted during the training process; wherein, the gradient adjustment between tasks based on the dynamic gradient optimization module includes: Obtain gradient information for each task in the parallel training phase and construct the gradient matrix; Assume the set of tasks currently activated in parallel is The set contains M tasks. The corresponding task loss functions are as follows: By using shared parameters Find the partial derivatives and construct the gradient matrix. Each column represents the gradient vector for a task: ; In the formula: Indicates parameters The corresponding gradient matrix has the following dimensions: ( (for parameter dimensions) This represents the loss function of the i-th task with respect to the shared parameters. The gradient vector; M represents the total number of tasks currently being trained in parallel.

[0024] The parameters are adjusted based on the gradient differences between tasks, and the gradient contributions are dynamically balanced. To quantitatively reveal the interactions and conflicts between tasks, this invention performs orthogonal decomposition of the gradient matrix based on condition number theory. First, the gradient matrix... Perform Singular Value Decomposition (SVD): ; The left singular vector matrix represents the orthogonal basis directions of the gradient space; Represents a singular value diagonal matrix. It is a singular value matrix, where Let i be the i-th singular value, and satisfy... ; This represents the transpose of the right singular vector matrix.

[0025] Based on this, the condition number of the gradient matrix is ​​used. To measure the uniformity of gradient distribution: ; In the formula: Represents the gradient matrix The maximum singular value reflects the strength of the dominant gradient direction; Represents the gradient matrix The smallest singular value.

[0026] A higher condition number means that the singular values ​​of the gradient matrix vary greatly. This can lead to gradients from some tasks contributing excessively to model updates, while the contributions from other tasks are suppressed. Therefore, controlling the condition number of the gradient matrix, especially by adjusting the distribution of singular values, can effectively balance the gradient contributions from different tasks, thereby reducing conflicts in gradient directions and optimizing the model update process.

[0027] The shared network parameters are updated based on the balanced gradient to alleviate gradient conflicts between tasks.

[0028] To optimize gradient updates and prevent gradients from dominating the training process for certain tasks, this step involves processing the singular value matrix. By performing nonlinear power scaling, the adjusted singular value matrix is ​​obtained. : ; In the formula: This represents the adjusted singular value matrix; This represents a regulation factor used to control the rate of singular value decay (i.e., the strength of gradient balancing). When hour, That is, the gradient matrix remains unchanged, the gradient contributions between tasks are updated according to the original matrix, and gradient conflicts are not adjusted. hour, In other words, the singular value matrix is ​​adjusted to an identity matrix, and the gradient matrix is ​​made orthogonal. This equalizes the contribution of gradients from all tasks to the model update, thereby eliminating gradient differences and conflicts between tasks. Although orthogonalizing the gradient matrix can eliminate gradient differences and conflicts between tasks, it may also lead to the "dilution" or "ignoring" of information from important tasks.

[0029] In some embodiments, since gradient differences between tasks reflect relative importance, forced balancing may weaken the learning signals of some tasks. Therefore, dynamically adjusting the value of 'a' can automatically balance the learning contributions of tasks based on gradient differences. When the gradient differences between tasks are large, decreasing 'a' can effectively suppress the excessive influence of large gradients on model updates, preventing gradients of some tasks from dominating the training process; while when the gradient differences between tasks are small, appropriately increasing 'a' can preserve differentiated learning between tasks and promote coordination between tasks.

[0030] Finally, the gradient matrix is ​​reconstructed based on the adjusted singular value matrix, and this balanced gradient is then used. Update the backbone network parameters: ; In the formula: This represents the gradient matrix after dynamic balancing, which is used for subsequent backpropagation updates.

[0031] The step of updating the shared network parameters based on the balanced gradient includes: The overall update direction is determined based on the consistency of gradient direction and the balance of magnitude among tasks; The backbone network parameters are optimized and updated based on the comprehensive update direction.

[0032] Step 3: Optimize the shared feature space based on the feature orthogonality constraint mechanism to reduce feature interference between tasks and enhance task-specific representation capabilities; wherein, the optimization of the shared feature space based on the feature orthogonality constraint mechanism includes: Introduce feature orthogonality constraints in shared convolutional feature layers; In the continuous learning process provided by this invention, there is often a strong correlation between convolutional feature layers, and features from different tasks will interfere with each other in the shared space.

[0033] Therefore, this invention proposes a feature optimization method based on orthogonal constraints.

[0034] Since different tasks share feature representations in the same layer of the model, mutual interference between features remains a potential problem in the model learning process. To further improve the independence between tasks and avoid information confusion in the shared feature layer, feature optimization methods must be introduced.

[0035] In convolutional neural networks, the feature extraction process can be represented as: ; In the formula: This represents the output feature map of the convolutional layer; This represents the input feature map of the convolutional layer; Represents the convolution kernel tensor; This represents the convolution operation.

[0036] Feature separation is achieved by limiting the correlation between convolutional kernels; Based on the autocorrelation property of convolution, this invention defines a constraint term based on soft orthogonal convolution regularization to ensure that the convolution kernels maintain orthogonality along the feature dimension, thereby reducing feature overlap between different tasks. Feature separation is achieved by restricting the product of the convolution kernel matrix and its transpose to approximate the identity matrix.

[0037] The parameters of the shared feature extraction module are updated based on the optimization results of orthogonal constraints.

[0038] To effectively integrate feature orthogonality constraints into the training process, this invention constructs a joint objective function that includes task loss and orthogonal regularization loss. The soft orthogonal regularization loss is introduced into the overall loss function of the backbone network and assigned corresponding weight coefficients, in the following form: ; In the formula: This represents the total loss function for model training; This represents the classification loss (such as cross-entropy loss) for a task-specific branch. Let represent the regularization weight coefficients of the orthogonal constraint term, and 0; This represents the orthogonal regularization loss term. This represents a convolution kernel rearranged into matrix form; Represents the convolution kernel matrix The transpose of the matrix; The identity matrix is ​​used to constrain the decorrelation between convolutional kernels and ensure the orthogonality between filters. This represents the Frobenius norm.

[0039] By minimizing this loss function The model can force the filters to remain orthogonal when updating parameters, thereby reducing the redundancy of learned features, generating more independent representations for different tasks, and effectively reducing negative transfer and inter-task interference.

[0040] Step 4: Update the model parameters jointly based on the gradient optimization results and feature optimization results.

[0041] First, the joint loss function with orthogonal constraints constructed in step 3 is used to calculate the joint loss and initial gradient. Calculate the total loss of the model at the current time step: ; In the formula: Loss is categorized for multi-task classification; The loss is the orthogonal regularization loss for the features; These are the weighting coefficients for the feature constraints.

[0042] Apply dynamic gradient optimization; During backpropagation, instead of directly using the original gradient for updates, the optimized gradient matrix based on singular value scaling determined in step 2 is applied. This matrix is ​​adjusted by factors Gradient magnitudes for different tasks were reweighted to mitigate gradient conflicts.

[0043] Execute parameter update; Utilizing the optimized gradient information By combining standard optimizers (such as stochastic gradient descent SGD or Adam) with shared backbone network parameters and task-specific classifier parameters... Iterative updates are performed to achieve balanced learning across multiple tasks while enhancing feature independence.

[0044] Preferably, the method further includes: The gradient optimization strength and feature constraint weights are adaptively adjusted based on the differences in tasks during training. Adaptively adjust gradient optimization strength; Dynamically adjust the singular value scaling index based on the gradient differences between tasks. Since gradient differences between tasks reflect relative importance, forcibly balancing them may weaken the learning signals for some tasks. Therefore, the gradient values ​​are dynamically set based on the distribution of singular values ​​in the gradient matrix. : When the gradient difference between tasks is detected to be too large, i.e., the condition number is... When it is high, reduce it appropriately. The value is adjusted to suppress the excessive dominance of large gradients in model updates and prevent gradient conflicts; when the gradient differences between tasks are small, the value is appropriately increased. A value (close to 1) is used to preserve differentiated learning information between tasks and promote coordination between tasks.

[0045] Adaptive adjustment of feature constraint weights; Based on the model's generalization performance on the validation set or the degree of feature overlap between tasks, dynamically adjust the weight coefficients of the feature orthogonality constraint. Increase [the intensity / strength] during the initial training phase or when there are significant differences in tasks. To enhance feature separation and establish independent feature subspaces; in the later stages of training, the size can be appropriately reduced. This allows the model to focus more on reducing classification loss. This improves the final recognition accuracy.

[0046] The parameters of the backbone network and the task-specific classifier are jointly updated based on the dynamic adjustment results.

[0047] Based on the adaptively adjusted parameters, the parameters of the backbone network and task-specific classifiers are updated in real time, thereby achieving a dynamic balance between the model's training stability and plasticity.

[0048] This invention implements a two-layer optimization mechanism based on feature orthogonal constraints and gradient decomposition in continuous learning scenarios. Compared with traditional parallel continuous learning methods, this invention can effectively reduce feature interference and gradient conflicts between tasks, and improve the adaptability and robustness of the model in multi-task dynamic environments.

[0049] The above method will be illustrated with specific examples below.

[0050] This invention conducted experiments on the CIFAR-100 benchmark dataset to evaluate the model's performance in both task-based incremental learning and category-based incremental learning scenarios. Task division and experimental design were tailored to the characteristics of each scenario. The CIFAR-100 dataset contains 100 categories, divided into 20 tasks, with each task assigned a fixed subset of category labels. This dataset is primarily used to validate the model's performance in medium-scale task scenarios.

[0051] To comprehensively evaluate the performance of this invention framework, this paper uses two performance metrics: average accuracy of the task (...). ) and forgetting rate ( The task average precision measures the overall accuracy of the model across all tasks, and its calculation formula is as follows: ; In the formula, M represents the total number of tasks. Let be the accuracy of task t at the end of the experiment. Forgetting rate ( This is used to quantify the degree of forgetting the model for each task, and is defined as: ; In the formula, This represents the initial accuracy for task t. These two metrics clearly assess the model's ability to retain knowledge from previous tasks when dealing with new ones. All experiments were conducted on hardware equipped with an RTX 4090 GPU.

[0052] Table 1 shows a comparison of the final performance accuracy and forgetting rate of the task incremental PCL model on the CIFAR-100 dataset.

[0053] Table 1

[0054] Figure 2 The figure illustrates the performance trajectory of the learned tasks during the model training process of this invention. Each point in the figure represents the state after the training of a task, reflecting the average test accuracy of all currently running and completed tasks. The results show that the proposed method significantly improves model performance on three benchmark datasets under both task incremental learning and category incremental learning settings, and consistently outperforms the baseline model, fully demonstrating its robustness and generalization ability.

[0055] Figure 3 In this invention, to analyze the impact of the value of parameter 'a' in the singular value adjustment strategy on model performance, experiments were conducted on a benchmark dataset, and curves showing the changes in the value of 'a' versus the average accuracy (Ae) of the task were plotted. In the experiments, the value range of parameter 'a' was [0,1], corresponding to the change from a fully orthogonal gradient matrix (a=0) to the original gradient matrix (a=1). Due to display limitations, the horizontal axis values ​​in the graph are presented within a compressed range to more intuitively observe performance changes in key areas.

[0056] The following results can be observed from this experiment. On both datasets, as 'a' increases, the model performance exhibits a trend of first increasing, then decreasing, and finally stabilizing. Specifically, when 'a' approaches 0, the model's gradient matrix tends to be orthogonal, the gradient distribution is more balanced, and gradient conflicts are significantly reduced, thus achieving optimal task average accuracy. However, as 'a' further increases, the gradient matrix gradually deviates from the orthogonal distribution. Although task-specific information is enhanced, the impact of gradient conflicts gradually becomes apparent, leading to a decrease in model performance, which eventually stabilizes when 'a' approaches 1. Notably, the experimental results on both datasets consistently show that the optimal value of model performance occurs in the region where 'a' is close to 0. This indicates that the singular value adjustment strategy, by reasonably scaling the singular value distribution, can achieve an effective balance between gradient optimization and task specificity. This conclusion not only verifies the effectiveness of the singular value adjustment strategy but also provides clear guidance for the selection of parameter 'a': in practical applications, a moderately small value of 'a' can maximize model performance and ensure training stability.

[0057] Figure 4 The study demonstrates that, on a benchmark dataset, the feature stabilization method using orthogonal constraints based on the DBT matrix (DBTorth) significantly reduces the condition number of convolutional layers, decreases filter correlation, and slightly reduces the filter norm compared to the model without orthogonal constraints (plain).

[0058] These results demonstrate that the DBT-based orthogonal constraint method can effectively improve the numerical properties of the model, thereby enhancing the stability of training and the independence of feature representations.

[0059] The above experimental results show that this invention provides a comprehensive solution based on gradient optimization and feature optimization to address the gradient conflict problem in the multi-task training process of parallel continuous learning.

[0060] This approach combines a dynamic gradient update direction adjustment mechanism with a feature optimization method based on orthogonal constraints, which significantly improves the model's adaptability and training stability in dynamic multi-task scenarios.

[0061] Specifically, this invention employs a gradient optimization method based on singular value decomposition to adjust the singular value distribution of the gradient matrix during multi-task training, thereby achieving a dynamic balance of gradient contributions between tasks, reducing update conflicts caused by inconsistent gradient directions, and thus optimizing the gradient interaction efficiency in parallel training.

[0062] Meanwhile, this invention combines a feature optimization mechanism based on orthogonal constraints to apply orthogonalization constraints to the convolutional kernels of the convolutional neural network, ensuring the independence between feature representations of different tasks, reducing feature interference from shared feature layers, and enhancing the expressive power of task-specific features.

[0063] Through the above technical solution, this invention can effectively improve the task coordination and training efficiency of the model in multi-task continuous learning scenarios, and its superior performance has been verified on multiple benchmark datasets. Experimental results show that this method has higher convergence stability and generalization ability than existing optimization strategies, and can maintain high prediction accuracy and robustness in complex dynamic task environments.

[0064] Furthermore, this invention exhibits excellent scalability, enabling its application in larger-scale or highly dynamic data environments. For more complex task scenarios, adaptive adjustment mechanisms can be incorporated to further enhance the model's computational efficiency and generalization performance.

[0065] In addition, corresponding to the above method, the present invention provides a parallel continuous learning apparatus for executing the parallel continuous learning method, comprising: The first building module is used to construct a parallel continuous learning model based on a unified backbone network structure and multi-task source data. The second building module is used to establish a dynamic gradient optimization module and perform dynamic balancing adjustments based on the gradient relationships between tasks. The first optimization module is used to optimize the shared feature space based on the feature orthogonality constraint mechanism. The update module is used to jointly update model parameters based on gradient optimization results and feature optimization results.

[0066] In addition, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the parallel continuous learning method described above.

[0067] In addition, the present invention provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the parallel continuous learning method described above.

[0068] In addition, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the parallel continuous learning method described above.

[0069] Finally, it should be noted that the above embodiments are merely illustrative and explanatory of the present invention, and are not intended to limit the present invention to the scope of the described embodiments. Furthermore, those skilled in the art will understand that the present invention is not limited to the above embodiments, and many more variations and modifications can be made based on the teachings of the present invention, all of which fall within the scope of protection claimed by the present invention.

Claims

1. A parallel continuous learning method, characterized in that, Including the following: A parallel continuous learning model is constructed based on a unified backbone network structure and multi-task source data. A dynamic gradient optimization module is constructed based on the parallel continuous learning model to dynamically adjust the gradient relationship between tasks during multi-task training. The shared feature space is optimized based on the feature orthogonal constraint mechanism to reduce feature interference between tasks and enhance task-specific representation capabilities. The model parameters are updated jointly based on the gradient optimization results and the feature optimization results.

2. The parallel continuous learning method according to claim 1, characterized in that, The parallel continuous learning model built based on a unified backbone network structure and multi-task source data includes: Determine general feature representations based on shared feature extraction networks; Establish task-specific classifiers based on the target data of different tasks; A multi-task joint training framework is formed based on the shared feature representation and the task-specific classifier.

3. The parallel continuous learning method according to claim 1, characterized in that, The dynamic The gradient optimization module performs gradient adjustments between tasks, including: Obtain gradient information for each task during the parallel training phase; The parameters are adjusted based on the gradient differences between tasks, and the gradient contributions are dynamically balanced. The shared network parameters are updated based on the balanced gradient to alleviate gradient conflicts between tasks.

4. The parallel continuous learning method according to claim 3, characterized in that, The method of updating shared network parameters based on the balanced gradient includes: The overall update direction is determined based on the consistency of gradient direction and the balance of magnitude among tasks; The backbone network parameters are optimized and updated based on the comprehensive update direction.

5. The parallel continuous learning method according to claim 1, characterized in that, The optimization of the shared feature space based on the feature orthogonal constraint mechanism includes: Introduce feature orthogonality constraints in shared convolutional feature layers; Feature separation is achieved by limiting the correlation between convolutional kernels; The parameters of the shared feature extraction module are updated based on the optimization results of orthogonal constraints.

6. The parallel continuous learning method based on gradient orthogonality according to claim 1, characterized in that, The method further includes: The gradient optimization strength and feature constraint weights are adaptively adjusted based on the differences in tasks during training. The parameters of the backbone network and the task-specific classifier are jointly updated based on the dynamic adjustment results.

7. A parallel continuous learning apparatus for performing the method as described in any one of claims 1-6, characterized in that, include: The first building module is used to construct a parallel continuous learning model based on a unified backbone network structure and multi-task source data. The second building module is used to establish a dynamic gradient optimization module and perform dynamic balancing adjustments based on the gradient relationships between tasks. The first optimization module is used to optimize the shared feature space based on the feature orthogonality constraint mechanism. The update module is used to jointly update model parameters based on gradient optimization results and feature optimization results.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the parallel continuous learning method as described in any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the parallel continuous learning method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the parallel continuous learning method as described in any one of claims 1 to 6.

Images