Training method of cross-domain continuous learning model for image classification and image classification method

Abstract

The invention relates to a training method of a cross-domain continuous learning model for image classification and an image classification method. According to an encoder of a Vision Transform module in the model, each Transform layer of the Vision Transform module comprises a multi-head self-attention layer and a feed-forward network; a low-rank adapter is inserted into each multi-head self-attention layer; the training method comprises the following steps: processing an input image through a Vision Transform module to obtain an image block embedded vector sequence, carrying out position coding on the vector sequence, and inputting the vector sequence into a coder; dynamically adjusting the rank in the low-rank adapter and outputting a rank value through an adjusting module according to the performance feedback of the model; the matrix A and the matrix B are initialized through rank values, and only the matrix A and the matrix B are updated in the training process; forward propagation is carried out based on an effective weight formed by the original weight matrix, A and B, and classification features are output; and inputting the classification features into a classification module for prediction to obtain a prediction result, and performing loss function calculation and parameter updating of the low-rank adapter according to the prediction result. According to the image classification method, the classification accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically, to a training method for a cross-domain continuous learning model for image classification and an image classification method. Background Technology

[0002] Continual learning aims to enable models to continuously learn new knowledge from a series of tasks without accessing historical task data, while effectively retaining the ability to recognize previously learned tasks. Exemplar-free continuous learning, as a highly challenging learning paradigm, strictly prohibits the use of any original samples from previous tasks, replay caches, or generative replay mechanisms, relying entirely on the adaptability of the model's internal structure to achieve long-term knowledge accumulation and transfer. The core challenge of this paradigm lies in catastrophic forgetting—when model parameters are updated to adapt to new tasks, the feature representations corresponding to old tasks are easily overwritten or distorted, leading to a sharp decline in performance.

[0003] To alleviate this problem, existing research mainly employs strategies such as regularization constraints, parameter isolation, or gradient control to seek a balance between "stability" and "plasticity" within limited model capacity. In recent years, architectures based on Vision Transformer (ViT) have gradually become the mainstream backbone network for continuous learning due to their powerful global modeling capabilities. The multi-head self-attention mechanism in ViT captures multi-level semantic dependencies in different subspaces through parallel execution of multiple attention heads, effectively fusing local and global contextual information and providing a solid foundation for cross-task feature abstraction. However, standard ViT still faces problems such as tight parameter coupling and easy drift in feature distribution in continuous learning scenarios.

[0004] To improve parameter efficiency and task isolation, LoRA (Low-Rank Adaptation) has been introduced into continuous learning frameworks. LoRA significantly reduces fine-tuning overhead and mitigates parameter interference to some extent by overlaying low-rank increment terms (i.e., ΔW=BA) onto a frozen pre-trained weight matrix, training only a small number of low-rank factors A and B. However, existing LoRA-based continuous learning methods still have two major limitations: First, the rank is fixed. Most methods pre-determine the same rank r for all tasks, and cannot dynamically adjust it according to task complexity, data domain differences, or model performance. This leads to redundant updates on simple tasks, while on complex tasks, the insufficient rank restricts expression, making it difficult to achieve efficient capacity allocation in long-term task sequences.

[0005] Secondly, there is a lack of feature consistency constraints. Since each task learns its LoRA parameters independently and does not explicitly model the evolution of the feature space, low-rank updates from new tasks often highly overlap with the feature subspaces of older tasks, leading to severe feature drift and compromising the stability of historical knowledge. This is especially problematic in cross-domain continuous learning scenarios, where the input data distribution is non-stationary, making existing methods particularly inadequate in ensuring cross-domain generalization capabilities. Summary of the Invention

[0006] This invention provides a training method for a cross-domain continuous learning model for image classification and an image classification method that can achieve task-aware capacity adaptation and improve classification accuracy.

[0007] According to a first aspect of this application, a training method for a cross-domain continuous learning model for image classification is provided, the cross-domain continuous learning model including a Vision Transformer module, a low-rank adapter, an adaptive dynamic rank adjustment module, and a classification module; wherein the encoder of the Vision Transformer module includes multiple stacked Transformer layers, each Transformer layer including a sequentially connected multi-head self-attention layer and a feedforward network; and a low-rank adapter is inserted into each multi-head self-attention layer. The training method includes: The input image is processed by the Vision Transformer module to obtain an image patch embedding vector sequence. The image patch embedding vector sequence is then position-encoded and input into the encoder. The adaptive dynamic rank adjustment module dynamically adjusts the rank in the low-rank adapter based on the model's performance feedback and outputs the rank value. The low-rank factorization matrices A and B of the low-rank adapter are initialized with the rank values, and only the low-rank factorization matrices A and B are updated during training, while the original weight matrix in the Vision Transformer module remains frozen. The Vision Transformer module performs forward propagation based on the effective weights formed by the original weight matrix and the low-rank decomposition factor matrices A and B, and outputs classification features. The classification features are input into the classification module to obtain the prediction results, and the loss function is calculated and the parameters of the low-rank adapter are updated based on the prediction results.

[0008] Optionally, the step of dynamically adjusting the rank in the low-rank adapter based on the model's performance feedback and outputting the rank value includes: Before each task t begins, the initial rank value of task t in the low-rank adapter is initialized according to the preset allocation strategy; During the training process of the model, the slope of the performance trajectory is calculated based on the average accuracy of the model on the seen categories in the most recent tasks. The rank of the task after task t is adjusted based on the slope of the performance trajectory.

[0009] Optionally, the slope of the performance trajectory The calculation formula is as follows: in, Indicates in the task The average accuracy of the model on the seen categories is then given, where M represents the M tasks closest to task t.

[0010] Optionally, adjusting the rank of the task after task t according to the slope of the performance trajectory includes: Preset performance trajectory slope threshold Threshold of rank r ; When the slope of the performance trajectory At that time, the rank value of task t+1 after task t in the low-rank adapter is determined according to... ,in This represents the rank value corresponding to task t+1. This represents the rank value corresponding to task t; when At that time, the rank value corresponding to task t+1 after task t is initialized using the preset allocation strategy.

[0011] Optionally, the adaptive dynamic rank scheduling module is equipped with a pause mechanism: if three consecutive tasks trigger rank adjustment, the rank adjustment of the subsequent two consecutive tasks is automatically paused so that the rank of the subsequent two consecutive tasks adopts the rank value of the most recently adjusted rank.

[0012] Optionally, the effective weights based on the original weight matrix and the low-rank decomposition factor matrices A and B are specifically as follows: in, Let W represent the weight matrix corresponding to task t, and let W represent the original weight matrix of the multi-head self-attention layer. Indicates task Low-rank update volume Indicates task The corresponding low-rank factorization matrix B, Indicates task The corresponding low-rank factorization matrix A, This represents the feature dimension of the input image.

[0013] Optionally, the model further includes a feature drift suppression module, and the training method further includes: The feature drift suppression module collects the input feature matrix of each Transformer layer in the model, calculates the covariance of the input feature matrix, constructs an anti-drift subspace through singular value decomposition, and projects the gradient parameters of the low-rank adapter onto the anti-drift subspace to obtain the corrected gradient parameters. Update the low-rank decomposition factor matrices A and B in the low-rank adapter based on the corrected gradient parameters.

[0014] Optionally, the covariance of the input feature matrix is ​​calculated and an anti-drift subspace is constructed through singular value decomposition. The gradient parameters of the low-rank adapter are projected onto the anti-drift subspace to obtain the corrected gradient parameters, including: Calculate the input feature matrix covariance matrix Used to describe the feature distribution of this layer in a new task: in, This indicates the number of the Transformer layer, where t is the t-th task; Perform singular value decomposition on the covariance matrix: in, It is a principal component orthogonal base. It is a singular value diagonal matrix; The principal components are filtered to obtain the orthogonal projection matrix corresponding to each Transformer layer. : in This represents the principal component submatrix composed of the k principal component directions after filtering. The principal component submatrix projects the gradient onto the anti-drift subspace. gradient parameters for each Transformer layer Apply the orthogonal projection matrix The corrected gradient parameters are obtained. : in, Indicates that task t is in the first position. The original gradient values ​​of the Transformer layer.

[0015] Optionally, the loss function L for: in, The standard classification loss for task t is represented by... This represents the auxiliary triplet loss for task t. and The weight balance parameters for the auxiliary triplet loss and the calibration regularization term are controlled separately. The calibration regularization term for task t is calculated using the following formula: in, This represents the element-wise Hadamard product. Denotes the Frobenius norm; Indicates task W represents the original weight matrix of the multi-head self-attention layer. Indicates task Low-rank update volume This represents the low-rank update term for task t.

[0016] According to a second aspect of this application, an image classification method is provided, wherein the cross-domain continuous learning model is trained using the training method described in the first aspect above to obtain a trained cross-domain continuous learning model. The trained cross-domain continuous learning model is used to classify the images to be classified and obtain the classification results.

[0017] According to a third aspect of this application, an electronic device is provided, comprising: Memory, used to store one or more computer programs; A processor, when the one or more computer programs are executed by the processor, implements the training method described in the first aspect above.

[0018] According to a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the training method described in the first aspect above.

[0019] Based on any of the above aspects, the training method for the cross-domain continuous learning model for image classification, the image classification method, the electronic device, and the computer-readable storage medium provided in the embodiments of this application have the following beneficial effects: (1) By setting an adaptive dynamic rank scheduling module in the model, the rank value of the low-rank adapter is dynamically adjusted according to the trend of task performance changes, so that the model capacity adapts to the task complexity, breaking through the limitation of the fixed rank of the traditional low-rank adapter, realizing task-aware capacity adaptation, and optimizing the model resource utilization efficiency under the premise that the total number of parameters is controllable. (2) By setting a low-rank adapter in the model and inserting a low-rank adaptation branch on the pre-trained weights, task-specific updates can be achieved with a smaller number of parameters; (3) By constructing an anti-drift subspace based on the negative accumulation of historical task weights in the feature drift suppression module in the model, the gradient direction is constrained in backpropagation to maintain feature stability, thereby explicitly constraining the parameter update direction and avoiding the destruction of the feature distribution of old tasks by new task learning. This significantly reduces cross-task interference and significantly improves the model's recognition accuracy of learned tasks without any historical sample playback. This improvement shows stronger knowledge stability, especially in long task sequences and cross-domain scenarios.

[0020] (4) By introducing a calibration regularization term into the model’s loss function, the weights of the historical and current tasks are fused to calibrate gradient differences, thereby enhancing cross-task knowledge consistency and mitigating forgetting.

[0021] (5) Based on the efficient structure of the low-rank adapter, this application introduces three key strategies: negative cumulative feature drift suppression mechanism, adaptive dynamic rank scheduling, and update calibration regularization term. These strategies work synergistically from three levels: spatial constraints, structural adjustment, and optimized calibration, to achieve a stable, flexible, and adaptive continuous learning process. Experiments have shown that this application outperforms other methods on both old and new tasks, demonstrating not only higher accuracy but also stronger stability and flexibility, providing a more reliable solution for knowledge accumulation and transfer in continuous learning scenarios. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is an illustrative application scenario diagram of the training method and / or image classification method for the cross-domain continuous learning model for image classification provided in this embodiment.

[0024] Figure 2 This is a flowchart illustrating the training method for the cross-domain continuous learning model for image classification provided in this embodiment.

[0025] Figure 3 This embodiment provides a schematic diagram of the feature drift suppression module training process.

[0026] Figure 4 This embodiment provides a schematic diagram of the training process for updating calibration regularization terms.

[0027] Figure 5 This embodiment provides a schematic diagram of the electronic device. Detailed Implementation

[0028] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application. To better illustrate the following embodiments, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] This embodiment provides a technical solution that can solve the above problems. The specific implementation of this application will be described in detail below with reference to the accompanying drawings.

[0032] Exemplary examples are illustrated in the diagram below, showing a training method for a cross-domain continuous learning model for image classification and / or an application scenario of an image classification method provided in this application. For instance... Figure 1 As shown, the application scenario includes at least a server 100 and a terminal 200 that can communicate with the server 100.

[0033] Understandably, the server 100 can be an independent electronic device or a cluster of multiple electronic devices; the terminal 200 can be a smartphone terminal, personal computer, tablet computer, vehicle terminal, etc., but is not limited to these.

[0034] In one possible implementation, server 100 and terminal 200 may respectively execute the training method and / or image classification method of the cross-domain continuous learning model for image classification provided in the embodiments of this application. Alternatively, part of the training method and / or image classification method of the cross-domain continuous learning model for image classification provided in the embodiments of this application may be executed in server 100 and part may be executed in terminal 200.

[0035] This embodiment provides a training method for a cross-domain continuous learning model for image classification. The cross-domain continuous learning model includes a Vision Transformer module, a low-rank adapter, an adaptive dynamic rank adjustment module, and a classification module. The encoder of the Vision Transformer module includes multiple stacked Transformer layers. Each Transformer layer includes a sequentially connected multi-head self-attention layer and a feedforward network. A low-rank adapter is inserted into each multi-head self-attention layer. like Figure 2 As shown, the training method can be further divided into the following steps: S100. The input image is processed by the Vision Transformer module to obtain an image block embedding vector sequence. The image block embedding vector sequence is then position-encoded and input into the encoder. Specifically, the Vision Transformer module, as the backbone network of the entire model, divides the input image into image blocks of fixed size, and obtains a serialized image block embedding sequence (pach embedding) by linear mapping of the image blocks. The image block embedding sequence is then position-encoded and input into the encoder for processing.

[0036] S200: The adaptive dynamic rank adjustment module dynamically adjusts the rank in the low-rank adapter based on the model's performance feedback and outputs the rank value. In its implementation, the Adaptive Dynamic Rank Scheduling (ADRS) module aims to address the trade-off between adaptability and stability in continuous learning where low-rank adapters use fixed-rank values ​​(r). In previous continuous learning frameworks without sample replay, each task often employed a uniform rank value (r), resulting in a lack of flexibility in allocating the model's expressive power across different tasks: larger ranks, while providing greater plasticity, easily lead to overfitting and feature drift; smaller ranks, while improving generalization, limit the representational power for complex tasks.

[0037] The adaptive dynamic rank scheduling module in this embodiment introduces a dynamically adjustable rank update mechanism, enabling the low-rank adapter of each task to adaptively adjust its rank value according to performance trends, thereby achieving a redistribution of capabilities among tasks and significantly improving the model's anti-forgetting performance while maintaining parameter efficiency.

[0038] Specifically, the core idea of ​​the Adaptive Dynamic Rank Scheduling (ADRS) module is to adaptively adjust the rank decay rate in the low-rank adapter using task performance changes as feedback signals. S210. Before each task t begins, initialize the initial rank value of task t in the low-rank adapter according to the preset allocation strategy. For example, the preset allocation strategy can be to decrease the rank value step by step based on the task number or empirical rules (e.g., the initial task is higher and the subsequent tasks are gradually lower).

[0039] S220. During the training process of the model, the performance trajectory slope is calculated based on the average accuracy of the model on the seen categories in the most recent tasks; in a preferred embodiment, the performance trajectory slope... The calculation formula is as follows: in, Indicates in the task The model then calculates the average accuracy on the seen categories, where M represents the M nearest tasks to task t. The adaptive dynamic rank scheduling module calculates the slope of the performance trajectory based on the trend of the average accuracy on the M nearest tasks. To measure the stability of the model.

[0040] S230. Adjust the rank of the task after task t according to the slope of the performance trajectory.

[0041] In specific implementation, step S230 may include the following steps: Preset performance trajectory slope threshold Threshold of rank r ; When the slope of the performance trajectory When this occurs, it indicates that the model performance has entered a stagnant or declining phase. At this point, the adaptive dynamic rank scheduling module triggers dynamic rank adjustment, adjusting the rank value of task t+1 after task t in the low-rank adapter according to... This is achieved by attenuating parameters, thereby limiting the parameter update space and preventing interference from new tasks from amplifying the problem; among which... This represents the rank value corresponding to task t+1. This represents the rank value corresponding to task t; when At that time, the rank value corresponding to task t+1 after task t is initialized using the preset allocation strategy.

[0042] Understandably, as tasks progress, earlier tasks have higher ranks, thus possessing a broader feature space and stronger expressive power; while subsequent tasks, due to the gradual shrinking of rank values, update only occurs in low-dimensional subspaces, effectively reducing the overlap with the feature space of older tasks, thereby suppressing feature drift and improving cross-task discriminability.

[0043] In the specific implementation process, the adaptive dynamic rank scheduling module is equipped with a pause mechanism: if three consecutive tasks trigger rank adjustment, the rank adjustment of the subsequent two consecutive tasks will be automatically paused so that the rank of the subsequent two consecutive tasks adopts the rank value of the most recent adjustment.

[0044] The pause mechanism avoids instability caused by excessively frequent rank adjustments, allowing the model to maintain a smooth transition during dynamic changes. This "adjust-pause" cycle enables the adaptive dynamic rank scheduling module to self-stabilize during training, balancing performance recovery and stable convergence.

[0045] The adaptive dynamic rank scheduling module implements a performance-driven structure adaptation mechanism. It dynamically adjusts the rank value based on task complexity and model performance, and implicitly constrains parameter update direction by controlling the rank decay rate, thereby achieving gradual shrinking and drift suppression of the low-rank adapter subspace. This mechanism provides an efficient, robust, and interpretable anti-forgetting optimization strategy for the combination of Vision Transformer and LoRA in continuous learning scenarios that do not rely on sample replay.

[0046] S300. Initialize the low-rank decomposition factor matrices A and B of the low-rank adapter using the rank values, and update only the low-rank decomposition factor matrices A and B during training, while keeping the original weight matrix in the Vision Transformer module frozen. Specifically, to achieve efficient parameter updates and knowledge transfer, the model in this embodiment inserts low-rank adapters (LoRAs) into each multi-head self-attention layer of the VisionTransformer encoder, updating only the parameters during training. The backbone parameters are kept frozen, thus significantly reducing the amount of updates.

[0047] In a preferred embodiment, the low-rank adapters are embedded into the query and value mapping layers of the corresponding multi-head self-attention layers: Here, K and V are the key vector and value vector, respectively, in the multi-head self-attention layer. Indicates the input image. Represents the key projection weight matrix. The value projection weight matrix represents the total weight. and All are low-rank decomposition factor matrices of the bond branches. and All are low-rank factorization matrices of the value branches. This method implements the addition of a low-rank adapter to the attention linear layer (e.g., ...) in the Vision Transformer module. , The rank value controlled by the adaptive dynamic rank scheduling module determines the task-specific low-rank adapter, enabling the model to flexibly adjust feature representation through low-rank branches in different tasks.

[0048] The S400 and Vision Transformer modules perform forward propagation based on the effective weights formed by the original weight matrix and the low-rank decomposition factor matrices A and B, and output classification features. Specifically, the effective weights based on the original weight matrix and the low-rank decomposition factor matrices A and B are as follows: in, Let W represent the weight matrix corresponding to task t, and let W represent the original weight matrix of the multi-head self-attention layer. Indicates task Low-rank update volume Indicates task The corresponding low-rank factorization matrix B, Indicates task The corresponding low-rank factorization matrix A, This represents the feature dimension of the input image.

[0049] Understandably, during the forward inference phase, the output of the low-rank adapter is added to the output of the backbone to form a fused feature, enabling rapid adaptation to the current task while preserving the semantic space stability of historical tasks, thereby achieving efficient transfer and low forgetting in continuous learning.

[0050] S500: Input the classification features into the classification module to make predictions and obtain prediction results. Then, calculate the loss function and update the parameters of the low-rank adapter based on the prediction results.

[0051] In specific implementation, the model in this application embodiment also includes a feature drift suppression module (DRS, Drift-Resistant Space) designed to address the problem of feature space drift as the task changes, thereby enhancing the model's resistance to forgetting. DRS explicitly models the feature distribution of each layer, constructing a constraint mechanism to resist feature drift, enabling the model to maintain stable representations of previous tasks when learning new tasks. Overall, DRS incorporates changes in the feature space into the training process by capturing the statistical structure of the input features of each layer, thereby guiding the model to maintain adaptability to new tasks while minimizing the destruction of historical knowledge during parameter updates.

[0052] Specifically, the training method described in this application embodiment further includes: A700: The feature drift suppression module collects the input feature matrix of each Transformer layer in the model, calculates the covariance of the input feature matrix and constructs an anti-drift subspace through singular value decomposition, and projects the gradient parameters of the low-rank adapter onto the anti-drift subspace to obtain the corrected gradient parameters. A900. Update the low-rank decomposition factor matrices A and B in the low-rank adapter according to the corrected gradient parameters.

[0053] like Figure 3 As shown, in the specific implementation process, the covariance of the input feature matrix is ​​calculated and an anti-drift subspace is constructed through singular value decomposition. The gradient parameters of the low-rank adapter are projected onto the anti-drift subspace to obtain the corrected gradient parameters, including: Calculate the input feature matrix covariance matrix Used to describe the feature distribution of this layer in a new task: in, This represents the Transformer layer number, where t is the t-th task; the covariance matrix... It characterizes the feature space structure of the new task in the corresponding layer.

[0054] Based on this, in order to construct a feature space that is resistant to feature drift, singular value decomposition is performed on the covariance matrix: in, It is a principal component orthogonal base. It is a singular value diagonal matrix; singular value decomposition can extract the main direction of the features of this layer and identify the feature components most sensitive to task transfer.

[0055] The principal components are filtered to obtain the orthogonal projection matrix corresponding to each Transformer layer. : in This represents the principal component submatrix composed of the k principal component directions after filtering. The principal component submatrix projects the gradient onto the anti-drift subspace. gradient parameters for each Transformer layer Apply the orthogonal projection matrix The corrected gradient parameters are obtained. That is, the low-rank update quantity: in, Indicates that task t is in the first position. The original gradient values ​​of the Transformer layer.

[0056] In this way, gradient updates are constrained within a feature drift-resistant space, thereby reducing the interference of new task training on historical task features. Combined with a low-rank adapter, DRS not only maintains the model's flexible adaptability to new tasks but also maximizes the retention of knowledge from previous tasks, achieving efficient anti-forgetting capabilities in continuous learning scenarios. The entire process forms a closed loop through layer-by-layer calculation of feature covariance, singular value decomposition, and gradient projection, systematically suppressing feature drift and ensuring model stability across multi-task sequences.

[0057] In its implementation, this application introduces an Updated Calibration Regularizer (UCR) to address the catastrophic forgetting problem caused by interference between parameters of new and old tasks in incremental learning. In continuous learning scenarios, whether using strategies with sample replay or efficient parameter fine-tuning methods without replay, it's inevitable that parameter updates for new tasks will be superimposed on the same shared backbone network, thus disrupting the stability of existing knowledge distribution. The core idea of ​​UCR is to finely control the magnitude and direction of parameter updates without weakening the learning ability of new tasks, ensuring that updates for new tasks do not significantly interfere with the knowledge representation of old tasks.

[0058] like Figure 4 As shown, in the overall framework, UCR is embedded into the model's training process to constrain the interaction behavior of the low-rank adapter across tasks. UCR applies a regularization penalty by calculating the element-wise overlap between the parameters of the old and current tasks, thereby establishing an "update calibration" mechanism at the parameter level to avoid excessive perturbation of the feature distribution formed by previous tasks. Assuming the initial weight matrix of the Vision Transformer module is W, the th... The low-rank adapter parameters for each task are: Then the cumulative weight up to task t-1 can be expressed as: During training the t-th task, the low-rank adapter generates new parameter updates. The area overlapping with the accumulated parameters of the old task is considered to be the part most prone to being forgotten.

[0059] To address this, UCR introduces the following calibration regularization term into the loss function: in, This represents the calibration regularization term for task t. This represents the element-wise Hadamard product. Denotes the Frobenius norm. Indicates task The low-rank update volume.

[0060] The calibration regularization term measures the overlap between the parameters of the old and new tasks at the element level. A higher regularization penalty is imposed when a parameter unit has been fully activated (large value) in the old task and is still strongly modified in the new task; conversely, the update penalty is weaker for parameter dimensions that were previously less important, thus preserving enough plasticity space for the new task.

[0061] During training, the calibration regularization term primarily functions during the backpropagation phase. After completing standard forward inference, the model obtains feature representations jointly generated by the backbone network and the adaptation module. At this point, the output features of each layer serve as crucial data for subsequent gradient calibration, used to construct the task-relevant feature embedding space and calculate the standard classification loss. With auxiliary triplet loss Subsequently, during backpropagation, the calibration regularization term constrains the gradient direction, ensuring that parameter updates avoid high activation dimensions already occupied by older tasks, thus achieving non-intrusive optimization at the gradient level. This parameter-interaction-based calibration mechanism effectively reduces cross-task gradient conflicts and prevents the model from drifting towards representations of older classes when learning new classes.

[0062] Specifically, the loss function L for: in, The standard classification loss for task t is represented by... This represents the auxiliary triplet loss for task t. and The weight balance parameters of the auxiliary triplet loss and the calibration regularization term are controlled separately.

[0063] in, In Continual Learning, the cross-entropy loss of the model after completing the t-th task can be expressed as: Where N represents the number of samples in the current task. Let be the number of categories for the t-th task. It is a sample The tag, and This indicates that after training the model on the t-th task, the parameters are... The loss is the predicted probability for category c. It measures the classification fit of the model after the t-th task and is the most commonly used basic optimization objective in continuous learning.

[0064] In continuous learning, Auxiliary triplet loss (ATL) is used to retain knowledge from old tasks while training on new tasks. Its standard form after training on t tasks is defined as: in , and These represent anchor points, positive samples, and negative samples, respectively. It is a feature extraction model. This represents the hinge loss function. UCR optimizes the "stability-plasticity" tradeoff in continuous learning by introducing explicit element-level perturbation constraints in the parameter space. It does not rely on sample replay mechanisms and boasts advantages such as simple structure and high pluggability, allowing seamless integration into any continuous learning framework based on low-rank adapters. This provides the model with an efficient parameter perturbation suppression approach and a stable knowledge evolution mechanism.

[0065] The auxiliary triplet loss is a loss function used to measure learning and feature embedding optimization. It aims to constrain the structure of the feature space by comparing three sets of samples—anchor samples, positive samples, and negative samples. Its goal is to minimize the distance between the anchor and the positive sample, while maximizing the distance between the anchor and the negative sample, thereby forming a more discriminative feature distribution in the embedding space. This not only enhances the model's generalization ability but also provides a stable feature constraint foundation for continuous learning and multi-task feature sharing.

[0066] According to a second aspect of this application, an image classification method is provided, wherein the cross-domain continuous learning model is trained using the training method described in the first aspect above to obtain a trained cross-domain continuous learning model. The trained cross-domain continuous learning model is used to classify the images to be classified and obtain the classification results.

[0067] This application also provides an image classification method. Optionally, the image classification method uses the training method described above to train the cross-domain continuous learning model to obtain a trained cross-domain continuous learning model; the trained cross-domain continuous learning model is then used to classify the image to be classified to obtain the classification result.

[0068] This application also provides an electronic device, the structure of which is as follows: Figure 5 As shown, the electronic device includes a memory 611, a processor 612, a communication module 613, and an input / output interface 614, etc. Optionally, the memory 611, the processor 612, the communication module 613, and the input / output interface 614 can be connected and communicate with each other through a bus 615.

[0069] The memory 611 is used to store one or more computer programs and to transfer the code of the computer programs to the processor 612; when the one or more computer programs are executed by the processor 612, a training method and / or image classification method for a cross-domain continuous learning model for image classification according to an embodiment of this application is implemented.

[0070] Optionally, the electronic device can be connected to a network via communication module 613 to communicate with other devices, such as terminals or servers, to achieve data interaction. The electronic device can be various forms of digital computers, exemplarily such as desktop computers, servers, workbenches, mainframes, or other types of computers. The electronic device can also be various forms of mobile terminals, exemplarily such as smartphones, tablets, wearable devices (such as helmets, glasses, watches, etc.), and other similar mobile terminals.

[0071] Optionally, the electronic device can connect to required input / output devices, such as a keyboard or display device, via the input / output interface 614. The electronic device itself may have a display device, and other display devices can also be connected externally via the input / output interface 614. Optionally, a storage device, such as a hard disk, can also be connected via the input / output interface 614 to store data from the electronic device, read data from the storage device, or store data from the storage device in the memory 611. It is understood that the input / output interface 614 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 614 can be a component of the electronic device or an external device connected to the electronic device when needed.

[0072] Optionally, the memory 611 may be a volatile memory and / or a non-volatile memory. The volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or a flash memory, etc.

[0073] Optionally, the computer program stored in the memory 611 can be divided into one or more modules, which are stored in the memory 611 and executed by the processor 612 to perform the method provided in this embodiment. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.

[0074] Optionally, the processor 612 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 612 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, and can also be any suitable controller, microcontroller, processor, etc. The processor 612 executes the various methods and processes of this embodiment, exemplarily, such as a training method and / or image classification method for a cross-domain continuous learning model for image classification according to an embodiment of this application.

[0075] Optionally, the bus 615 may include a path for transmitting information. Depending on its function, the bus 615 may be divided into an address bus, a data bus, a control bus, etc.

[0076] In an optional implementation, this application embodiment also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods described in the above method embodiments. Part or all of the computer program can be loaded and / or installed on the memory 611 of an electronic device. When the computer program is executed by the processor 612, it can perform one or more steps of a training method and / or image classification method for a cross-domain continuous learning model for image classification according to the embodiments of this application.

[0077] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc.

[0078] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A training method for a cross-domain continuous learning model for image classification, characterized in that, The cross-domain continuous learning model includes a Vision Transformer module, a low-rank adapter, an adaptive dynamic rank adjustment module, and a classification module; wherein the encoder of the Vision Transformer module includes multiple stacked Transformer layers, each Transformer layer including a sequentially connected multi-head self-attention layer and a feedforward network; a low-rank adapter is inserted into each multi-head self-attention layer; The training method includes: The input image is processed by the Vision Transformer module to obtain an image patch embedding vector sequence. The image patch embedding vector sequence is then position-encoded and input into the encoder. The adaptive dynamic rank adjustment module dynamically adjusts the rank in the low-rank adapter based on the model's performance feedback and outputs the rank value. The low-rank factorization matrices A and B of the low-rank adapter are initialized with the rank values, and only the low-rank factorization matrices A and B are updated during training, while the original weight matrix in the Vision Transformer module remains frozen. The Vision Transformer module performs forward propagation based on the effective weights formed by the original weight matrix and the low-rank decomposition factor matrices A and B, and outputs classification features. The classification features are input into the classification module to obtain the prediction results, and the loss function is calculated and the parameters of the low-rank adapter are updated based on the prediction results.

2. The method according to claim 1, characterized in that, The step of dynamically adjusting the rank in the low-rank adapter based on the model's performance feedback and outputting the rank value includes: Before each task t begins, the initial rank value of task t in the low-rank adapter is initialized according to the preset allocation strategy; During the training process of the model, the slope of the performance trajectory is calculated based on the average accuracy of the model on the seen categories in the most recent tasks. The rank of the task after task t is adjusted based on the slope of the performance trajectory.

3. The method according to claim 2, characterized in that, The slope of the performance trajectory The calculation formula is as follows: in, Indicates in the task The average accuracy of the model on the seen categories is then given, where M represents the M tasks closest to task t.

4. The method according to claim 2, characterized in that, The step of adjusting the rank of task t after adjusting the performance trajectory slope includes: Preset performance trajectory slope threshold Threshold of rank r ; When the slope of the performance trajectory At that time, the rank value of task t+1 after task t in the low-rank adapter is determined according to... ,in This represents the rank value corresponding to task t+1. This represents the rank value corresponding to task t; when At that time, the rank value corresponding to task t+1 after task t is initialized using the preset allocation strategy.

5. The method according to claim 2, characterized in that, The adaptive dynamic rank scheduling module has a pause mechanism: if three consecutive tasks trigger rank adjustment, the rank adjustment of the subsequent two consecutive tasks will be automatically paused so that the rank of the subsequent two consecutive tasks adopts the rank value of the most recent adjustment.

6. The method according to claim 2, characterized in that, The effective weights, which are based on the original weight matrix and the low-rank decomposition factor matrices A and B, are specifically as follows: in, Let W represent the weight matrix corresponding to task t, and let W represent the original weight matrix of the multi-head self-attention layer. Indicates task Low-rank update volume Indicates task The corresponding low-rank factorization matrix B, Indicates task The corresponding low-rank factorization matrix A, This represents the feature dimension of the input image.

7. The method according to any one of claims 1-6, characterized in that, The model also includes a feature drift suppression module, and the training method further includes: The feature drift suppression module collects the input feature matrix of each Transformer layer in the model, calculates the covariance of the input feature matrix, constructs an anti-drift subspace through singular value decomposition, and projects the gradient parameters of the low-rank adapter onto the anti-drift subspace to obtain the corrected gradient parameters. Update the low-rank decomposition factor matrices A and B in the low-rank adapter based on the corrected gradient parameters.

8. The method according to claim 7, characterized in that, The covariance of the input feature matrix is ​​calculated and a drift-resistant subspace is constructed using singular value decomposition. The gradient parameters of the low-rank adapter are projected onto the drift-resistant subspace to obtain the corrected gradient parameters, including: Calculate the input feature matrix covariance matrix Used to describe the feature distribution of this layer in a new task: in, This indicates the number of the Transformer layer, where t is the t-th task; Perform singular value decomposition on the covariance matrix: in, It is a principal component orthogonal base. It is a singular value diagonal matrix; The principal components are filtered to obtain the orthogonal projection matrix corresponding to each Transformer layer. : in This represents the principal component submatrix composed of the k principal component directions after filtering. The principal component submatrix projects the gradient onto the anti-drift subspace. gradient parameters for each Transformer layer Apply the orthogonal projection matrix The corrected gradient parameters are obtained. : in, Indicates that task t is in the first position. The original gradient values ​​of the Transformer layer.

9. The method according to any one of claims 1-6, characterized in that, The loss function L for: in, The standard classification loss represents task t. This represents the auxiliary triplet loss for task t. and The weight balance parameters for the auxiliary triplet loss and the calibration regularization term are controlled separately. The calibration regularization term for task t is calculated using the following formula: in, This represents the element-wise Hadamard product. Denotes the Frobenius norm; Indicates task W represents the original weight matrix of the multi-head self-attention layer. Indicates task Low-rank update volume This represents the low-rank update term for task t.

10. An image classification method, characterized in that, The cross-domain continuous learning model is trained using the training method described in any one of claims 1-9 to obtain a trained cross-domain continuous learning model; The trained cross-domain continuous learning model is used to classify the images to be classified and obtain the classification results.

Images