Image classification system, method and device based on low-rank feature subspace adaptive management

Abstract

The application discloses an image classification system, method and device based on low-rank feature subspace adaptive management, which comprises a pre-trained main neural network and a plurality of low-rank adaptive modules. A low-rank adaptive module is connected to each linear mapping layer of the main neural network. When the system is subjected to online learning, the main network parameters are not updated, and only the parameters of the low-rank adaptive modules are optimized. Gradient projection method is used to optimize the current adaptive subspace parameters, and only the interpretable update components in the subspace are used to adjust the parameters. Whether the adaptive subspace is mismatched is detected based on the gradient energy distribution. If the current subspace is mismatched for N times, the subspace basis matrix and the coefficient parameters of the current stage are solidified. And based on the residual update requirement outside the subspace, a new low-rank adaptive subspace is constructed. The application enables the model to adapt to new image data with a small parameter increment, and significantly reduces the calculation and storage overhead in the online processing process.

Description

Technical Field

[0001] This invention belongs to the field of machine learning and artificial intelligence technology, specifically relating to an image classification system, method and device based on adaptive management of low-rank feature subspace. Background Technology

[0002] Currently, in recent years, with the widespread adoption of smart cameras, industrial vision, and edge intelligence devices, image data is continuously generated in the form of data streams and used for tasks such as target recognition and image classification. This type of continuous image big data typically exhibits obvious objective patterns of change, such as variations in illumination and viewing angle, background disturbances, sensor noise, and gradual or abrupt changes in target shape and texture, causing image feature distribution to drift over time. Under these dynamic conditions, the model needs to continuously extract discriminative features effective for classification from newly arriving image data while maintaining existing "feature-category" relationships to meet real-time and stability requirements.

[0003] Existing image classification update strategies for data streams mostly rely on the following approaches: playback methods maintain old knowledge by storing or generating historical samples, but suffer from problems such as high storage overhead, privacy risks, and insufficient online real-time performance; regularization methods protect existing relationships by constraining update magnitude, but are prone to a decline in discriminative ability under long-term data streams or significant distribution drift; and architecture-scaling methods achieve isolation by adding new structures to mitigate forgetting, but the model size grows over time, making it difficult to deploy on resource-constrained terminals.

[0004] Furthermore, while efficient parameter tuning techniques can reduce overhead by freezing the basic representation model, in the absence of clear stage signals and with continuously changing distributions, the update direction may still interfere with existing correlations, leading to degradation of historical classification capabilities or unstable adaptation. Therefore, there is an urgent need for an image classification method and system that can adaptively identify feature changes and discriminate structural deficiencies in continuous image big data, and achieve low overhead and stable updates under limited resource constraints. Summary of the Invention

[0005] This invention provides an image classification system, method, and device based on adaptive management of low-rank feature subspace to solve the technical problems existing in the prior art.

[0006] The technical solution adopted by this invention to solve the technical problems existing in the prior art is as follows: An image classification system based on low-rank feature subspace adaptive management is disclosed. The system includes a backbone neural network for image classification and several low-rank adaptation modules. The backbone neural network is a pre-trained neural network. A low-rank adaptation module is connected to each linear mapping layer of the backbone neural network to perform online learning. During learning, the parameters of the backbone network are not updated, but only the parameters of each low-rank adaptation module are optimized.

[0007] Furthermore, the backbone neural network comprises several sequentially connected Transformer models, each containing a self-attention mechanism, and a low-rank adaptor module connected to the linear mapping layer of each Transformer model; for the , The weight parameters of the low-rank adaptation modules are expressed as follows: ; In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; This refers to the sequence number of the low-rank adaptation module; For the first The first low-rank adaptation module The first online phase Time step weight parameter matrix; These are the frozen pre-trained weights; For the first The first low-rank adaptation module The first online phase The low-rank increment corresponding to the time step; The low-rank increment is represented in the form of subspace and coefficient decomposition as follows: ; In the formula: For the first The first low-rank adaptation module A fixed column subspace basis matrix for each online stage; For the first The first low-rank adaptation module The row subspace basis matrix is ​​fixed for each online stage; For the first The first low-rank adaptation module The first online phase Trainable coefficient matrix in time step space; in, and All satisfy orthogonal constraints, and in the th case... The online phases are relatively fixed. In the Each online phase will be optimized and updated.

[0008] The present invention also provides an image classification method based on low-rank feature subspace adaptive management, utilizing the above-mentioned image classification system based on low-rank feature subspace adaptive management. This method includes the following steps: Step 1: Construct a backbone neural network for image classification and pre-train the backbone neural network; connect each linear mapping layer of the pre-trained backbone neural network to a low-rank adaptation module to construct an image classification system; the rank adaptation module introduces low-rank parameters into the linear mapping layer to update the structure, and expresses the low-rank parameters as a form consisting of a fixed subspace basis matrix and coefficients within the subspace. Step 2: When tuning the system parameters, first fix a low-dimensional adaptation subspace, and update the parameters only in the current low-rank subspace while freezing the parameters of the pre-trained backbone neural network. The gradient projection method is used to optimize the parameters of the current fitting subspace, and the parameters in the subspace are adjusted only using the interpretable update components within the subspace; Step 3: For new learning requirements, detect whether the adaptation subspace is mismatched based on the gradient energy distribution. By evaluating the distribution of gradients inside and outside the subspace, determine online whether the current adaptation subspace remains valid. Step 4: If a mismatch in the current subspace is detected N times in a row, the subspace basis matrix and coefficient parameters of the current stage are fixed and their subsequent updates are stopped; and a new low-rank fitting subspace is constructed based on the residual update requirements outside the subspace. Step 5: Repeat steps 2 to 4 until the system parameter tuning is complete.

[0009] Further, in step 1, a backbone neural network is constructed by sequentially connecting several Transformer models containing self-attention mechanisms, and a low-rank adaptation module is connected to the linear mapping layer of each Transformer model; the increment of the adaptation linear mapping parameter of any low-rank adaptation module is represented as follows: ; In the formula: To adapt the linear mapping parameter increment matrix; Let be the basis matrix of the column subspace; ; Let be the basis matrix of the row subspace; ; is the trainable coefficient matrix within the subspace; ; To represent the output dimension; For input dimensions; The rank of the low-rank decomposition; in, U and V The orthogonal constraint matrix satisfies the following equation: ; In the formula: It is the identity matrix; Maintain in the current learning phase U and V Fixed, only update the coefficient matrix C .

[0010] Furthermore, in step 2, the method for updating the parameters within the current low-rank subspace includes the following steps: For any adapted low-rank adapting module, its weight parameter matrix is ​​represented as follows: ; In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; For the first The first online phase Time step weight parameter matrix; The pre-trained weight matrix is ​​frozen; For the first The first online phase The low-rank increment matrix corresponding to the time step; The low-rank increment can be further decomposed into the product of the subspace basis matrix and the coefficients within the subspace using the following formula: ; In the formula: For the first The column subspace basis matrix of each online stage; ; For the first The row subspace basis matrix of each online stage; ; For the first The transpose of the row subspace basis matrices of each online stage; For the first The first online phase The updatable coefficient matrix in the time step space; ; To represent the output dimension; For input dimensions; The rank of the low-rank decomposition; in, The following orthogonal constraints must be satisfied: ; In the formula: s is the dimension of the identity matrix; It is an s-dimensional identity matrix; = ; In the Within each online phase, the subspace basis matrix , Keep fixed, only for coefficients within the subspace Update For the first The updatable coefficient matrix within each online stage subspace.

[0011] Furthermore, in step 2, the method for optimizing the current fitting subspace parameters using the gradient projection method includes the following steps: In response to the The first online phase For the data samples or small batches arriving at the time step, calculate the gradient of the loss function with respect to the parameters using the following formula: ; In the formula: For the first The first online phase The gradient matrix of the time step parameters; For the first The first online phase Time step loss; For the weight parameter matrix Find the gradient operator; Because parameter updates are limited by the current subspace { , Projecting the gradient onto the feasible update set spanned by this subspace yields the following gradient components within the subspace: ; In the formula: For the first The first online phase Gradient component matrix in time step subspace; Meanwhile, the residual gradient components outside the subspace are defined as follows: ; In the formula: For the first The first online phase The residual gradient component matrix outside the time step space; The residual gradient matrix is ​​used to characterize update requirements that cannot be expressed in the current subspace; Based on the chain rule, the gradient matrix of the coefficient matrix in the subspace is represented as follows: ; In the formula: For the first The first online phase Time-step loss function for low-rank subspace coefficient matrix The gradient matrix; The coefficients in the subspace are updated according to the following formula: ; In the formula: For the first The first online phase The updatable coefficient matrix in the time step space; This is the learning rate.

[0012] Furthermore, step 3 includes the following method steps: Based on the gradient component matrix in the subspace obtained in step 2 With the subspace external residual gradient component matrix Construct a mismatch index to measure the expressive power of the current subspace, and set... The mismatch index is calculated as follows: : ; in Describing the Frobenius norm, To prevent positive constants with unstable values; To reduce the noise impact of single gradient fluctuations, the mismatch degree is smoothed over time according to the following formula to obtain a smoothed mismatch signal: ; In the formula: This is the smoothed mismatch metric at the current time step t; To step t in the previous time The smoothed mismatch metric of 1; For smoothing coefficients; ; When the smooth mismatch signal Exceeding the threshold within a consecutive preset time window τ When this occurs, it is determined that there is a persistent mismatch between the current low-rank subspace and the data distribution; By evaluating the distribution of gradients inside and outside the subspace, it can be determined online whether the current adaptation subspace can effectively express new learning needs, and achieve adaptive recognition of changes in data distribution without relying on explicit task boundaries or historical data replay.

[0013] Furthermore, in step 4, the method for constructing the new low-rank fit subspace includes the following steps: When a persistent subspace mismatch is detected, the first... The subspace parameters corresponding to each online stage are fixed, that is, their subspace basis matrix and coefficients are frozen; so that they will not participate in parameter updates in subsequent online training, thus serving as stable storage of historical knowledge. Based on the subspace-outside residual gradient obtained in step 2, a new low-rank fitting subspace is constructed as follows: The residual gradient is accumulated over time and then exponentially weighted averaged according to the following formula: ; In the formula: For the first The cumulative residual gradient matrix at each time step; For the first The cumulative residual gradient matrix at each time step; This is the cumulative coefficient; For the cumulative residual gradient Perform low-rank decomposition and extract its principal directions as the basis matrix of the new subspace: ; In the formula: The singular vector index; For the first The singular value matrix of a singular vector; For the first The basis of the singular vector output space; For the first A basis for the singular vector input space; Before selection A new subspace basis matrix is ​​constructed using singular vectors: ; In the formula: This is the matrix for output direction; The matrix represents the input direction; For the first Output basis matrix of time step subspace; For the first The input basis matrix of the time step subspace; For the first Time, step, space; This indicates that a value has been assigned. The coefficients in the subspace of the new stage are initialized to either a zero matrix or a singular value matrix according to the following formula. Align the initial values ​​and switch to the newly constructed subspace to continue the image classification process: or ; In the formula: For the first Core weights in the temporal step space phase; It is a singular value matrix; This indicates that a value has been assigned.

[0014] The present invention also provides an apparatus for an image classification method based on low-rank feature subspace adaptive management, comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program and, when executing the computer program, implement the image classification method steps as described above based on low-rank feature subspace adaptive management.

[0015] The advantages and positive effects of this invention are: (1) By restricting the feature update process of continuous image data to a fixed low-rank feature subspace, the changes in image features always proceed along a stable and interpretable dominant direction, thereby avoiding the problem of the destruction of the existing “feature-category” relationship caused by the disordered change of the feature expression structure in the existing efficient parameter tuning methods. In dynamic scenarios without clear stage identifiers, the degradation of historical discrimination ability is effectively alleviated.

[0016] (2) The present invention constructs a mismatch detection mechanism based on the energy ratio of feature changes inside and outside the subspace. This mechanism directly reflects the objective feature distribution change law in continuous image data. It can realize online recognition of insufficient feature expression ability without relying on explicit task boundary labeling or historical sample playback, and trigger adaptive structural adjustment. It has the technical advantages of being lightweight and real-time.

[0017] (3) While ensuring image classification performance and discrimination stability, this invention only needs to introduce a small number of low-rank feature adaptation components to complete the effective processing of newly arrived image data, significantly reducing the consumption of computing and storage resources in the continuous data processing process. It is suitable for the long-term stable deployment of large-scale pre-trained visual models in edge devices, smart terminals and real-time image recognition systems, and has good engineering practical value. Attached Figure Description

[0018] Figure 1 This is a flowchart of an image classification method based on adaptive management of low-rank feature subspace according to the present invention.

[0019] In the picture: This refers to the iteration time step number; Q stands for Query; V stands for Value; K stands for Key; A represents the attention weights matrix, calculated from Q and K. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] In the description of this invention, the terms "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," and "bottom," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and do not require the invention to be constructed and operated in a specific orientation; therefore, they should not be construed as limitations on the invention. The terms "connected" and "linked" used in this invention should be interpreted broadly. For example, they can refer to a fixed connection or a detachable connection; a direct connection or an indirect connection through intermediate components; or an electrical connection or signal transmission. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0022] Please see Figure 1 An image classification system based on low-rank feature subspace adaptive management is characterized in that the system includes a backbone neural network for image classification and several low-rank adaptation modules. The backbone neural network is a pre-trained neural network, and a low-rank adaptation module is connected to each linear mapping layer of the backbone neural network to perform online learning of the system. During learning, the parameters of the backbone network are not updated, but only the parameters of each low-rank adaptation module are optimized.

[0023] Preferably, the backbone neural network may include several sequentially connected Transformer models, each Transformer model containing a self-attention mechanism, and a low-rank adaptor module connected to the linear mapping layer of each Transformer model; for the , The weight parameters of the low-rank adaptation modules are expressed as follows: ; In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; This refers to the sequence number of the low-rank adaptation module; For the first The first low-rank adaptation module The first online phase Time step weight parameter matrix; These are the frozen pre-trained weights; For the first The first low-rank adaptation module The first online phase The low-rank increment corresponding to the time step.

[0024] The low-rank increment is represented in the form of subspace and coefficient decomposition as follows: ; In the formula: For the first The first low-rank adaptation module A fixed column subspace basis matrix for each online stage; For the first The first low-rank adaptation module The row subspace basis matrix is ​​fixed for each online stage; For the first The first low-rank adaptation module The first online phase Trainable coefficient matrix in time step space; in, and All satisfy orthogonal constraints, and in the th case... The online phases are relatively fixed. In the Each online phase will be optimized and updated.

[0025] The Transformer model is a deep learning architecture based on self-attention mechanisms. It employs an encoder-decoder architecture, consisting of multiple stacked modules. The encoder transforms the input sequence (e.g., a sentence) into a series of contextual representation vectors. Each layer contains two sub-layers: a multi-head self-attention mechanism and a feedforward neural network, followed by residual connections and layer normalization. The decoder, based on the encoder's output and the generated partial target sequence, progressively generates the complete target sequence (e.g., a translation result). Each layer contains three sub-layers: a masked multi-head self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network.

[0026] The present invention also provides an image classification method based on low-rank feature subspace adaptive management, utilizing the above-mentioned image classification system based on low-rank feature subspace adaptive management. This method includes the following steps: Step 1: Construct a backbone neural network for image classification and pre-train the backbone neural network; connect each linear mapping layer of the pre-trained backbone neural network to a low-rank adaptation module to construct the image classification system; the rank adaptation module introduces low-rank parameters into the linear mapping layer to update the structure, and expresses the low-rank parameters as a form consisting of a fixed subspace basis matrix and coefficients within the subspace.

[0027] Step 2: When tuning the system parameters, first fix a low-dimensional adaptation subspace. With the pre-trained backbone neural network parameters frozen, update the parameters only within the current low-rank subspace. Use gradient projection to optimize the parameters of the current adaptation subspace, adjusting the parameters within the subspace only using interpretable update components.

[0028] Step 3: For new learning requirements, detect whether the adaptation subspace is mismatched based on the gradient energy distribution. By evaluating the distribution of gradients inside and outside the subspace, determine online whether the current adaptation subspace remains valid.

[0029] Step 4: If a mismatch in the current subspace is detected N times in a row, the subspace basis matrix and coefficient parameters of the current stage are fixed and their subsequent updates are stopped; and a new low-rank fitting subspace is constructed based on the residual update requirements outside the subspace.

[0030] Step 5: Repeat steps 2 to 4 until the system parameter tuning is complete.

[0031] Preferably, in step 1, a backbone neural network can be constructed by sequentially connecting several Transformer models containing self-attention mechanisms, with a low-rank adaptation module connected to the linear mapping layer of each Transformer model; the increment of the adaptation linear mapping parameter of any low-rank adaptation module is expressed as follows: ; In the formula: To adapt the linear mapping parameter increment matrix; Let be the basis matrix of the column subspace; ; Let be the basis matrix of the row subspace; ; is the trainable coefficient matrix within the subspace; ; To represent the output dimension; For input dimensions; It is the rank of the low-rank decomposition.

[0032] in, U and V The orthogonal constraint matrix satisfies the following equation: ; In the formula: It is the identity matrix; Maintain in the current learning phaseU and V Fixed, only update the coefficient matrix C .

[0033] Preferably, in step 2, the method for updating the parameters within the current low-rank subspace may include the following steps: For any adapted low-rank adapting module, its weight parameter matrix is ​​represented as follows: ; In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; For the first The first online phase Time step weight parameter matrix; The pre-trained weight matrix is ​​frozen; For the first The first online phase The low-rank increment matrix corresponding to the time step.

[0034] The low-rank increment can be further decomposed into the product of the subspace basis matrix and the coefficients within the subspace using the following formula: ; In the formula: For the first The column subspace basis matrix of each online stage; ; For the first The row subspace basis matrix of each online stage; ; For the first The transpose of the row subspace basis matrices of each online stage; For the first The first online phase The updatable coefficient matrix in the time step space; ; To represent the output dimension; For input dimensions; It is the rank of the low-rank decomposition.

[0035] in, The following orthogonal constraints must be satisfied: ; In the formula: s is the dimension of the identity matrix; It is an s-dimensional identity matrix; = ; In the Within each online phase, the subspace basis matrix , Keep fixed, only for coefficients within the subspace Update For the first The updatable coefficient matrix within each online stage subspace.

[0036] Preferably, in step 2, the method for optimizing the current fitting subspace parameters using the gradient projection method may include the following steps: In response to the The first online phase For the data samples or small batches arriving at the time step, the gradient of the loss function with respect to the parameters can be calculated using the following formula: ; In the formula: For the first The first online phase The gradient matrix of the time step parameters; For the first The first online phase Time step loss; For the weight parameter matrix Find the gradient operator.

[0037] Because parameter updates are limited by the current subspace { , Projecting the gradient onto the feasible update set spanned by this subspace yields the following gradient components within the subspace: ; In the formula: For the first The first online phase The gradient component matrix in the time step subspace.

[0038] Meanwhile, the residual gradient components outside the subspace are defined as follows: ; In the formula: For the first The first online phase The residual gradient component matrix outside the time step space.

[0039] The residual gradient matrix is ​​used to characterize update requirements that cannot be expressed in the current subspace; Based on the chain rule, the gradient matrix of the coefficient matrix in the subspace can be represented as follows: ; In the formula: For the first The first online phase Time-step loss function for low-rank subspace coefficient matrix The gradient matrix; The coefficients in the subspace are updated according to the following formula: ; In the formula: For the first The first online phase The updatable coefficient matrix in the time step space; This is the learning rate.

[0040] Preferably, step 3 may include the following method steps: Based on the gradient component matrix in the subspace obtained in step 2 With the subspace external residual gradient component matrix Construct a mismatch index to measure the expressive power of the current subspace, and set... The mismatch index is calculated as follows: : ; in Describing the Frobenius norm, To prevent positive constants with unstable values.

[0041] To reduce the noise impact of single gradient fluctuations, the mismatch degree is smoothed over time according to the following formula to obtain a smoothed mismatch signal: ; In the formula: This is the smoothed mismatch metric at the current time step t; To step t in the previous time The smoothed mismatch metric of 1; For smoothing coefficients; .

[0042] When the smooth mismatch signal Exceeding the threshold within a consecutive preset time window τ When this occurs, it is determined that there is a continuous mismatch between the current low-rank subspace and the data distribution.

[0043] By evaluating the distribution of gradients inside and outside the subspace, it can be determined online whether the current adaptation subspace can effectively express new learning needs, and achieve adaptive recognition of changes in data distribution without relying on explicit task boundaries or historical data replay.

[0044] Preferably, in step 4, the method for constructing a new low-rank fit subspace may include the following steps: When a persistent subspace mismatch is detected, the first... The subspace parameters corresponding to each online stage are fixed by freezing their subspace basis matrix and coefficients; this prevents them from participating in parameter updates during subsequent online training, thus serving as stable storage of historical knowledge.

[0045] Based on the subspace-outside residual gradient obtained in step 2, a new low-rank fitting subspace is constructed as follows: The residual gradient is accumulated over time and then exponentially weighted averaged according to the following formula: ; In the formula: For the first The cumulative residual gradient matrix at each time step; For the first The cumulative residual gradient matrix at each time step; This is the cumulative coefficient.

[0046] For the cumulative residual gradient Perform low-rank decomposition and extract its principal directions as the basis matrix of the new subspace: ; In the formula: The singular vector index; For the first The singular value matrix of a singular vector; For the first The basis of the singular vector output space; For the first A basis for the input space of singular vectors.

[0047] Before selection A new subspace basis matrix is ​​constructed using singular vectors: ; In the formula: This is the matrix for output direction; The matrix represents the input direction; For the first Output basis matrix of time step subspace; For the first The input basis matrix of the time step subspace; For the first Time, step, space; This indicates that a value has been assigned.

[0048] The coefficients in the subspace of the new stage are initialized to either a zero matrix or a singular value matrix according to the following formula. Align the initial values ​​and switch to the newly constructed subspace to continue the image classification process: or ; In the formula: For the first Core weights in the temporal step space phase; It is a singular value matrix; This indicates that a value has been assigned.

[0049] The present invention also provides an apparatus for an image classification method based on low-rank feature subspace adaptive management, comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program and, when executing the computer program, implement the image classification method steps as described above based on low-rank feature subspace adaptive management.

[0050] The workflow and working principle of the present invention will be further described below with reference to a preferred embodiment: like Figure 1As shown, an image classification system based on low-rank feature subspace adaptive management is presented. This system includes a backbone neural network for image classification and several low-rank adaptation modules. The backbone neural network is a pre-trained neural network. A low-rank adaptation module is connected to each linear mapping layer of the backbone neural network for online learning. During learning, the parameters of the backbone network are not updated; only the parameters of each low-rank adaptation module are optimized. The backbone neural network includes several sequentially connected Transformer models. Each Transformer model contains a self-attention mechanism, and a low-rank adaptation module is connected to the linear mapping layer of each Transformer model.

[0051] Image classification systems include visual transformers, which consist of multilayer perceptrons, layer normalization modules, and Transformer models. The Transformer model includes a multi-head attention mechanism.

[0052] An image classification method based on adaptive management of low-rank feature subspace includes the following steps: S1. Based on the continuously arriving data stream samples and downstream task requirements, construct an online learning network structure based on a pre-trained model. The network structure includes a frozen pre-trained backbone network and a parameter efficient tuning module connected to it. The parameter efficient tuning module adopts a low-rank parameter form and is used to adapt the model online without updating the backbone parameters.

[0053] S2. Design a low-rank subspace parameterization and constraint update mechanism. This mechanism explicitly models the efficient parameter tuning process of the model as a combination of a fixed low-dimensional fitting subspace and updatable parameters within the subspace. Under the premise of freezing the backbone parameters of the pre-trained model, parameter updates are only allowed to occur within the current low-rank subspace, thereby limiting the range of parameter changes in online learning and avoiding the implicit destruction of historical knowledge due to unconstrained parameter updates.

[0054] S3. Design a subspace projection update mechanism for stable updating of model parameters during online training. This mechanism projects the current gradient into the adaptation subspace and adjusts the parameters in the subspace using only the interpretable update components within the subspace, thereby achieving "learning in a fixed subspace". This effectively suppresses the implicit rotation of the subspace structure during parameter updates and reduces the risk of catastrophic forgetting.

[0055] S4. Design a subspace mismatch detection mechanism based on gradient energy distribution. This mechanism evaluates the distribution of gradients inside and outside the subspace and determines online whether the current adapted subspace can still effectively express new learning requirements. Thus, it can achieve adaptive recognition of data distribution changes without relying on explicit task boundaries or historical data replay.

[0056] S5. Design a subspace solidification and residual-driven expansion mechanism. When a persistent subspace mismatch is detected, the mechanism solidifies the current fitting subspace to protect the learned historical capabilities and constructs a new low-rank fitting subspace based on the residual update requirements outside the subspace, so that the model can quickly adapt to the new data distribution stage without interfering with the existing knowledge.

[0057] The scheme for model structure construction in step S1 includes: By freezing the backbone parameters of the pre-trained model, trainable parameters are introduced only for the low-rank adaptation module, making the backpropagation computation of the model during online training independent of the number of historical stages, thus satisfying the computational constraints of real-time online learning.

[0058] The operations related to low-rank subspace parameterization in step S2 include: At any stage of online learning a Internally, the linear mapping parameters to be adapted in the model. The parameter updates are restricted to a fixed low-rank subspace, and the parameter expression is as follows: ; In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; For the first The first online phase Time step weight parameter matrix; The pre-trained weight matrix is ​​frozen; For the first The first online phase The low-rank increment matrix corresponding to the time step.

[0059] The low-rank increment can be further decomposed into the product of the subspace basis matrix and the coefficients within the subspace using the following formula: ; In the formula: For the first The column subspace basis matrix of each online stage; ; For the first The row subspace basis matrix of each online stage; ; For the first The transpose of the row subspace basis matrices of each online stage; For the first The first online phase The updatable coefficient matrix in the time step space; ; To represent the output dimension; For input dimensions; It is the rank of the low-rank decomposition.

[0060] in, The following orthogonal constraints must be satisfied: ; In the formula: s is the dimension of the identity matrix; It is an s-dimensional identity matrix; = .

[0061] In the Within each online phase, the subspace basis matrix , Keep fixed, only for coefficients within the subspace Update For the first The updatable coefficient matrix within each online stage subspace.

[0062] During the current online phase, the subspace base , Keep fixed, only for coefficients within the subspace Update the code to explicitly distinguish between "subspace structure" and "learning within a subspace".

[0063] The operations related to the projection update within the subspace in step S3 include: In response to the The first online phase For the data samples or small batches arriving at the time step, calculate the gradient of the loss function with respect to the parameters using the following formula: ; In the formula: For the first The first online phase The gradient matrix of the time step parameters; For the first The first online phase Time step loss; For the weight parameter matrix Find the gradient operator.

[0064] Because parameter updates are limited by the current subspace { , Projecting the gradient onto the feasible update set spanned by this subspace yields the following gradient components within the subspace: ; In the formula: For the first The first online phase The gradient component matrix in the time step subspace.

[0065] Meanwhile, the residual gradient components outside the subspace are defined as follows: ; In the formula: For the first The first online phase The residual gradient component matrix outside the time step space.

[0066] The residual gradient matrix is ​​used to characterize update requirements that cannot be expressed in the current subspace; Based on the chain rule, the gradient matrix of the coefficient matrix in the subspace is represented as follows: ; In the formula: For the first The first online phase Time-step loss function for low-rank subspace coefficient matrix The gradient matrix.

[0067] The coefficients in the subspace are updated according to the following formula: ; In the formula: For the first The first online phase The updatable coefficient matrix in the time step space; This is the learning rate.

[0068] The above operations enable restricted gradient updates within a fixed subspace, avoiding implicit rotations of the subspace basis during parameter updates.

[0069] The subspace mismatch detection operations in step S4 include: Based on the gradient component matrix within the subspace With the subspace external residual gradient component matrix Construct a mismatch index to measure the expressive power of the current subspace, and set... The mismatch index is calculated as follows: : ; in Describing the Frobenius norm, To prevent positive constants with unstable values; To reduce the noise impact of single gradient fluctuations, the mismatch degree is smoothed over time according to the following formula to obtain a smoothed mismatch signal: ; In the formula: This is the smoothed mismatch metric at the current time step t; To step t in the previous time The smoothed mismatch metric of 1; For smoothing coefficients; ; When the smooth mismatch signal Exceeding the threshold within a consecutive preset time window τ When this occurs, it is determined that there is a persistent mismatch between the current low-rank subspace and the data distribution; By evaluating the distribution of gradients inside and outside the subspace, it can be determined online whether the current adaptation subspace can effectively express new learning needs, and achieve adaptive recognition of changes in data distribution without relying on explicit task boundaries or historical data replay.

[0070] Step S5, concerning the solidification and expansion of the subspace, includes: When persistent subspace mismatch is detected, the current stage a The corresponding subspace parameters are solidified, i.e., the subspace basis and coefficients within the subspace are frozen as follows: ; This prevents it from participating in parameter updates during subsequent online training, thus serving as a stable storage of historical knowledge.

[0071] Subsequently, based on the residual gradient outside the subspace obtained in step S3, a new low-rank fitting subspace is constructed. Specifically, the residual gradient is accumulated over time or subjected to an exponentially weighted average: The residual gradient is accumulated over time and then exponentially weighted averaged according to the following formula: ; In the formula: For the first The cumulative residual gradient matrix at each time step; For the first The cumulative residual gradient matrix at each time step; This is the cumulative coefficient; For the cumulative residual gradient Perform low-rank decomposition and extract its principal directions as the basis matrix of the new subspace: ; In the formula: The singular vector index; For the first The singular value matrix of a singular vector; For the first The basis of the singular vector output space; For the first A basis for the singular vector input space; Before selection A new subspace basis matrix is ​​constructed using singular vectors: ; In the formula: This is the matrix for output direction; The matrix represents the input direction; For the first Output basis matrix of time step subspace; For the first The input basis matrix of the time step subspace; For the first Time, step, space; This indicates that a value has been assigned. The coefficients in the subspace of the new stage are initialized to either a zero matrix or a singular value matrix according to the following formula. Align the initial values ​​and switch to the newly constructed subspace to continue the image classification process: or ; In the formula: For the first Core weights in the temporal step space phase; It is a singular value matrix; This indicates that a value has been assigned.

[0072] Switch to the newly constructed subspace to continue the image classification process.

[0073] This invention proposes an adaptive image classification method for continuous image data processing. Based on an adaptive management mechanism of low-rank feature subspaces, this method uses artificial intelligence algorithms to perform feature analysis and classification decisions on continuously arriving image data. Even when image data distribution changes over time and lacks clear stage markers, this invention can stably mine and maintain the natural correlation between image features and category discrimination results, thus preventing the degradation of existing discrimination capabilities as data changes. This method is applicable to continuous image data stream scenarios, achieving controlled adjustment of the image feature representation structure without the need for explicit stage segmentation information or historical sample replay, while maintaining low computational and storage overhead in resource-constrained environments. Its overall processing flow is as follows: Figure 1 As shown, the main steps include: restricted feature update based on low-rank feature subspace, feature representation capability mismatch detection, and adaptive expansion of feature subspace based on residual information.

[0074] A Transformer model incorporating self-attention is selected as the backbone network, and a low-rank adaptor module is introduced into its linear mapping layer. The backbone neural network consists of several sequentially connected Transformer models, each containing a self-attention mechanism, with a low-rank adaptor module connected to the linear mapping layer of each Transformer model; for the , The weight parameters of the low-rank adaptation modules are expressed as follows: ; In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; This refers to the sequence number of the low-rank adaptation module; For the first The first low-rank adaptation module The first online phase Time step weight parameter matrix; These are the frozen pre-trained weights; For the first The first low-rank adaptation module The first online phase The low-rank increment corresponding to the time step.

[0075] The low-rank increment is represented in the form of subspace and coefficient decomposition as follows: ; In the formula: For the first The first low-rank adaptation module A fixed column subspace basis matrix for each online stage; For the first The first low-rank adaptation module The row subspace basis matrix is ​​fixed for each online stage; For the first The first low-rank adaptation module The first online phase Trainable coefficient matrix in time step space; in, and All satisfy orthogonal constraints, and in the th case... The online phases are relatively fixed. In the Each online phase will be optimized and updated.

[0076] Perform projection update within subspace: No. The first low-rank adaptation module The first online phase The time step reaches a sample or a small batch of data Then, the loss function is calculated, and the weight gradient of the corresponding module is obtained: ; In the formula: For the first The first low-rank adaptation module The first online phase The gradient matrix of the time step parameters; For the first The first low-rank adaptation module The first online phase Time step loss; For the first The first low-rank adaptation module The gradient matrix of the online stage weight parameter matrix.

[0077] Projecting this gradient onto the current low-rank subspace yields the achievable update components: ; In the formula: For the first The first low-rank adaptation module The first online phase Gradient component matrix in time step subspace; Simultaneously, calculate the residual gradient outside the subspace: ; In the formula: For the first The first low-rank adaptation module The first online phase The residual gradient component matrix outside the time step space.

[0078] Subsequently, the coefficient parameters in the subspace are updated using only the projected gradient:

[0079] In the formula: For the first The first low-rank adaptation module The first online phase Trainable coefficient matrix in time step space; This is the learning rate.

[0080] This step ensures that the model is in the [missing information]. Each online phase learns only within a fixed subspace, avoiding implicit rotations of the subspace structure.

[0081] Perform subspace mismatch detection: To determine whether the current low-rank subspace can still effectively support online learning needs, a subspace mismatch metric based on gradient energy ratio is constructed. For the ... A low-rank adaptation module, whose subspace alignment is defined as: ; For the first The first low-rank adaptation module The first online phase Time step spatial alignment.

[0082] in ε To prevent numerically unstable constants.

[0083] Accordingly, the subspace mismatch degree is defined as: ; The mismatch degrees of multiple adapted modules are aggregated, and an exponential moving average is used to obtain a stable online criterion: ; In the formula: This is the smoothed mismatch metric at the current time step t; To step t in the previous time The smoothed mismatch metric of 1; For smoothing coefficients; ; The number of low-rank adaptation modules; For the first The first low-rank adaptation module The first online phase Temporal step space mismatch; β This is the smoothing coefficient.

[0084] Execution subspace solidification and residual-driven extension: When the mismatch signal exceeds the preset threshold τ a set number of times consecutively, it is considered that the current subspace cannot fully express the new update requirements. At this point, the basis and coefficient parameters of the current subspace are fixed, and subsequent updates are stopped.

[0085] Subsequently, a new low-rank subspace is constructed based on the residual gradient. The residual gradient is then accumulated over time. ; In the formula: For the first The first low-rank adaptation module The cumulative residual gradient matrix at each time step; For the first The first low-rank adaptation module The cumulative residual gradient matrix at each time step; This is the cumulative coefficient.

[0086] And perform truncated singular value decomposition on it according to the following formula: ; In the formula: For the first The matrix of output directions of each low-rank adaptation module; For the first Singular value matrix of a low-rank adaptation module; For the first The matrix of input directions for each low-rank adaptation module.

[0087] Take before r The singular vectors serve as the basis of the new stage subspace: ; In the formula: For the first The first low-rank adaptation module A fixed column subspace basis matrix for each online stage; For the first The first low-rank adaptation module The row subspace basis matrix is ​​fixed for each online stage; For the first The first low-rank adaptation module Initial matrix of trainable coefficients within each online stage subspace; After the new stage is activated, repeat steps 2 to 4 to achieve continuous online adaptation of the model.

[0088] The aforementioned backbone neural network, low-rank adaptation module, Transformer model, self-attention mechanism, and visual transformers, including multilayer perceptron, layer normalization module, and Transformer model, can all adopt applicable functional modules and algorithms in the existing technology, or adopt functional modules and algorithms in the existing technology and construct and implement them using conventional technical means.

[0089] The embodiments described above are only used to illustrate the technical ideas and features of the present invention. Their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The patent scope of the present invention should not be limited by these embodiments. That is, any equivalent changes or modifications made in accordance with the spirit disclosed in the present invention still fall within the patent scope of the present invention.

Claims

1. An image classification system based on adaptive management of low-rank feature subspace, characterized in that, The system includes a backbone neural network for image classification and several low-rank adaptation modules. The backbone neural network is a pre-trained neural network. A low-rank adaptation module is connected to each linear mapping layer of the backbone neural network to learn the system online. During learning, the parameters of the backbone network are not updated, but only the parameters of each low-rank adaptation module are optimized.

2. The image classification system based on low-rank feature subspace adaptive management according to claim 1, characterized in that, The backbone neural network consists of several sequentially connected Transformer models, each containing a self-attention mechanism, and a low-rank adaptor module connected to the linear mapping layer of each Transformer model; for the , The weight parameters of the low-rank adaptation modules are expressed as follows: ； In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; This refers to the sequence number of the low-rank adaptation module; For the first The first low-rank adaptation module The first online phase Time step weight parameter matrix; These are the frozen pre-trained weights; For the first The first low-rank adaptation module The first online phase The low-rank increment corresponding to the time step; The low-rank increment is represented in the form of subspace and coefficient decomposition as follows: ； In the formula: For the first The first low-rank adaptation module A fixed column subspace basis matrix for each online stage; For the first The first low-rank adaptation module The row subspace basis matrix is ​​fixed for each online stage; For the first The first low-rank adaptation module The first online phase Trainable coefficient matrix in time step space; in, and All satisfy orthogonal constraints, and in the th case... The online phases are relatively fixed. In the Each online phase will be optimized and updated.

3. An image classification method based on low-rank feature subspace adaptive management using the image classification system based on low-rank feature subspace adaptive management as described in claim 1, characterized in that, The method includes the following steps: Step 1: Construct a backbone neural network for image classification and pre-train the backbone neural network; connect each linear mapping layer of the pre-trained backbone neural network to a low-rank adaptation module to construct an image classification system; the rank adaptation module introduces low-rank parameters into the linear mapping layer to update the structure, and expresses the low-rank parameters as a form consisting of a fixed subspace basis matrix and coefficients within the subspace. Step 2: When tuning the system parameters, first fix a low-dimensional adaptation subspace, and update the parameters only in the current low-rank subspace while freezing the parameters of the pre-trained backbone neural network. The gradient projection method is used to optimize the parameters of the current fitting subspace, and the parameters in the subspace are adjusted only using the interpretable update components within the subspace; Step 3: For new learning requirements, detect whether the adaptation subspace is mismatched based on the gradient energy distribution. By evaluating the distribution of gradients inside and outside the subspace, determine online whether the current adaptation subspace remains valid. Step 4: If a mismatch in the current subspace is detected N times in a row, the subspace basis matrix and coefficient parameters of the current stage are fixed and their subsequent updates are stopped; and a new low-rank fitting subspace is constructed based on the residual update requirements outside the subspace. Step 5: Repeat steps 2 to 4 until the system parameter tuning is complete.

4. The image classification method based on low-rank feature subspace adaptive management according to claim 3, characterized in that, In step 1, a backbone neural network is constructed by sequentially connecting several Transformer models that include self-attention mechanisms. A low-rank adaptation module is connected to the linear mapping layer of each Transformer model. The increment of the adaptation linear mapping parameters of any low-rank adaptation module is represented as follows: ； In the formula: To adapt the linear mapping parameter increment matrix; Let be the basis matrix of the column subspace; ; Let be the basis matrix of the row subspace; ; is the trainable coefficient matrix within the subspace; ; To represent the output dimension; For input dimensions; The rank of the low-rank decomposition; in, U and V The orthogonal constraint matrix satisfies the following equation: ； In the formula: It is the identity matrix; Maintain in the current learning phase U and V Fixed, only update the coefficient matrix C .

5. The image classification method based on low-rank feature subspace adaptive management according to claim 3, characterized in that, Step 2, the method for updating parameters within the current low-rank subspace includes the following steps: For any adapted low-rank adapting module, its weight parameter matrix is ​​represented as follows: ； In the formula: This refers to the iteration time step number; This refers to the online stage sequence number; For the first The first online phase Time step weight parameter matrix; The pre-trained weight matrix is ​​frozen; For the first The first online phase The low-rank increment matrix corresponding to the time step; The low-rank increment can be further decomposed into the product of the subspace basis matrix and the coefficients within the subspace using the following formula: ； In the formula: For the first The column subspace basis matrix of each online stage; ; For the first The row subspace basis matrix of each online stage; ; For the first The transpose of the row subspace basis matrices of each online stage; For the first The first online phase The updatable coefficient matrix in the time step space; ; To represent the output dimension; For input dimensions; The rank of the low-rank decomposition; in, The following orthogonal constraints must be satisfied: ； In the formula: s is the dimension of the identity matrix; It is an s-dimensional identity matrix; = ; In the Within each online phase, the subspace basis matrix , Keep fixed, only for coefficients within the subspace Update For the first The updatable coefficient matrix within each online stage subspace.

6. The image classification method based on low-rank feature subspace adaptive management according to claim 5, characterized in that, Step 2, the method for optimizing the current fitting subspace parameters using the gradient projection method includes the following steps: In response to the The first online phase For the data samples or small batches arriving at the time step, calculate the gradient of the loss function with respect to the parameters using the following formula: ； In the formula: For the first The first online phase The gradient matrix of the time step parameters; For the first The first online phase Time step loss; For the weight parameter matrix Find the gradient operator; Because parameter updates are limited by the current subspace { , Projecting the gradient onto the feasible update set spanned by this subspace yields the following gradient components within the subspace: ； In the formula: For the first The first online phase Gradient component matrix in time step subspace; Meanwhile, the residual gradient components outside the subspace are defined as follows: ； In the formula: For the first The first online phase The residual gradient component matrix outside the time step space; The residual gradient matrix is ​​used to characterize update requirements that cannot be expressed in the current subspace; Based on the chain rule, the gradient matrix of the coefficient matrix in the subspace is represented as follows: ； In the formula: For the first The first online phase Time-step loss function for low-rank subspace coefficient matrix The gradient matrix; The coefficients in the subspace are updated according to the following formula: ； In the formula: For the first The first online phase The updatable coefficient matrix in the time step space; This is the learning rate.

7. The image classification method based on low-rank feature subspace adaptive management according to claim 6, characterized in that, Step 3 includes the following steps: Based on the gradient component matrix in the subspace obtained in step 2 With the subspace external residual gradient component matrix Construct a mismatch index to measure the expressive power of the current subspace, and set... The mismatch index is calculated as follows: : ； in Describing the Frobenius norm, To prevent positive constants with unstable values; To reduce the noise impact of single gradient fluctuations, the mismatch degree is smoothed over time according to the following formula to obtain a smoothed mismatch signal: ； In the formula: This is the smoothed mismatch metric at the current time step t; To step t in the previous time The smoothed mismatch metric of 1; For smoothing coefficients; ; When the smooth mismatch signal Exceeding the threshold within a consecutive preset time window τ When this occurs, it is determined that there is a persistent mismatch between the current low-rank subspace and the data distribution; By evaluating the distribution of gradients inside and outside the subspace, it can be determined online whether the current adaptation subspace can effectively express new learning needs, and achieve adaptive recognition of changes in data distribution without relying on explicit task boundaries or historical data replay.

8. The image classification method based on low-rank feature subspace adaptive management according to claim 6, characterized in that, Step 4 involves constructing a new low-rank fit subspace, which includes the following steps: When a persistent subspace mismatch is detected, the first... The subspace parameters corresponding to each online stage are fixed, that is, their subspace basis matrix and coefficients are frozen; so that they will not participate in parameter updates in subsequent online training, thus serving as stable storage of historical knowledge. Based on the subspace-outside residual gradient obtained in step 2, a new low-rank fitting subspace is constructed as follows: The residual gradient is accumulated over time and then exponentially weighted averaged according to the following formula: ； In the formula: For the first The cumulative residual gradient matrix at each time step; For the first The cumulative residual gradient matrix at each time step; This is the cumulative coefficient; For the cumulative residual gradient Perform low-rank decomposition and extract its principal directions as the basis matrix of the new subspace: ； In the formula: The singular vector index; For the first The singular value matrix of a singular vector; For the first The basis of the singular vector output space; For the first A basis for the singular vector input space; Before selection A new subspace basis matrix is ​​constructed using singular vectors: ； In the formula: This is the matrix for output direction; The matrix represents the input direction; For the first Output basis matrix of time step subspace; For the first The input basis matrix of the time step subspace; For the first Time, step, space; This indicates that a value has been assigned. The coefficients in the subspace of the new stage are initialized to either a zero matrix or a singular value matrix according to the following formula. Align the initial values ​​and switch to the newly constructed subspace to continue the image classification process: or ; In the formula: For the first Core weights in the temporal step space phase; It is a singular value matrix; This indicates that a value has been assigned.

9. An image classification method based on low-rank feature subspace adaptive management, comprising a memory and a processor, characterized in that, The memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the steps of the image classification method based on low-rank feature subspace adaptive management as described in any one of claims 1 to 8.

Images