A domain incremental image target detection method and system based on orthogonal knowledge refreshing

Abstract

The application discloses a domain incremental image target detection method and system based on orthogonal knowledge refreshing, relates to the technical field of target detection, and adds a current domain exclusive low-rank adaptive branch to a feature extractor linear layer after a new domain image input model, freezes a history branch, relies on linear additivity to fuse each branch weight to complete image forward reasoning. The model loaded with the history branch weight is used to extract a current image feature matrix, a main singular vector is selected through singular value decomposition to construct a history knowledge subspace. During training, a detection loss gradient is calculated according to image labeling, a projection component of the gradient falling in the history subspace is removed, and the current domain low-rank parameter is updated by using an orthogonal gradient. Meanwhile, a global semantic topology is constructed by generating a category prototype relying on a basic domain image, a topology consistency loss is introduced to be jointly optimized with the detection loss, and the same-class target semantics alignment across domains is constrained. The application alleviates the catastrophic forgetting and semantic fragmentation of domain incremental learning, and improves the generalization performance of multi-domain image target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a domain incremental image target detection method and system based on orthogonal knowledge refresh. Background Technology

[0002] Domain incremental object detection is an important research direction in continuous learning and object detection. Its goal is to enable the model to progressively adapt to multiple domains from different data distributions or visual styles, such as natural scenes, artistic scenes, and scenes under different weather or image degradation conditions, while keeping the set of object categories constant. Simultaneously, it improves its adaptability to new domains while maintaining detection performance in already learned domains. Existing object detection models typically consist of a feature extraction network and a detection prediction head. The former extracts visual features from the image, while the latter performs object classification and bounding box regression based on these features. With the development of visual Transformers and pre-trained models, detection frameworks based on pre-trained models have gradually become mainstream. These frameworks learn general representations on large-scale data and then fine-tune them for the task.

[0003] Existing methods based on full fine-tuning or shared parameter fine-tuning directly alter the model's shared feature space when learning new domain data, disrupting the target response, feature statistics, and class discrimination structures already formed in the historical domain, leading to catastrophic forgetting. While existing efficient parameter fine-tuning methods such as cue learning, bias adjustment, and domain-specific parameter expansion can reduce the number of training parameters, their adaptability to complex domain shifts remains limited. Implicit coupling may still occur between different domain branches in the shared feature space, and gradient updates in the new domain can easily interfere with historical domain knowledge, resulting in performance degradation. Some existing methods require explicitly determining the domain to which the test sample belongs and selecting the corresponding model branch or parameter module based on the domain identifier, relying on additional domain selection mechanisms, making them susceptible to domain identification errors and increasing inference complexity. When different domains learn independent parameters, although direct parameter overlay can be reduced, semantic fragmentation is likely to occur, i.e., feature representations of the same category gradually disperse in different domains, leading to inconsistent cross-domain semantic structures and reducing the model's generalization ability in multi-domain environments. Furthermore, some methods rely on historical sample replay, which presents storage costs and privacy constraints in practical applications.

[0004] In the domain incremental object detection scenario, the model often cannot access historical data or can only use very little historical information. Therefore, it is necessary to deal with the distribution differences caused by visual style, environmental changes or image quality degradation during the continuous learning process. How to effectively utilize existing knowledge and improve the adaptability to new domains without relying on historical data has become a key problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a domain-incremental image target detection method based on orthogonal knowledge refresh. By constructing a domain-specific low-rank subspace (LoRA) under continuous domain change conditions, the gradient of the new domain is orthogonally refreshed while maintaining cross-domain semantic topological consistency. Without saving historical samples, relying on explicit domain selection, or requiring multiple redundant forward inferences, this method achieves effective adaptation to the new domain and stable maintenance of historical domain knowledge. This avoids the new domain learning from overwriting historical knowledge and alleviates catastrophic forgetting. At the same time, the low-rank subspace expansion reduces inter-domain parameter interference, the gradient orthogonal refresh guides the update direction to avoid the historical knowledge subspace, and the topological consistency constraint alleviates the semantic fragmentation problem. Ultimately, this improves the stability, plasticity, and cross-domain generalization ability of the target detection model during multi-domain continuous learning.

[0006] Another objective of this invention is to propose a domain incremental image target detection system based on orthogonal knowledge refresh.

[0007] To achieve the above objectives, a first aspect of the present invention proposes a domain incremental image target detection method based on orthogonal knowledge refresh, comprising: S1. When the new domain image data arrives, the input image in the domain is fed into the pre-trained target detection model. In the linear transformation layer of the model feature extractor, an independent low-rank adaptation branch is introduced for the current image domain, and the low-rank adaptation branches corresponding to all historical image domains are frozen. The domain branches are fused into a unified weight in a linearly additive manner, and forward inference is performed on the input image. S2, based on a model that integrates low-rank branches of the historical image domain, feeds the current domain input image into the model, extracts the image feature matrix output by each linear layer of the feature extractor and performs singular value decomposition, extracts the first few singular vectors that retain the main energy of the image features, and constructs the historical knowledge subspace; S3, when training the low-rank adaptation parameters of the current image domain, calculate the original gradient of the target detection loss with respect to the low-rank adaptation parameters by combining the detection annotations corresponding to the input image, remove the components of the original gradient that fall into the historical knowledge subspace, retain only the gradient components that are orthogonal to the historical knowledge subspace, and use the orthogonal gradient to update the low-rank adaptation parameters of the current domain. S4. Based on the input image of the base domain, extract the image feature prototypes of each category to construct a global semantic topology. Input the current domain image and extract the corresponding category feature prototypes. Calculate the topological consistency loss between the feature prototypes of each category in the current domain and the global semantic topology. Then, jointly optimize the topological consistency loss with the object detection loss to constrain the category prototypes of the same type of object in different image domains to align with the semantic structure corresponding to the base domain.

[0008] In one embodiment of the present invention, introducing an independent low-rank adaptation branch for the current image domain includes: At each linear transformation layer of the feature extractor, a low-rank fitting branch consisting of a dimensionality reduction matrix and an increased dimensionality matrix is ​​constructed for the current image domain; where... The dimensionality reduction matrix is ​​used to map the hidden state output by the linear transformation layer of the input image to a low-dimensional feature space, and the dimensionality increase matrix is ​​used to restore and map the low-dimensional space features back to the original feature dimension space.

[0009] In one embodiment of the present invention, the step of fusing the domain branches into a unified weight in a linearly additive manner and performing forward inference on the input image includes: superimposing the weight of the current image domain low-rank adaptation branch with the weight matrix after fusing all historical image domain low-rank adaptation branches to obtain a new unified weight matrix, and using the unified weight matrix to complete the forward calculation of the input image.

[0010] In one embodiment of the present invention, the singular value decomposition of the feature matrices output by each linear layer of the feature extractor for the current domain training samples, based on a model that incorporates low-rank branches from the historical domain, includes: The training samples of the current domain input image are input into the model parameterized by the low-rank branch weights of the fused historical image domain. The hierarchical image feature matrix output by each linear layer of the feature extractor is obtained, and singular value decomposition is performed on the hierarchical image feature matrix to obtain the orthogonal matrix and the singular value matrix.

[0011] In one embodiment of the present invention, removing the components of the original gradient that fall into the historical knowledge subspace and retaining only the gradient components orthogonal to the historical knowledge subspace includes: calculating the projection component of the original gradient on the historical knowledge subspace and subtracting the projection component from the original gradient to obtain the orthogonal gradient.

[0012] In one embodiment of the present invention, the construction of a global semantic topology based on the feature prototypes of each category in the base domain includes: for each category in the base domain, calculating a weighted average feature as the semantic prototype of the category based on the features of all target instances of that category and their classification confidence, and forming a global semantic topology set by combining the semantic prototypes of all categories.

[0013] In one embodiment of the present invention, calculating the topological consistency loss between the feature prototypes of each category in the current domain and the global semantic topology includes: Calculate the semantic matching weights between the feature prototypes of each category in the current domain and the semantic prototypes of each category in the base domain, and construct the topological consistency loss based on the semantic matching weights and cosine similarity.

[0014] In one embodiment of the present invention, the joint optimization of the topology consistency loss and the target detection loss includes: weighted summing of the target detection loss and the topology consistency loss to obtain a joint loss function, and calculating the original gradient of the low-rank adaptation parameter of the current domain based on the joint loss function.

[0015] In one embodiment of the present invention, the method further includes: when the first domain arrives, training a pre-trained object detection model using a standard object detection loss to obtain a base domain model, and freezing most of the parameters of the base domain model.

[0016] To achieve the above objectives, a second aspect of the present invention proposes a domain incremental image target detection system based on orthogonal knowledge updating, comprising: The low-rank subspace expansion fusion inference module is used to feed the input image in the domain into the pre-trained target detection model when new domain image data arrives. In the linear transformation layer of the model feature extractor, an independent low-rank adaptation branch is introduced for the current image domain, and the low-rank adaptation branches corresponding to all historical image domains are frozen. The domain branches are fused into a unified weight in a linearly additive manner to perform forward inference on the input image. The historical knowledge subspace construction module is used to construct the historical knowledge subspace based on a model that integrates the low-rank branch of the historical image domain. The current domain input image is fed into the model, the image feature matrix output by each linear layer of the feature extractor is extracted and singular value decomposition is performed, and the first few singular vectors that retain the main energy of the image features are extracted. The gradient orthogonal refresh module is used to calculate the original gradient of the target detection loss with respect to the low-rank adaptation parameters when training the low-rank adaptation parameters of the current image domain, in combination with the detection annotations corresponding to the input image, remove the components of the original gradient that fall into the historical knowledge subspace, retain only the gradient components that are orthogonal to the historical knowledge subspace, and use the orthogonal gradient to update the low-rank adaptation parameters of the current domain. The topology-aware consistency constraint module is used to extract image feature prototypes of each category based on the input image of the base domain to construct a global semantic topology. It takes the current domain image as input and extracts the corresponding category feature prototypes, calculates the topology consistency loss between the feature prototypes of each category in the current domain and the global semantic topology, and jointly optimizes the topology consistency loss with the object detection loss to constrain the category prototypes of the same type of object in different image domains to align with the semantic structure corresponding to the base domain.

[0017] The domain incremental image target detection method and system based on orthogonal knowledge refresh according to the embodiments of the present invention can effectively alleviate catastrophic forgetting and maintain cross-domain semantic consistency without relying on historical samples and explicit domain selection, and significantly improve the accuracy and robustness of domain incremental target detection.

[0018] Compared with existing technologies, this invention expands the low-rank subspace to allocate relatively independent knowledge representation spaces for different domains, and utilizes the linear additivity of low-rank branches to achieve unified model fusion, thereby maintaining high inference efficiency without explicit domain judgment and multiple forward inferences. At the same time, the gradient orthogonal refresh mechanism restricts the update of new domains to the orthogonal direction of the historical knowledge subspace, effectively reducing interference with historical knowledge and mitigating catastrophic forgetting. Combined with topology-aware consistency constraints, it maintains the consistency of cross-domain category semantic structure and avoids semantic fragmentation. In addition, this invention does not require the storage of historical samples, reducing storage and privacy costs, and exhibits good adaptability and robustness in various domain offset scenarios.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart of a domain incremental image target detection method based on orthogonal knowledge refresh provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a domain incremental image target detection framework based on orthogonal knowledge refresh provided in an embodiment of the present invention; Figure 3 The following are performance analysis results diagrams for each component module provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of the target detection results provided in an embodiment of the present invention; Figure 5 This is a structural diagram of a domain incremental image target detection system based on orthogonal knowledge refresh, provided in an embodiment of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] The following describes, with reference to the accompanying drawings, a domain incremental image target detection method and system based on orthogonal knowledge refresh according to an embodiment of the present invention.

[0024] This embodiment provides a domain incremental image target detection method based on orthogonal knowledge updating. For example... Figure 1 As shown, it includes: S1. When the new domain image data arrives, the input image in the domain is fed into the pre-trained target detection model. In the linear transformation layer of the model feature extractor, an independent low-rank adaptation branch is introduced for the current image domain, and the low-rank adaptation branches corresponding to all historical image domains are frozen. The domain branches are fused into a unified weight in a linearly additive manner, and forward inference is performed on the input image. When a new domain arrives, in the feature extractor of the pre-trained object detection model, an independent low-rank adaptation parameter set is constructed for the current domain for its linear transformation layers. This parameter set is used to learn knowledge representations specific to the current domain, and its dimension is much lower than that of the original weight matrix, thus introducing new domain adaptation capabilities in a parameter-efficient manner. Simultaneously, the low-rank adaptation parameter sets corresponding to all historical domains are frozen, meaning their parameters are no longer updated in subsequent training to maintain the learned historical domain knowledge. Through the linear additivity of the low-rank adaptation branches, the low-rank adaptation parameters of the current domain and all historical domains can be linearly fused with the original weight matrix to form a unified equivalent weight matrix. This allows the model to obtain an output that integrates knowledge from all domains with only one forward computation during the inference phase, without needing to perform branch selection based on the domain identifier of the input sample or multiple forward propagations. As one implementation method, for any linear layer in the feature extractor, its original weight matrix is... The input is hidden. When it has been processed When there are multiple domains, the forward computation is represented as follows: ,in and The first The expression for the dimensionality-reduced and dimensionality-increasing low-rank matrices corresponding to each domain can be equivalent to: That is, all low-rank branches are merged with the original weights into a unified weight. .

[0025] This step achieves parameter-level isolation of knowledge representation between domains by constructing an independent low-rank adaptation branch for each new domain and freezing historical branches, effectively reducing the risk of direct parameter overwriting of historical domain knowledge by new domain learning. At the same time, by utilizing the linear additivity of low-rank branches, the parameters of multiple domains are fused into unified weights for inference, avoiding the computational overhead and recognition errors caused by explicit domain identification and multiple forward propagations. While maintaining high inference efficiency, it provides a stable parameter foundation for subsequent gradient orthogonal refresh and topology consistency constraints.

[0026] S2, based on a model that integrates low-rank branches of the historical image domain, feeds the current domain input image into the model, extracts the image feature matrix output by each linear layer of the feature extractor and performs singular value decomposition, extracts the first few singular vectors that retain the main energy of the image features, and constructs the historical knowledge subspace.

[0027] Specifically, in the domain incremental learning process, to effectively represent the main distribution direction of historical domain knowledge in the model feature space, a historical knowledge subspace needs to be constructed based on the model that has fused the low-rank branches of the historical domain. Specifically, the training samples of the current domain are input into a feature extractor whose weights are parameterized by fusion of all low-rank branches of the historical domain, obtaining the feature matrix generated by each linear layer under the corresponding input. This feature matrix contains the response pattern of historical domain knowledge on the current data, reflecting the discriminative feature structure established by the historical domain. By performing singular value decomposition on this feature matrix, it can be decomposed into a product of singular vectors and singular values. The singular vectors span a set of orthogonal bases in the feature space, while the magnitude of the singular values ​​corresponds to the information energy carried by each base direction. To obtain a compact and representative historical knowledge subspace, the top few singular vectors that can retain the main energy of the feature matrix need to be selected from the decomposition results. That is, by setting an energy retention threshold, the minimum number of singular vectors is selected so that the low-rank approximation matrix formed by these singular vectors and their corresponding singular values ​​satisfies the energy proportion requirement in the Frobenius norm sense. These selected singular vectors form a set of orthogonal bases for the historical knowledge subspace, thereby confining the main characteristic directions of historical knowledge within this subspace.

[0028] As one implementation method, the feature matrix can be decomposed into an orthogonal matrix and a singular value matrix. The number of singular vectors to be retained is determined based on the energy retention threshold. Then, the first few column vectors in the orthogonal matrix are used as the basis vectors of the historical knowledge subspace to construct the subspace.

[0029] By constructing a historical knowledge subspace, we can effectively capture the main distribution direction of historical domain knowledge in the feature space in a low-dimensional and compact manner, providing an accurate reference benchmark for the constraint of subsequent gradient update direction, thereby achieving effective representation and protection of historical knowledge structure without the need to save historical samples.

[0030] S3, when training the low-rank adaptation parameters of the current image domain, calculate the original gradient of the target detection loss with respect to the low-rank adaptation parameters by combining the detection annotations corresponding to the input image, remove the components of the original gradient that fall into the historical knowledge subspace, retain only the gradient components that are orthogonal to the historical knowledge subspace, and use the orthogonal gradient to update the low-rank adaptation parameters of the current domain.

[0031] Specifically, when training the low-rank fitting parameters for the current domain, this method constrains the parameter update direction to reduce the interference of new domain learning on historical domain knowledge. Specifically, firstly, the original gradient of the low-rank fitting parameters for the current domain is calculated based on the object detection loss. This original gradient reflects the optimization direction of the current domain data for parameter updates. Then, using a pre-constructed historical knowledge subspace, components of the original gradient falling into this subspace are removed, retaining only gradient components orthogonal to the historical knowledge subspace, thus obtaining an orthogonal gradient. The update direction indicated by this orthogonal gradient is orthogonal to the historical knowledge subspace, that is, perpendicular to the main feature directions of historical domain knowledge, ensuring that the parameter updates for the current domain avoid the parameter space occupied by historical knowledge as much as possible. Finally, this orthogonal gradient is used to update the low-rank fitting parameters for the current domain, thereby absorbing new domain knowledge while minimizing the destruction of the feature representations already formed in the historical domain.

[0032] As one implementation method, the above process can be achieved by projecting the original gradient onto the orthogonal complement direction of the historical knowledge subspace. For example, a projection matrix can be constructed using the basis vectors of the historical knowledge subspace, and the orthogonal gradient can be obtained by subtracting its projection component on the historical knowledge subspace from the original gradient. ,in Let be the basis vector matrix of the historical knowledge subspace.

[0033] Through this gradient orthogonal refresh mechanism, the parameter updates of the new domain are guided to a direction orthogonal to the historical knowledge subspace, effectively avoiding the intrusion of the new domain gradient into the historical domain knowledge subspace. This significantly reduces inter-domain parameter interference, alleviates catastrophic forgetting, and improves the stability and effectiveness of new domain learning while maintaining the model's performance in detecting historical domains.

[0034] S4. Based on the input image of the base domain, extract the image feature prototypes of each category to construct a global semantic topology. Input the current domain image and extract the corresponding category feature prototypes. Calculate the topological consistency loss between the feature prototypes of each category in the current domain and the global semantic topology. Then, jointly optimize the topological consistency loss with the object detection loss to constrain the category prototypes of the same type of object in different image domains to align with the semantic structure corresponding to the base domain.

[0035] Specifically, in the domain incremental object detection process, to alleviate the problem of inconsistent semantic representations of the same category in different domains due to independent learning in different domains, this method introduces a cross-domain semantic alignment mechanism based on a global semantic topology. Specifically, after training in the base domain, a global semantic topology is constructed based on the feature prototypes of each category in the base domain. This topology reflects the semantic structural relationships between categories in the base domain and serves as a semantic reference benchmark for all subsequent domain learning. When a new domain arrives, the feature prototypes of each category in the current domain are calculated, and the semantic consistency between the current domain category prototypes and the global semantic topology is measured, thereby constructing a topology consistency loss. This topology consistency loss measures the deviation of the current domain category prototypes from the base domain semantic topology. By minimizing this loss, the feature representations of each category in the current domain can be guided to align with the semantic structure of the corresponding category in the base domain, thus maintaining the consistency of cross-domain category semantic structures. During joint optimization, this topology consistency loss and the standard object detection loss are used together as training objectives, enabling the model to maintain cross-domain stability of category semantics while adapting to the data distribution of the new domain.

[0036] As one implementation method, in the base domain, a category prototype can be obtained by weighting the features of each type of target instance and its classification confidence to form a basic semantic topology set; in the new domain, the current domain category prototype is calculated in the same way, and the topology consistency loss is constructed by calculating the matching weight and cosine similarity between the current domain prototype and the base domain prototype, and then jointly optimized with the target detection loss in a weighted manner.

[0037] By constructing a global semantic topology and constraining the current domain category prototype to align with it, this step can effectively alleviate the semantic fragmentation problem caused by independent parameter learning between different domains, enhance the consistency of feature representation of the same category in different domains, and thus improve the model's cross-domain generalization ability and detection stability in a multi-domain environment.

[0038] Example 2 Figure 2 This is a logic diagram of a domain incremental image target detection method based on orthogonal knowledge refresh according to the present invention, as shown below. Figure 2 The proposed method for domain incremental image object detection based on orthogonal knowledge refresh (OKR) is illustrated in this invention. This method is based on a pre-trained object detection model and includes an image input module, a feature extraction module, a low-rank subspace expansion module, a gradient orthogonal refresh module, a topology-aware consistency module, a detection prediction module, and a parameter update module. This method achieves continuous domain adaptation under conditions without sample replay by constructing independent low-rank adaptation branches for different domains and constraining the gradient update direction to be orthogonal to the historical knowledge subspace during the training process of the new domain.

[0039] For ease of description, the domain incremental target detection task includes: Each of the following domains arrives sequentially, and each domain corresponds to a training dataset, denoted as: Among them, the first The dataset for each domain is represented as follows: .in, Indicates the first Input images in each domain, This represents the corresponding object detection annotation, which includes the object category and the object bounding box. The set of object categories remains consistent across domains, but differences exist due to image style, weather, scene, or visual degradation type. In the... During each training phase, the model can only access data from the current domain. Historical domain data cannot be accessed. to .

[0040] Suppose the object detection model consists of a feature extractor and detection prediction head Composition, in which Indicates the feature extractor parameters. This represents the detection prediction head parameters. Input image. Visual features are obtained through feature extractor Then, the detection and prediction head outputs the target category prediction and bounding box prediction.

[0041] Based on the above description, the specific process of this embodiment of the invention may include the following steps: S101, Basic Domain Model Training Steps: In the first domain Upon arrival, the pre-trained object detection model is used as the initial model. The model's feature extractor is initialized with pre-trained weights, and the detection prediction head is used to output the object classification result and the object bounding box. The first domain training image is input into the model, and the model is trained using the standard object detection loss. The object detection loss includes classification loss and bounding box regression loss, which can be expressed as: .in, Indicates the target classification loss. This represents the bounding box regression loss. After training, a base domain model is obtained. Then, most of the parameters of the base model are frozen as a stable foundation for subsequent incremental domain learning.

[0042] S102, Low-rank subspace expansion step: When the... When each domain arrives, among them This invention introduces a new low-rank adaptation branch at the linear transformation layer in the feature extractor. For any linear layer, let its original weight matrix be... The input is hidden. For the first Each domain corresponds to a low-rank fitting branch consisting of two low-rank matrices. and Composition, in which Used for dimensionality reduction mapping Used for dimensional restoration.

[0043] In the During the training phase of each domain, the low-rank adaptation branch corresponding to the historical domain { , , ..., , Keep the page frozen, and only add low-rank adaptation branches corresponding to the new domains. and Trainable. The forward computation of a linear layer is represented as: It can also be written as: .in, This represents the weight matrix after incorporating the low-rank adapted branches from the historical domain. This represents the weight matrix after further incorporating the low-rank adaptation branch of the current domain.

[0044] In this way, the present invention constructs an independent low-rank subspace for each new domain. Historical domain knowledge is carried by frozen historical low-rank branches, and new domain knowledge is carried by newly added low-rank branches. Since the low-rank adaptation branches have linear additivity, the branches of each domain can be directly merged into a unified weight for forward inference. Therefore, it is not necessary to select a specific domain branch based on the input sample, nor is it necessary to perform multiple forward propagations.

[0045] S103, Approximation Steps for the Historical Knowledge Subspace: Simply freezing the low-rank branches of the historical domain is insufficient to completely avoid knowledge interference. Since all low-rank branches will eventually merge into the same model weights, the update of the current domain parameters may still affect the feature space already formed in the historical domain. Therefore, this invention further constructs a historical knowledge subspace and constrains the gradient update direction of the new domain to be orthogonal to the historical knowledge subspace.

[0046] Before training in the t-th domain, the historical fusion weights are first obtained based on the fused historical low-rank branches. Then, the current domain training samples are input into the system. The parameterized model yields the feature matrix of the corresponding linear layer. : .in, This represents the hierarchical feature matrix formed by the current data under the influence of historical knowledge. Subsequently, [the following is done / then...] Perform singular value decomposition:

[0047] in and It is an orthogonal matrix. Let be a singular value matrix. To obtain a compact historical knowledge subspace, this invention selects the elements that preserve the main energy of the characteristic matrix. There are singular vectors. Specifically, the smallest one is selected. , so that:

[0048] in Denotes the Frobenius norm. Indicates from the previous A low-rank approximate matrix consisting of singular values ​​and their corresponding singular vectors. The energy retention threshold.

[0049] Then The former Using the column vectors as basis vectors for the historical knowledge subspace, we obtain the historical knowledge subspace: .in, This represents the main feature subspace of historical domain knowledge in the current model layer.

[0050] S104, Gradient Orthogonal Refresh Step: During the training process in the t-th domain, only the newly added low-rank adaptation parameters are updated. and Suppose that the current target detection loss affects the newly added low-rank parameters. The original gradient is: To reduce the interference of current domain updates on historical domain knowledge, this invention incorporates the historical knowledge subspace into the original gradient. The gradient components are removed, retaining only those orthogonal to the historical knowledge subspace. Specifically, the gradient update is modified as follows:

[0051] in, This represents the projection matrix over the historical knowledge subspace. This represents the projection component of the original gradient into the historical knowledge subspace. This represents the refreshed orthogonal gradient. Subsequently, the refreshed orthogonal gradient is used to update the current domain's low-rank adaptation parameters. and Through this step, the current domain learning is guided to the orthogonal complement direction of the historical knowledge subspace, so that the new domain knowledge occupies the parameter update direction that complements the historical domain knowledge as much as possible, thereby reducing inter-domain interference and mitigating catastrophic forgetting.

[0052] S105, Topology-Aware Consistency Constraint Step: Low-rank subspace expansion and gradient orthogonal refresh can reduce parameter interference between different domains, but since each domain uses independent low-rank branches, the semantic representation of the same category may be inconsistent in different domains, resulting in semantic dispersion. Therefore, this invention introduces a topology-aware consistency constraint to maintain the consistency of cross-domain category semantic structure. First, a category prototype is constructed for each category in the base domain. For each category... Suppose that this category has a total of Each target instance has the following characteristics: The classification confidence level is Then the category The basic domain prototype is defined as follows: .in, Represents the categories in the base field The semantic prototypes. The basic domain prototypes of all categories constitute the basic semantic topology set: Where N represents the number of categories. This basic semantic topology set remains fixed during subsequent domain incremental training and is used as a global semantic reference. When the t-th domain is reached, the category prototype set for the current domain is calculated in the same way: To maintain topological consistency between the current domain category prototype and the base domain category prototype, this invention calculates the semantic matching weight between the current domain prototype and the base domain prototype: .in, This represents the matching confidence between the prototype of the i-th category in the current domain and the prototype of the j-th category in the base domain. Then, a topology-aware consistency loss is constructed: .

[0053] in, Let represent the cosine similarity function. This is achieved by minimizing... This can constrain the alignment of category prototypes in the current domain with the semantic structure corresponding to the base domain, thereby enhancing the semantic consistency of the same category across different domains and effectively alleviating the problem of cross-domain semantic fragmentation.

[0054] S106, Joint Optimization and Parameter Update Steps: During the first domain training, the model is optimized using only the standard object detection loss: During training in the t-th domain, where t ≥ 2, this invention uses both the standard object detection loss and the topology-aware consistency loss for training: Where λ represents the weighting coefficient of the topology-aware consistency loss. During backpropagation, firstly according to... Calculate the new low-rank adaptation parameters and The original gradient; then utilizing the historical knowledge subspace The original gradient is orthogonally refreshed to obtain the orthogonal gradient; finally, the current domain low-rank adaptation branch is updated based on the orthogonal gradient. The parameters of the historical domain low-rank adaptation branch and the pre-trained backbone are kept frozen.

[0055] In summary, this invention achieves continuous domain adaptation under conditions without historical sample replay through low-rank subspace expansion, gradient orthogonal refresh, and topology-aware consistency. Upon arrival of each new domain, a dedicated LoRA low-rank adaptation branch is introduced, and historical branches are frozen. Simultaneously, the branches of each domain are fused into a unified weight using a linearly additive approach, thereby avoiding explicit domain selection and multiple forward propagations during the inference phase. The historical subspace is obtained by performing singular value decomposition on the features of the fused model, and the gradient of the new domain is projected onto its orthogonal complement direction, reducing interference with historical knowledge. A global semantic topology is constructed using basic domain category prototypes, and new domain category prototypes are constrained to maintain consistency with it, thereby alleviating cross-domain semantic fragmentation and improving multi-domain generalization ability.

[0056] Furthermore, to verify the effectiveness of the proposed method, experiments were conducted on three typical domain incremental object detection datasets: Pascal VOC, BDD100K, and VOC-Corruption. The experiments used ViTDet as the basic object detection framework, performing continuous domain learning without historical sample playback. The evaluation metric was the mean accuracy of object detection (mAP).

[0057] To verify the function of each component module of this invention, ablation experiments were conducted on low-rank subspace expansion, gradient orthogonal refresh, and topology-aware consistency. The experimental results are shown in Table 1. Table 1 shows that when only the basic fine-tuning method is used, the model achieves only 44.2% mAP in the final stage. After introducing low-rank subspace expansion, the performance in the final stage improves to 60.9%, indicating that establishing independent low-rank subspaces for different domains can effectively reduce parameter interference between new and old domains. Further adding gradient orthogonal refresh improves the final stage performance to 64.9%, indicating that gradient orthogonal constraints can effectively protect historical knowledge and alleviate catastrophic forgetting. Adding topology-aware consistency constraints further maintains the consistency of category semantic structure between different domains. When the three modules work together, this invention achieves stage detection accuracies of 73.0%, 67.3%, and 65.2%, obtaining the best detection performance, indicating that the three technical modules proposed in this invention have a good synergistic effect.

[0058] To verify the overall performance of this invention, it was compared with existing domain incremental object detection methods. The experimental results are shown in Table 2. Table 2 shows that in the Pascal VOC continuous domain learning task, this invention achieved a final mAP of 65.2%, which is 8.4 percentage points higher than the existing sample-free method LDB and 5.6 percentage points higher than LDB+SOYO. Even compared with the TP-DIOD-B method using historical sample playback, this invention still achieves better performance, indicating that it can effectively maintain historical domain detection capability under conditions without historical samples. In the BDD100K continuous domain learning task, this invention achieved a final mAP of 58.2%, which is 7.1 percentage points higher than LDB and 6.5 percentage points higher than LDB+SOYO. This indicates that this invention is also applicable to continuous domain changes in real-world autonomous driving scenarios and can effectively improve the cross-domain adaptability of the object detection model. To further verify the robustness of this invention in complex continuous domain offset environments, experiments were conducted on the VOC-Corruption dataset. The experimental results are shown in Table 3. As shown in Table 3, this invention achieves superior detection performance compared to existing methods under various visual degradation conditions, including continuous noise, blurring, weather, and digit degradation, reaching a final mAP of 61.0%, an improvement of 10.3 percentage points compared to LDB. Furthermore, this invention maintains high detection accuracy in both old and new domains, indicating that the proposed low-rank subspace expansion, gradient orthogonal refresh, and topology-aware consistency mechanisms effectively balance historical knowledge preservation and new domain learning capabilities, demonstrating good stability and adaptability.

[0059] Table 1 Ablation experiment results of each module of the present invention

[0060] Table 2 Comparison of experimental results of the present invention and existing methods on Pascal VOC series and BDD100K series.

[0061] Table 3 Performance evolution results of the present invention at different stages of the VOC-Corruption series

[0062] Figure 3 Further experimental analysis results of the present invention are presented. Figure 3 (a) in the figure verifies the changes in model performance under different LoRA subspace rank settings. It can be seen that the present invention maintains stable performance under different rank parameter settings, indicating that the proposed low-rank subspace extension has good parameter robustness. Figure 3(b) in the figure shows the effect of topology-aware consistency constraints on the preservation of category semantic structure. It can be seen that after adding topology consistency constraints, the feature distribution of the same category in different domains is more compact, and cross-domain semantic consistency is significantly enhanced. Figure 3 (c) shows the changes in the loss function during training. It can be seen that orthogonal gradient refresh does not affect the normal convergence of the model, and can effectively stabilize the optimization direction during the continuous learning process.

[0063] Figure 4 Visualizations of detection results from different methods in the domain incremental target detection task are presented. It can be seen that, compared to existing methods, this invention effectively reduces target missed detections, false detections, and bounding box localization errors. Even in complex domain offset scenarios, it can still accurately detect target position and category, demonstrating that this invention can fully preserve historical domain knowledge and quickly adapt to new domain features, exhibiting better detection accuracy and cross-domain generalization ability. Figure 4 In (a), Ground Truth (GT, true label / benchmark truth) is the standard correct result in the dataset, which serves as a reference standard for evaluating the quality of the algorithm and is the result of the experimental benchmark. Figure 4 In (b), LDB is an abbreviation for a comparison algorithm / baseline model. LDB is a comparison experiment result diagram of the classic Local Difference Binary algorithm. Figure 4 In the diagram (c), Ours refers to the self-developed method proposed in this invention, which is used to compare the results with the benchmark and other existing methods. It can be seen that the method of this invention reduces prediction errors and produces more accurate bounding box regression, demonstrating higher detection reliability.

[0064] In summary, the domain incremental image target detection method proposed in this invention, through three key technologies—low-rank subspace expansion, gradient orthogonal refresh, and topology-aware consistency—achieves an effective balance between preserving historical knowledge and learning new domains without requiring the storage of historical samples, explicit domain identification, or multiple forward inferences. Experimental results on multiple public domain incremental target detection datasets demonstrate that this invention can significantly improve the model's detection performance, continuous learning ability, and cross-domain generalization ability, possessing high engineering application value.

[0065] Example 3 To implement the methods of the above embodiments, the present invention also provides a domain incremental image target detection system 10 based on orthogonal knowledge updating, such as... Figure 5 As shown, it includes: The low-rank subspace expansion fusion inference module 100 is used to send the input image in the domain into the pre-trained target detection model when the new domain image data arrives. In the linear transformation layer of the model feature extractor, an independent low-rank adaptation branch is introduced for the current image domain, and the low-rank adaptation branches corresponding to all historical image domains are frozen. The domain branches are fused into a unified weight in a linearly additive manner to perform forward inference on the input image. The historical knowledge subspace construction module 200 is used to construct the historical knowledge subspace based on a model that integrates the low-rank branch of the historical image domain. The current domain input image is fed into the model, the image feature matrix output by each linear layer of the feature extractor is extracted and singular value decomposition is performed, and the first few singular vectors that retain the main energy of the image features are extracted. The gradient orthogonal refresh module 300 is used to calculate the original gradient of the target detection loss with respect to the low-rank adaptation parameters when training the low-rank adaptation parameters of the current image domain, in combination with the detection annotations corresponding to the input image, remove the components of the original gradient that fall into the historical knowledge subspace, retain only the gradient components that are orthogonal to the historical knowledge subspace, and use the orthogonal gradient to update the low-rank adaptation parameters of the current domain. The topology-aware consistency constraint module 400 is used to extract image feature prototypes of each category based on the input image of the base domain to construct a global semantic topology. It inputs the current domain image and extracts the corresponding category feature prototypes, calculates the topology consistency loss between the feature prototypes of each category in the current domain and the global semantic topology, and jointly optimizes the topology consistency loss with the target detection loss to constrain the category prototypes of the same type of target in different image domains to align with the semantic structure corresponding to the base domain.

[0066] The domain incremental image target detection system based on orthogonal knowledge refresh in this invention achieves effective adaptation to new domains and stable maintenance of historical domain knowledge without storing historical samples, relying on explicit domain selection, or requiring multiple redundant forward inferences. This avoids the new domain learning from overwriting historical knowledge and alleviates catastrophic forgetting. At the same time, it reduces inter-domain parameter interference through low-rank subspace expansion, guides the update direction to avoid historical knowledge subspaces through gradient orthogonal refresh, and alleviates semantic fragmentation problems through topological consistency constraints. Ultimately, it improves the stability, plasticity, and cross-domain generalization ability of the target detection model in the process of continuous learning in multiple domains.

[0067] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0068] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0069] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A domain incremental image target detection method based on orthogonal knowledge refresh, characterized in that, include: S1. When the new domain image data arrives, the input image in the domain is fed into the pre-trained target detection model. In the linear transformation layer of the model feature extractor, an independent low-rank adaptation branch is introduced for the current image domain, and the low-rank adaptation branches corresponding to all historical image domains are frozen. The domain branches are fused into a unified weight in a linearly additive manner, and forward inference is performed on the input image. S2, based on a model that integrates low-rank branches of the historical image domain, feeds the current domain input image into the model, extracts the image feature matrix output by each linear layer of the feature extractor and performs singular value decomposition, extracts the first few singular vectors that retain the main energy of the image features, and constructs the historical knowledge subspace; S3, when training the low-rank adaptation parameters of the current image domain, calculate the original gradient of the target detection loss with respect to the low-rank adaptation parameters by combining the detection annotations corresponding to the input image, remove the components of the original gradient that fall into the historical knowledge subspace, retain only the gradient components that are orthogonal to the historical knowledge subspace, and use the orthogonal gradient to update the low-rank adaptation parameters of the current domain. S4. Based on the input image of the base domain, extract the image feature prototypes of each category to construct a global semantic topology. Input the current domain image and extract the corresponding category feature prototypes. Calculate the topological consistency loss between the feature prototypes of each category in the current domain and the global semantic topology. Then, jointly optimize the topological consistency loss with the object detection loss to constrain the category prototypes of the same type of object in different image domains to align with the semantic structure corresponding to the base domain.

2. The method as described in claim 1, characterized in that, The introduction of an independent low-rank adaptation branch for the current image domain includes: At each linear transformation layer of the feature extractor, a low-rank fitting branch consisting of a dimensionality reduction matrix and an increased dimensionality matrix is ​​constructed for the current image domain; where... The dimensionality reduction matrix is ​​used to map the hidden state output by the linear transformation layer of the input image to a low-dimensional feature space, and the dimensionality increase matrix is ​​used to restore and map the low-dimensional space features back to the original feature dimension space.

3. The method as described in claim 1, characterized in that, The step of fusing the branches of each domain into a unified weight in a linearly additive manner and performing forward inference on the input image includes: superimposing the weights of the current low-rank adaptation branches of the image domain with the weight matrix after fusing all historical low-rank adaptation branches of the image domain to obtain a new unified weight matrix, and using the unified weight matrix to complete the forward calculation of the input image.

4. The method as described in claim 1, characterized in that, The model, which incorporates low-rank branches from the historical domain, performs singular value decomposition on the feature matrices output by each linear layer of the feature extractor for the current domain training samples, including: The training samples of the current domain input image are input into the model parameterized by the low-rank branch weights of the fused historical image domain. The hierarchical image feature matrix output by each linear layer of the feature extractor is obtained, and singular value decomposition is performed on the hierarchical image feature matrix to obtain the orthogonal matrix and the singular value matrix.

5. The method as described in claim 1, characterized in that, Removing the components of the original gradient that fall into the historical knowledge subspace and retaining only the gradient components orthogonal to the historical knowledge subspace includes: calculating the projection component of the original gradient onto the historical knowledge subspace and subtracting the projection component from the original gradient to obtain the orthogonal gradient.

6. The method as described in claim 1, characterized in that, The construction of a global semantic topology based on feature prototypes of each category in the base domain includes: for each category in the base domain, calculating a weighted average feature as the semantic prototype of the category based on the features of all target instances of that category and their classification confidence, and forming a global semantic topology set by combining the semantic prototypes of all categories.

7. The method as described in claim 1, characterized in that, The calculation of the topological consistency loss between the feature prototypes of each category in the current domain and the global semantic topology includes: Calculate the semantic matching weights between the feature prototypes of each category in the current domain and the semantic prototypes of each category in the base domain, and construct the topological consistency loss based on the semantic matching weights and cosine similarity.

8. The method as described in claim 1, characterized in that, The joint optimization of the topology consistency loss and the target detection loss includes: weighted summation of the target detection loss and the topology consistency loss to obtain a joint loss function, and calculation of the original gradient of the low-rank adaptation parameter of the current domain based on the joint loss function.

9. The method as described in claim 1, characterized in that, The method further includes: when the first domain arrives, training the pre-trained object detection model using the standard object detection loss to obtain the base domain model, and freezing most of the parameters of the base domain model.

10. A domain incremental image target detection system based on orthogonal knowledge updating, characterized in that, include: The low-rank subspace expansion fusion inference module is used to feed the input image in the domain into the pre-trained target detection model when new domain image data arrives. In the linear transformation layer of the model feature extractor, an independent low-rank adaptation branch is introduced for the current image domain, and the low-rank adaptation branches corresponding to all historical image domains are frozen. The domain branches are fused into a unified weight in a linearly additive manner to perform forward inference on the input image. The historical knowledge subspace construction module is used to construct the historical knowledge subspace based on a model that integrates the low-rank branch of the historical image domain. The current domain input image is fed into the model, the image feature matrix output by each linear layer of the feature extractor is extracted and singular value decomposition is performed, and the first few singular vectors that retain the main energy of the image features are extracted. The gradient orthogonal refresh module is used to calculate the original gradient of the target detection loss with respect to the low-rank adaptation parameters when training the low-rank adaptation parameters of the current image domain, in combination with the detection annotations corresponding to the input image, remove the components of the original gradient that fall into the historical knowledge subspace, retain only the gradient components that are orthogonal to the historical knowledge subspace, and use the orthogonal gradient to update the low-rank adaptation parameters of the current domain. The topology-aware consistency constraint module is used to extract image feature prototypes of each category based on the input image of the base domain to construct a global semantic topology. It takes the current domain image as input and extracts the corresponding category feature prototypes, calculates the topology consistency loss between the feature prototypes of each category in the current domain and the global semantic topology, and jointly optimizes the topology consistency loss with the object detection loss to constrain the category prototypes of the same type of object in different image domains to align with the semantic structure corresponding to the base domain.

Images