Domain increment-based target detection method, device, equipment, medium and product

Abstract

The application relates to the technical field of target detection and discloses a target detection method and device based on domain increment, equipment, a medium and a product. The method comprises the following steps: receiving a prediction sample; extracting a target sample general feature of the prediction sample; matching the target sample general feature with domain general features of a plurality of domain increments configured by model pre-training, determining a target domain corresponding to the prediction sample, and taking a target detection model trained based on sample data of the target domain as a target model; and performing target detection on the prediction sample based on the target model. Therefore, the matching degree of the prediction sample and the domain increment is used to select applicable target detection model parameters, the adaptability of the target detection model to a new domain is effectively solved, and the stability and the accuracy of the target detection result of the target detection model are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and specifically to target detection methods, devices, equipment, media, and products based on domain increment. Background Technology

[0002] Object detection is a crucial task in computer vision, with significant application prospects in fields such as autonomous driving and national defense. However, in traditional object detection, even well-performing models gradually become ineffective due to environmental factors such as weather and seasons, as well as changes in the color and shape of the detected targets, making it difficult to continuously adapt to the evolving real world. To address the dynamic evolution of similar targets in the real world, domain incremental object detection has emerged. Domain incremental object detection incrementally trains a pre-trained model using new data, endowing the model with continuous learning capabilities while saving the cost of retraining old tasks, thus possessing significant research value and application prospects. However, while domain incremental object detection provides models with continuous learning capabilities, it often leads to irreversible performance degradation in older domains, a phenomenon known as catastrophic forgetting.

[0003] To address this problem, an effective solution is to add representative training samples from the old domain to the training data of the new domain, thereby reviewing the old domain and mitigating forgetting. A classic approach is iCaRL (Incremental Classifier and Representation Learning). iCaRL uses the sample features extracted by the model as a basis, retaining several sets of sample data for each domain that are closest to the average features of the samples. However, such methods are often limited by storage space and data privacy constraints. Therefore, low-rank adaptation methods, which store only a small number of model parameters for each domain, are gaining increasing attention. Representative low-rank adaptation methods include InfLoRA (Interference-Free Low-Rank Adaptation for Continual Learning) and SD-LoRA (Scalable Decoupled Low-Rank Adaptation for ClassIncremental Learning). These low-rank adaptation methods largely mitigate catastrophic forgetting by maintaining the stability of the old domain's feature distribution. In addition, there is another method called LDB (Non-exemplarDomain Incremental Object Detection via Learning Domain Bias). LDB proposes to learn a unique bias only for each domain, which can also adapt to huge domain differences between different domains with a small number of stored parameters. LDB can effectively alleviate the catastrophic forgetting phenomenon.

[0004] However, the object detection methods mentioned above all ignore the problem that the features learned by the model may be non-target features. Non-target features are usually features specific to a certain domain, which can easily lead to the plasticity-stability dilemma. Figure 1 This paper illustrates the detection results and model-focused features of SD-LoRA, InfLoRA, and LDB methods after sequentially learning two domains. (Reference) Figure 1Based on the Ground Truth annotations of Domain D1 and Domain D2, this paper provides a very intuitive demonstration of the Plasticity and Stability of related techniques such as SD-LoRA, InfLoRA, and LDB. These methods, after learning the first domain, treat the "Grassland" feature, which is unrelated to the "Sheep" objective, as an important feature. This leads to the missing "Grassland" feature in the new domain, misleading the model's predictions, reducing the model's performance in the new domain, and affecting the model's Plasticity. Furthermore, maintaining the non-existent "Grassland" feature in the new domain is difficult, thus leading to the forgetting of the "Grassland" feature and affecting the model's Stability. Summary of the Invention

[0005] This invention provides a target detection method, apparatus, device, medium, and product based on domain increment, to solve the problem of the adaptability of target detection models to new domains in related technologies, effectively improve the stability of target detection models, and enhance the accuracy of target detection results.

[0006] In a first aspect, the present invention provides a target detection method based on domain increment, the method comprising: Receive predicted samples; Extract the common features of the target samples from the prediction samples; The target sample's general features are matched with the domain general features of multiple domain increments configured in the model pre-training to determine the target domain corresponding to the predicted sample. The target detection model trained based on the sample data of the target domain is then used as the target model. Based on the target model, target detection is performed on the predicted samples.

[0007] This invention presents a domain-increment-based target detection method. It pre-trains models on multiple new domains using different domain increments, obtaining multiple sets of parameters for target detection models based on different domains. Based on the extracted common features of the target samples, the target domain corresponding to the predicted sample is determined, and the predicted sample is then detected using the target detection model corresponding to the target domain. Thus, by selecting appropriate target detection model parameters based on the degree of matching between the predicted sample and the domain increment, the adaptability of the target detection model to new domains is effectively solved, significantly improving the stability of the target detection model and the accuracy of the target detection results.

[0008] In some alternative implementations, the method further includes, before receiving the predicted samples: For each new domain after the domain increment, a target detection model based on the new domain is trained using the sample dataset of the new domain.

[0009] In some alternative implementations, a new domain-based object detection model is trained based on a sample dataset from the new domain, including: For multiple batches of training samples, the gradient information of each batch of training samples is calculated sequentially. Based on gradient information, the gradient correlation between the current new domain and multiple historical domains is calculated respectively; Based on gradient correlation, the directional similarity between the current new domain and multiple historical domains is calculated respectively; Based on gradient correlation and directional similarity, the gradient information of the new domain is corrected, and the target detection model is optimized based on the corrected gradient information to obtain the model parameters based on the new domain. For the current batch of training samples, the pre-training knowledge common to all domains of the object detection network is used to extract sample features, and the domain average feature of the new domain is calculated based on the extracted sample features, until training is completed for all batches of training samples.

[0010] In some optional implementations, before sequentially calculating the gradient information of multiple batches of training samples, the method further includes: The backbone network based on the object detection model pre-trained parameters in each layer is configured with first new domain parameters and second new domain parameters for the new domain.

[0011] In some optional implementations, the gradient correlation between the current new domain and multiple historical domains is calculated based on gradient information, including: The gradient projection method is used to calculate the average gradient correlation component of the current new domain relative to the historical domain; Based on the average gradient correlation of the current new domain with respect to the historical domains, the gradient correlation between the current new domain and multiple historical domains is calculated.

[0012] In some optional implementations, based on gradient correlation, the directional similarity between the current new domain and multiple historical domains is calculated, including: Using the cosine similarity calculation method, the directional similarity between the average gradient correlation component of the current new domain relative to the historical domain and the average gradient of the historical domain is calculated, and the calculation result is used as the directional similarity between the current new domain and multiple historical domains.

[0013] Secondly, the present invention provides a target detection device based on domain increment, the device comprising: The receiving module is used to receive the predicted samples; The feature extraction module is used to extract the common features of the target sample in the prediction sample; The target domain module is used to match the general features of the target samples with the general features of the multiple domain increments configured in the model pre-training, determine the target domain corresponding to the predicted sample, and use the target detection model trained based on the sample data of the target domain as the target model. The detection module is used to perform target detection on the predicted samples based on the target model.

[0014] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the target detection method based on the first aspect or any corresponding embodiment described above.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the domain increment-based target detection method of the first aspect or any corresponding embodiment described above.

[0016] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the domain increment-based target detection method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This paper shows the detection results and model-focused features of SD-LoRA, InfLoRA, and LDB methods after sequentially learning two domains. Figure 2 This is a schematic diagram of the model architecture used in the target detection method based on domain increment of this invention; Figure 3 This is a schematic diagram of the first process of the target detection method based on domain increment according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a second process for a target detection method based on domain increment according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating a specific application example of the target detection method based on domain increment according to an embodiment of the present invention; Figure 6This is a schematic diagram illustrating the effect of a specific application example of the target detection method based on domain increment according to an embodiment of the present invention; Figure 7 This is a structural block diagram of a domain-incremental target detection device according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] This invention provides a target detection method, apparatus, device, medium, and product based on domain increment, to solve the problem of the adaptability of target detection models to new domains in related technologies, effectively improve the stability of target detection models, and enhance the accuracy of target detection results.

[0023] Specifically, the object detection method based on domain increment provided by this invention is an object-aware low-rank adaptation (OA-LoRA) method. It encourages the model to learn target features while restricting the learning of non-target features, thus alleviating the plasticity-stability dilemma faced by domain increment object detection and improving the continuous adaptability and application capability of existing object detection methods in the real world. Furthermore, since different domains in domain increment object detection have the same target types, target features are similar across domains. If the model can learn target features well, the target features learned in the old domain can also assist in the detection of targets in the new domain, improving the model's plasticity. In addition, the target features learned in the old domain are similar to those in the new domain, thus effectively mitigating the forgetting of old domain features in the new domain and improving the model's stability. Based on this, this invention approximates the distinction between target and non-target features based on inter-domain gradient similarity, encouraging the model to learn target features and restricting the learning of non-target features by correcting the gradient. However, considering the special case that non-target features may also be similar in the old domain, this approximation may not effectively distinguish between target and non-target features. To mitigate the forgetting phenomenon of the model obtained using the above training method in this special case, the present invention first predicts the domain source of the sample in the inference stage of applying the object detection model, and then selects appropriate model parameters for inference prediction.

[0024] Figure 2 This is a schematic diagram of the model architecture used in the target detection method based on domain increment of this invention. (Refer to...) Figure 2 Pre-trained W0 represents the pre-trained parameters of each layer in the backbone network. To enable the model to continuously learn new knowledge, for the newly added domain The sample dataset, the pre-trained parameters of each layer in the backbone network Insert new LoRA parameters and , and This represents the new parameters inserted when learning the (t-1)th domain sample dataset. For each new domain, model training is performed based on the domain increment. Each training iteration yields model parameters based on the domain increment, which can also be referred to as the object detection model trained on the sample data of the new domain. For each new domain, the model parameters and domain-specific features related to that domain are recorded. That is... Figure 1 The old parameters and new parameters.

[0025] OAGC (Object Gradient Correction) determines the correlation between the new and old gradient spaces based on the average gradient correlation between the new and old gradient spaces. A weak correlation indicates a negligible relationship, and the new and old domains are considered uncorrelated. A strong correlation, in which case cosine similarity can be used to determine whether the correlation is positive or negative. If the correlation is positive, the general features of the old domain are designated as object features, and promoted learning is applied. If the correlation is negative, the general features of the old domain are designated as non-object features, and restricted learning is applied.

[0026] During the inference process using the object detection model, Feature Similarity-based Domain Selection (FSDS) is first performed to select target domains that share common features with the target samples in the prediction sample. For details, please refer to the following... Figure 3 Steps S301 to S303 of the illustrated embodiment will not be repeated here.

[0027] This embodiment provides a target detection method based on domain increment, which can be used in the target detection process of computer vision. Figure 3 This is a schematic diagram of the first process of the target detection method based on domain increment according to an embodiment of the present invention, as follows: Figure 3 As shown, the process includes the following steps: Step S301: Receive the predicted sample.

[0028] Here you can refer to Figure 1 The test sample (prediction sample) can be any prediction sample to be inferred. .here Figure 1As shown by z, when a predicted sample is received, if the parameters of a unified object detection model are directly used to perform object detection on the predicted sample, the problems mentioned in the background art may exist. Meanwhile, to avoid the huge time overhead caused by using model parameters for inference in each different domain, this invention utilizes pre-trained knowledge to extract its common features as the basis for predicting the domain source of the sample, and predicts that the sample originates from the mininum distance (the most similar domain). .here Figure 1 The target domain for the predicted sample is shown as "D?", meaning we first need to determine which domain's general features the predicted sample has a higher similarity to. For example, after feature extraction, the predicted sample has a higher similarity to the general features of domain t-1. If the similarity is higher, then it will be with As the target domain. It should be noted that... Figure 1 The comparison with domain t-1 and domain t is only shown in the figure. Here, it is necessary to compare the general features of the target sample of the predicted sample with the general features of all historical domains.

[0029] Step S302: Extract the target sample general features of the predicted sample.

[0030] In some alternative implementations, feature extraction can be performed using a general feature extraction method.

[0031] Step S303: Match the general features of the target sample with the general features of the multiple domain increments configured in the model pre-training to determine the target domain corresponding to the predicted sample, and use the target detection model trained based on the sample data of the target domain as the target model.

[0032] In some alternative implementations, the target domain can be determined using the following formula:

[0033] Where k represents the index of the target domain; This means finding the norm that satisfies The parameter j with the smallest value, j∈[1,t]; This represents the backbone network output using pre-trained parameters (W0).

[0034] It should be noted that the output of the backbone network of the object detection model will be discussed in the following text. Figure 4 Step S401 of the illustrated embodiment will be described, and will not be repeated here.

[0035] Furthermore, regardless of which new domain the target detection model is trained on, it is based on the same pre-trained model parameters and model architecture of the same backbone network to obtain the corresponding new model parameters for the new domain. For each new domain, the values ​​of the corresponding new model parameters are recorded during model training.

[0036] In some alternative implementations, the target detection model trained based on sample data from the target domain is used as the target model. Essentially, this involves setting the model parameters for other domains trained after the target domain to zero, and then training the newly added model parameters using the domain-specific features corresponding to the determined target domain. The newly added model parameters can be referenced above. Figure 2 The Ai and Bi shown.

[0037] Specifically, to avoid the catastrophic forgetting phenomenon caused by learning future domains, this invention selects appropriate domain parameters for inference. For the sample source target domain obtained according to step S302... ,Bundle After , ...and so on, setting all parameters of the new domain to 0, yields the output of each layer that performs inference on the predicted sample zi:

[0038] Step S304: Based on the target model, perform target detection on the predicted samples.

[0039] Specifically, step S303 is used to obtain the input and output relationship of the modified target detection model, and the target detection result can be obtained by performing target detection or inference prediction on the predicted sample.

[0040] The input-output relationship of the target detection model is described below. Figure 4 Step S401 of the illustrated embodiment will not be described again here.

[0041] This invention presents a domain-increment-based target detection method. It pre-trains models on multiple new domains using different domain increments, obtaining multiple sets of parameters for target detection models based on different domains. Based on the extracted common features of the target samples, the target domain corresponding to the predicted sample is determined, and the predicted sample is then detected using the target detection model corresponding to the target domain. Thus, by selecting appropriate target detection model parameters based on the degree of matching between the predicted sample and the domain increment, the adaptability of the target detection model to new domains is effectively solved, significantly improving the stability of the target detection model and the accuracy of the target detection results.

[0042] This embodiment provides a target detection method based on domain increment, which can be used in the target detection process of computer vision. Figure 4This is a schematic diagram of a second process for a target detection method based on domain increment according to an embodiment of the present invention, as shown below. Figure 4 As shown, the process includes the following steps: Step S401: For each new domain after the domain increment, train a target detection model based on the sample dataset of the new domain.

[0043] Specifically, step S401 may include: Step S4011: Based on the pre-trained parameters in each layer of the backbone network of the object detection model, configure the first new domain parameters and the second new domain parameters for the new domain.

[0044] Specifically, to enable the model to continuously learn new knowledge, for newly added domains... The sample dataset is used to pre-train parameters in each layer of the backbone network. Insert new LoRA parameters and Specifically, the output of each layer With input The relationship is as follows:

[0045] Where e represents the input to each layer of the target detection model for the i-th domain; h represents the output of each layer of the target detection model for the i-th domain; This represents the model parameters for the target detection model in the i-th domain, including A. i and B i ; Indicates a new domain The first new field parameter, Indicates a new domain The second new field parameter; This represents the pre-trained parameters in each layer of the backbone network.

[0046] here, and You can refer to the above. Figure 2 China The explanation.

[0047] Step S4012: For multiple batches of training samples, calculate the gradient information of each batch of training samples sequentially.

[0048] Specifically, to learn about the new domain, the first step is to set the new domain parameters. For trainable parameters, the loss function is calculated in batches using any object detection method, and the gradient information of each batch of training samples is obtained. .

[0049] Step S4013: Based on gradient information, calculate the gradient correlation between the current new domain and multiple historical domains respectively.

[0050] In some alternative implementations, step S4013 may include: Step a1: Use the gradient projection method to calculate the average gradient correlation part of the current new domain relative to the historical domain.

[0051] Specifically, since any object detection method used in step S4012 may learn non-target features, and different domains in domain-incremental object detection have the same target category, the target features are similar across domains. Therefore, the calculated gradient can be used to determine the target features. This approximates the distinction between target features and non-target features. Therefore, a momentum update method is first used to comprehensively record the gradient information of the current domain. :

[0052] in, This represents a pre-defined hyperparameter, which is a constant and can be set to 0.95; Represents gradient information for the current domain; This represents the gradient information of the historical domain, which can be any historical domain that has already been trained.

[0053] Furthermore, to distinguish between target features and non-target features, the gradients of the entire learned historical domain and the new domain are compared to see if they have similar gradients. First, the historical domain is loaded. The gradient information recorded in step a1 is used during training. .

[0054] Furthermore, since different domains may emphasize different features of the target, in order to distinguish between target features and non-target features, the historical domain should first be determined. New domains of current learning Does it have a relevant feature gradient? Therefore, this invention obtains the gradient in the new domain that is related to the old domain based on gradient projection. That is, the gradient projection method is used to calculate the average gradient correlation component of the current new domain relative to the historical domain.

[0055]

[0056] in, This represents the part of the gradient in field t that is correlated with the average gradient in field k. The average gradient in the domain k; The gradient in the domain t is represented by . Representation matrix The transpose of .

[0057] Step a2: Based on the average gradient correlation of the current new domain with respect to the historical domains, calculate the gradient correlation between the current new domain and multiple historical domains.

[0058] Specifically, in order to determine whether the historical domain and the new domain are correlated, this invention calculates the gradient correlation between the current new domain and multiple historical domains based on the correlation gradient obtained in step a1. Gradient correlation This value is used to measure the degree of overlap between parameter solution spaces of different domains; a higher value indicates a greater degree of overlap. It can be calculated using the following formula:

[0059] in, This represents the gradient correlation between domain t and domain k; express The norm; express The norm of .

[0060] Step S4014: Based on gradient correlation, calculate the directional similarity between the current new domain and multiple historical domains respectively.

[0061] In some optional implementations, the cosine similarity calculation method is used to calculate the directional similarity between the average gradient correlation part of the current new domain relative to the historical domain and the average gradient of the historical domain, and the calculation result is used as the directional similarity between the current new domain and multiple historical domains to achieve step S4014.

[0062] Specifically, to accurately measure whether related domains have similar gradients and thus approximately distinguish target features, it is necessary to more accurately measure the similarity of their optimization directions within the overlapping parameter solution space. Here, cosine similarity can be used to calculate the relevant gradients. gradient with old domain directional similarity between directional similarity The element The calculation formula is as follows:

[0063] in, express Average gradient with domain k The directional similarity between them is a matrix; Representation matrix The Middle One element; express The List; express The List, This represents the dot product of vectors.

[0064] when When the value of approaches 1, the corresponding gradient is more likely to represent the target feature; conversely, when its value approaches -1, it is more likely to correspond to a non-target feature.

[0065] Step S4015: Based on gradient correlation and directional similarity, the gradient information of the current new domain is corrected, and the target detection model is optimized based on the corrected gradient information to obtain the model parameters based on the new domain.

[0066] Based on the above operation steps S4013 and S4014, the gradient correlation and directional similarity between the current domain and multiple historical domains can be calculated. To encourage the model to learn target features and restrict the model from learning non-target features, this invention comprehensively utilizes information from historical domains, encouraging it to optimize and update in more similar directions within historical domains with higher correlation. Regarding parameter A... t and parameter B t Calculate the corrected gradients respectively. As shown below:

[0067] in, Represents a matrix of all ones. Represents the Hadamard product of matrices; Represents the original gradient; t indicates that the t-th domain is being studied, k∈(0,t-1).

[0068] Step S4016: For the current batch of training samples, use the pre-training knowledge common to each domain of the object detection network to extract sample features, and calculate the domain average feature of the new domain based on the extracted sample features, until all batches of training samples have completed training.

[0069] Specifically, to further mitigate the impact of catastrophic forgetting, this invention records the feature information of each domain as a reference for the source of sample domains. Specifically, for the current batch of training samples... It uses pre-trained knowledge common to various domains to extract features from samples and calculates the domain average features. :

[0070] in, The domain average characteristic of the domain t; This represents the current sample batch size. This represents the total number of trained samples. This represents the output of the backbone network of the object detection model.

[0071] Furthermore, existing optimization algorithms are used to optimize the model parameters based on the corrected gradients until all batches are trained.

[0072] Furthermore, during model training, the domain gradient information of each new domain is saved and recorded. Domain average characteristics The training is terminated once the model parameters Ai and Bi obtained from the training are used.

[0073] Step S402: Receive the predicted sample.

[0074] Please refer to the above for details. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0075] Step S403: Extract the target sample general features of the predicted sample.

[0076] Please refer to the above for details. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0077] Step S404: Match the general features of the target sample with the general features of the multiple domain increments configured in the model pre-training to determine the target domain corresponding to the predicted sample, and use the target detection model trained based on the sample data of the target domain as the target model.

[0078] Please refer to the above for details. Figure 3 Step S303 of the illustrated embodiment will not be described again here.

[0079] Step S405: Based on the target model, perform target detection on the predicted samples.

[0080] Please refer to the above for details. Figure 3 Step S304 of the illustrated embodiment will not be described again here.

[0081] This embodiment provides a target detection method based on domain increment, which can be used in the target detection process of computer vision. Figure 5This is a flowchart illustrating a specific application example of the target detection method based on domain increment according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps: Step S501: Pre-train parameters in each layer of the backbone network of the object detection model. Insert new LoRA parameters .

[0082] Step S502, determine the new LoRA parameters Does the training termination condition meet? If yes, proceed to step S509; otherwise, proceed to step S503.

[0083] Step S503: Calculate the loss function in batches using any object detection method and obtain the gradient information of each batch of training samples. .

[0084] Step S504: Calculate the gradient using the training data.

[0085] Specifically, to learn about a new domain, the parameters of the new domain can be... Set the parameters to trainable parameters, and calculate the loss function in batches using object detection methods such as LDB, InfLoRA, and SD-LoRA, and then calculate its gradient for each batch. .

[0086] Step S505: Use momentum update to comprehensively record the gradient information of the current domain. .

[0087] Step S506: Determine whether to traverse the entire history domain. If yes, proceed to step S508; otherwise, proceed to step S507.

[0088] Step S507: Project the gradient of the new domain onto the historical domain, and calculate the gradient correlation and orientation similarity between the domains.

[0089] For details, please refer to the above. Figure 4 Steps S4012 to S4014 of the illustrated embodiment will not be repeated here.

[0090] Step S508: Correct the gradient by combining the correlation and orientation similarity of the gradient in the historical domain, update the parameters using the corrected gradient, and update the domain average feature.

[0091] For details, please refer to the above. Figure 4 Step S4015 of the illustrated embodiment will not be described again here.

[0092] Step S509: Save the recorded gradient information of the new domain. Average characteristics Terminate training based on the obtained model parameters.

[0093] In a specific application example of the target detection method based on domain increment of this invention, the inference stage of the sample, that is, the process of target detection of the target sample, can be described with the parameters above. Figure 3 The illustrated embodiment will not be described in detail here. Other detailed processes for model training can also be found above. Figure 4 The embodiments shown are not described in detail here.

[0094] Figure 6 This is a schematic diagram illustrating the effect of a specific application example of the target detection method based on domain increment according to an embodiment of the present invention. (Refer to...) Figure 6 This paper presents a comparative analysis of the key feature visualizations of various object detection methods on the Clipart, Watercolor, and Comic domains of the Pascal VOC series datasets. SD-LoRA and InfLoRA primarily focus on the overall style features of the image, leading to missed detections in the latter two domains. While the LDB method captures the edges of haystacks as important features in the second domain, it also results in missed detections in the third domain. In contrast, the domain-incremental object detection method OA-LoRA in this invention consistently focuses on cross-domain object features, thus achieving accurate detection of birds in the image. Therefore, the object detection accuracy of the domain-incremental object detection method in this invention is significantly improved.

[0095] This embodiment also provides a target detection device based on domain increment, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0096] This embodiment provides a target detection device based on domain increment, such as Figure 7 As shown, it includes: Receiver module 701 is used to receive prediction samples; Feature extraction module 702 is used to extract the general features of the target sample of the prediction sample; The target domain module 703 is used to match the general features of the target sample with the general features of the multiple domain increments configured in the model pre-training, determine the target domain corresponding to the predicted sample, and use the target detection model trained based on the sample data of the target domain as the target model. The detection module 704 is used to perform target detection on the predicted sample based on the target model.

[0097] In some alternative implementations, the apparatus further includes, before receiving the predicted sample: The training module is used to train a target detection model based on the sample dataset of each new domain after the domain increment.

[0098] In some alternative implementations, the training module includes: The gradient unit is used to calculate the gradient information of multiple batches of training samples sequentially. The correlation unit is used to calculate the gradient correlation between the current new domain and multiple historical domains based on gradient information. The similarity unit is used to calculate the directional similarity between the current new domain and multiple historical domains based on gradient correlation. The gradient optimization unit is used to correct the gradient information of the new domain based on gradient correlation and directional similarity, and to optimize the target detection model based on the corrected gradient information to obtain the model parameters based on the new domain. The feature unit is used to extract sample features for the current batch of training samples using pre-trained knowledge common to all domains of the object detection network, and to calculate the domain average feature of the new domain based on the extracted sample features, until training is completed for all batches of training samples.

[0099] In some alternative implementations, the training module further includes: The configuration unit is used to configure the first new domain parameters and the second new domain parameters for the new domain based on the pre-trained parameters in each layer of the backbone network of the object detection model before calculating the gradient information of the batch training samples sequentially for multiple batch training samples.

[0100] In some optional implementations, the gradient correlation between the current new domain and multiple historical domains is calculated based on gradient information, including: The projection sub-unit is used to calculate the average gradient correlation component of the current new domain relative to the historical domain using the gradient projection method. The correlation calculation subunit is used to calculate the gradient correlation between the current new domain and multiple historical domains based on the average gradient correlation component of the current new domain relative to the historical domains.

[0101] In some alternative implementations, the similarity unit includes: The cosine calculation subunit is used to calculate the directional similarity between the average gradient correlation part of the current new domain relative to the historical domain and the average gradient of the historical domain using the cosine similarity calculation method. The calculation result is used as the directional similarity between the current new domain and multiple historical domains.

[0102] The target detection device based on domain increment provided in this embodiment of the invention can execute the target detection method based on domain increment provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0103] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0104] The following is a detailed reference. Figure 8 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 801, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 802 or a program loaded from memory 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0105] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0106] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a memory 808, or installed from a ROM 802. When the computer program is executed by the processor 801, it performs the functions defined in the domain increment-based target detection method of the embodiments of the present invention.

[0107] Figure 8The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0108] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the domain increment-based target detection method shown in the above embodiments is implemented.

[0109] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0110] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended technical solutions.

Claims

1. A target detection method based on domain increment, characterized in that, The method includes: Receive predicted samples; Extract the target sample general features from the predicted samples; The target sample's general features are matched with the domain general features of multiple domain increments configured in the model pre-training to determine the target domain corresponding to the predicted sample. The target detection model trained based on the sample data of the target domain is then used as the target model. Based on the target model, target detection is performed on the predicted samples.

2. The method according to claim 1, characterized in that, Prior to receiving the predicted sample, the method further includes: For each new domain after the domain increment, a target detection model based on the sample dataset of the new domain is trained.

3. The method according to claim 2, characterized in that, The training of the object detection model based on the new domain using the sample dataset based on the new domain includes: For multiple batches of training samples, the gradient information of each batch of training samples is calculated sequentially. Based on the gradient information, the gradient correlation between the current new domain and multiple historical domains is calculated respectively. Based on the gradient correlation, the directional similarity between the current new domain and multiple historical domains is calculated respectively. Based on the gradient correlation and the directional similarity, the gradient information of the current new domain is corrected, and the target detection model is optimized based on the corrected gradient information to obtain the model parameters based on the new domain. For the current batch of training samples, the pre-training knowledge common to each domain of the object detection network is used to extract sample features, and the domain average feature of the new domain is calculated based on the extracted sample features, until all batches of training samples have completed training.

4. The method according to claim 3, characterized in that, Before sequentially calculating the gradient information of multiple batches of training samples, the step of training the target detection model based on the new domain sample dataset further includes: Based on the pre-trained parameters in each layer of the backbone network of the object detection model, the first new domain parameters and the second new domain parameters are configured for the new domain.

5. The method according to claim 3, characterized in that, The step of calculating the gradient correlation between the current new domain and multiple historical domains based on the gradient information includes: The gradient projection method is used to calculate the average gradient correlation component of the current new domain relative to the historical domain; Based on the average gradient correlation of the current new domain with respect to the historical domains, the gradient correlation between the current new domain and multiple historical domains is calculated.

6. The method according to claim 3, characterized in that, The step of calculating the directional similarity between the current new domain and multiple historical domains based on the gradient correlation includes: Using the cosine similarity calculation method, the directional similarity between the average gradient correlation component of the current new domain relative to the historical domain and the average gradient of the historical domain is calculated, and the calculation result is used as the directional similarity between the current new domain and multiple historical domains.

7. A target detection device based on domain increment, characterized in that, The device includes: The receiving module is used to receive the predicted samples; The feature extraction module is used to extract the general features of the target sample from the predicted sample; The target domain module is used to match the general features of the target sample with the general features of multiple domain increments configured in the model pre-training, determine the target domain corresponding to the predicted sample, and use the target detection model trained based on the sample data of the target domain as the target model. The detection module is used to perform target detection on the predicted sample based on the target model.

8. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the target detection method based on domain increment as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the domain-increment-based target detection method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the domain-increment-based target detection method according to any one of claims 1 to 6.

Images