Small sample target detection method and device
By dividing the plasticity and stability modules in the small sample object detection model, and adopting a balanced and coordinated migration strategy and Ensembling integrated learning mechanism, the problem of insufficient generalization and robustness performance of the existing small sample object detection algorithm is solved, achieving more efficient and Stable migration effect.
Patent Information
- Application Number
- CN202411405389.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-02-11
AI Technical Summary
The existing small sample object detection algorithm has insufficient generalization and robust performance in long-tail application scenarios, and it is difficult to fully utilize the high generalization potential of the object detection model based on the Transformer model.
By re-initializing the vector dimensionality reduction classifier in the cascading Decoder layer of the attention mechanism object detection model, it is divided into plasticity modules and stability modules, and adopting a balanced and coordinated migration strategy, combined with the Ensembling integrated learning mechanism, Improve the generalization and robustness of the model.
It alleviates the contradiction between different requirements for migration strategies of plastic components and stability components during the migration process, improves the migration efficiency and stability of the model, and improves the generalization performance of small samples.
Smart Images
Figure CN119418035A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image target detection, and specifically provides a small sample target detection method and device. Background Art
[0002] With the continuous deepening of the application of large-scale model technology based on Transformer models and massive data pre-training in the field of computer vision, the performance of target detection algorithms based on large-scale model technology has made great progress in recent years. These target detection algorithms are based on the high generalization of large models and have great potential to achieve efficient model migration in personalized and customized long-tail application scenarios under small sample conditions.
[0003] In long-tail application scenarios, it is often difficult to collect a large number of training data samples, and the target detectors obtained based on the migration of a small number of samples often have insufficient generalization and robust performance for detecting new types of task targets. This greatly limits the stability and reliability of such target detection algorithms in actual production scenarios.
[0004] Meta-learning, regularization, or freezing strategies commonly used in existing small-sample target detection algorithms are often inapplicable or difficult to apply to the attention mechanism target detection model based on the Transformer model because they are relatively complex and model-specific. When these traditional methods are applied to large target detection models, they often fail to exert their high generalization potential. Therefore, when performing small-sample new task transfer training for large target detection models, a direct single-stage full or partial parameter fine-tuning solution is generally adopted. This simple solution does not have a targeted design for the limitations of the large target detection model migration process, and still has many limitations in fully exerting the generalization potential. Summary of the invention
[0005] The present invention aims to address the deficiencies of the above-mentioned prior art and provides a small sample target detection method with strong practicality.
[0006] A further technical task of the present invention is to provide a small sample target detection device that is reasonably designed, safe and applicable.
[0007] The technical solution adopted by the present invention to solve the technical problem is:
[0008] A small sample target detection method re-initializes the parameters of the vector dimension reduction classifier in the cascade decoder layer of the attention mechanism target detection model. The classifier parameters are divided into a plasticity module and a stability module.
[0009] A migration strategy for balancing and coordinating a plasticity module and a stability module, wherein the stability module includes a backbone network portion and a portion of components in a cascade decoder layer, and the plasticity module includes another portion of components in a cascade decoder layer;
[0010] Through the Ensembling integrated learning mechanism, the generalization robustness of the attention mechanism target detection model after fine-tuning and migration is improved.
[0011] Furthermore, there are two generalization robust improvement members in the ensemble learning mechanism, namely ensemble member 1 and ensemble member 2. The ensemble member 1 is a CLIP ViT model pre-trained with a large amount of Web data. After pre-training with Internet big data, the member has strong generalization robustness of category discrimination ability.
[0012] The ensemble member 2 is an initial attention mechanism target detection model obtained by pre-training. This member does not perform a few-sample fine-tuning transfer adaptation for downstream new tasks, and retains the generalization robustness of the target positioning capability that is complementary to the model after transfer adaptation.
[0013] Furthermore, the attention mechanism target detection model obtains the migration and adaptation model through the plastic fine-tuning migration warm-up stage and the steady-state fine-tuning migration adaptation stage. During the reasoning stage, the migration and adaptation model enters the integrated reasoning configuration state Ensembling to output the reasoning results under generalized robust integration improvement.
[0014] Furthermore, when the plastic fine-tuning migration warm-up phase is started, the attention mechanism target detection model is initialized and loaded from the parameter image obtained in the pre-training phase. Plastic fine-tuning refers to the fine-tuning phase in which the migration strategy gives priority to fully meeting the plasticity requirements of the plastic modules involved in the migration under the complete freezing protection of the stability module.
[0015] Furthermore, the measures taken in the plastic fine-tuning migration preheating stage are:
[0016] (1) No freezing is performed on the plasticity module, giving it maximum plasticity;
[0017] (2) Completely freeze the stability module to provide maximum protection;
[0018] (3) Extend the training schedule to provide sufficient migration warm-up.
[0019] Furthermore, steady-state fine-tuning refers to a fine-tuning phase that gradually inherits the fine-tuning migration warm-up parameters of the previous stage, expands the range of parameters involved in migration adjustment, and considers the design of migration strategy to meet the needs of moderate fine-tuning migration under stability constraints;
[0020] When the steady-state fine-tuning adaptation transfer phase starts, the attention mechanism target detection model is initialized and loaded from the parameter image obtained in the plastic fine-tuning transfer warm-up phase.
[0021] Furthermore, in the steady-state fine-tuning migration adaptation stage, a migration adaptation strategy with a more harmonious and balanced plasticity and stability of the plastic and stability modules is provided. The measures are as follows:
[0022] (1) The plasticity module is not frozen;
[0023] (2) The stability module partially freezes the underlying backbone network to provide appropriate protection;
[0024] (3) Appropriately shorten the training schedule to prevent the pre-training knowledge from being forgotten.
[0025] Furthermore, the centroid is initialized first in the integrated reasoning stage. When entering the integrated reasoning stage for the first time, it is necessary to initialize the centroid based on the available few-sample dataset S in advance. c Initialize and construct the category centroid groups of integration members 1 and 2;
[0026] For ensemble member 2, the class centroid group p c The initialization construction algorithm is as follows:
[0027]
[0028] Among them, |S c | is the sample size Shot of the small sample data set, is the label of sample j, B represents the ViT model of CLIP, R represents the ROIAlign method, and b sp represents the target box coordinates in the sample label; Formula (2) shows that the centroid of this group of categories is constructed based on the vector u, which is obtained by intercepting the activation map obtained after the image sample x is input into B according to the coordinates of b using the ROIAlign method;
[0029] For ensemble member 2, the class centroid group p c The initialization construction algorithm is as follows:
[0030]
[0031] Among them, |S c | is the sample size Shot of the small sample data set, is the label of sample j, N represents the original pre-trained DETR target detection model without migration adaptation. H Indicates the number of stages in the Decoder layer of the target detection model, b spRepresents the target box coordinates in the sample label;
[0032] Formula (4) shows that the centroid of this group of categories is constructed based on the vector v, which is the image sample x input into S and the target box of each stage is fixed to the true value b sp The Query vector of the final Decoder layer is obtained.
[0033] Furthermore, during ensemble reasoning, for the reasoning sample x, vector u and vector v are obtained as shown in formula (2) and formula (4), and the cosine similarity between them and the category centroid is calculated to obtain the confidence s Im and
[0034] The target category confidence in the result obtained by the transfer attention mechanism target detection model for sample x reasoning is integrated and enhanced, as shown in the following formula:
[0035]
[0036] α+β+γ=1; (5)
[0037] The integration coefficients of different target detection models are different, but they are consistent with the method of integrating and aggregating the results of this stage. For different DETR-like target detection models, they are adjusted and configured according to the verification accuracy of the integrated reasoning.
[0038] A small sample target detection device comprises: at least one memory and at least one processor;
[0039] The at least one memory is used to store a machine-readable program;
[0040] The at least one processor is used to call the machine-readable program to execute a small sample target detection method.
[0041] Compared with the prior art, the small sample target detection method and device of the present invention have the following outstanding beneficial effects:
[0042] The present invention alleviates the contradiction between the different stability and plasticity requirements of the plasticity component and the stability component for the migration strategy during the migration process by fine-tuning the two-stage independent decoupling of the migration stage.
[0043] In the warm-up phase of plastic fine-tuning migration, the configuration of the migration strategy gives plastic components greater migration plasticity than non-decoupled conventional fine-tuning migration, thereby improving their migration efficiency. For stability components, the complete freezing strategy implemented in this phase gives them more thorough pre-training knowledge protection than non-decoupled conventional fine-tuning migration, and they will not be affected by the plastic migration process of plastic components and lose pre-training knowledge.
[0044] In the steady-state fine-tuning migration adaptation stage, the configuration of the migration strategy, on the one hand, suppresses the forgetting of pre-training knowledge through strategies such as moderate freezing and shortening rounds, and improves the stability of model migration. On the other hand, the breadth and range of parameters involved in the adaptive migration are expanded to moderately improve the plasticity of model migration. The migration in this stage achieves a comprehensive balance between stability and plasticity, further improving the performance of the model after migration.
[0045] The steady-state fine-tuning migration adaptation phase inherits and loads the parameters of the independently decoupled plastic fine-tuning warm-up migration phase. The migration fine-tuning of these two phases constitutes a curriculum learning strategy that proceeds from easy to difficult in machine learning. The warm-up migration phase fully initializes the parameters of the adaptation migration phase, reducing the training difficulty of the adaptation migration phase. The adaptation phase makes migration adjustments over a larger parameter range based on the warm-up phase. This joint migration adjustment further improves the overall post-migration performance of the model compared to the warm-up migration phase.
[0046] The independent unmigrated members in the integrated reasoning stage retain the generalization robustness of the target detection model in target classification and target positioning to the greatest extent, following the member diversity principle in ensembling learning. The unmigrated members form a complementary relationship with the migrated model, and the small sample generalization performance of the model is further improved by aggregating the results during reasoning. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0048] Attached Figure 1 It is a schematic diagram of the division of stable and plastic modules in a small sample target detection method;
[0049] Attached Figure 2 It is a schematic diagram of the algorithm operation flow in a small sample target detection method;
[0050] Attached Figure 3 It is a schematic diagram of the preheating stage of plastic fine-tuning migration in a small sample target detection method;
[0051] Attached Figure 4 It is a schematic diagram of the training round plan of Sparse R-CNN in the plastic fine-tuning migration warm-up stage in the small sample target detection method;
[0052] Attached Figure 5 It is a schematic diagram of the steady-state fine-tuning migration adaptation stage in a small sample target detection method;
[0053] Attached Figure 6 It is a schematic diagram of the training round plan of Sparse R-CNN in the steady-state fine-tuning adaptation migration stage in the small sample target detection method;
[0054] Attached Figure 7 It is a schematic diagram of the application process on the typical DETR-like target detection model DINO. DETAILED DESCRIPTION
[0055] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] A best embodiment is given below:
[0057] A small sample target detection method in this embodiment reinitializes parameters of a vector dimensionality reduction classifier in a cascade decoder layer of an attention mechanism target detection model, and the classifier parameters are divided into a plasticity module and a stability module.
[0058] In the fine-tuning migration process of the attention mechanism target detection model, the number of new categories of the downstream task changes, which causes the vector dimensionality reduction classifier in the cascade Decoder layer to re-initialize parameters and lose the knowledge learned in pre-training. These components require a highly plastic migration fine-tuning strategy to achieve knowledge re-learning.
[0059] like Figure 1 As shown, the orange part in the figure represents the classifier parameters in the decoding Decoder layer of each level. The present invention divides these components into plasticity modules.
[0060] The blue and green parts in the figure represent other components in the decoding Decoder layer of each level. Although they also need to be adjusted and adapted for downstream tasks, these components also need to prevent catastrophic forgetting and loss of pre-training knowledge during fine-tuning migration. Therefore, a migration strategy with a balanced coordination of plasticity and stability is required.
[0061] These components are divided into stability modules, which include the backbone network part represented by green in the model and the other components in the cascaded decoder layer represented by blue except the plasticity module part.
[0062] In order to further explain the division of plasticity modules and stability modules in detail, we take the 0th level of the Decoder layer of the typical DETR-like object detection model Sparse R-CNN as an example for formal explanation. This division is similar to other DETR-like object detectors and is not limited to the Sparse R-CNN model. It is only used as an example here.
[0063] The 0th level in the Decoder decoding layer in Sparse R-CNN can be formally written as:
[0064] x0←R pool (x FPN ,b0), (1a)
[0065]
[0066] The above formula is the Sparse R-CNN model pipeline, where q represents the learnable object Query feature. represents the query features after multi-head self-attention, x FPN represents the features output by Backbone with FPN
[32] , b represents the predicted box, c represents the predicted classification label, and R pool represents the RoI pooling operation, A self Denotes the multi-head self-attention module, De cls represents a series of decoding layers that decode Query features into categorical features, F cls Represents the last fully connected layer that outputs the classlogits score based on the classification features.
[0067] In the Sparse R-CNN model, according to the above plasticity division criteria, we can get Added to the plasticity module collection. The components added to the plasticity module collection of other stages are similar. Stability modules are other parameter components in the model pipeline.
[0068] Since transfer fine-tuning will inevitably cause the loss of generalization of the pre-trained knowledge of the large target detection model, in order to further alleviate this problem and improve the generalization robustness of the model after transfer, the present invention also proposes an independent component that does not undergo downstream small sample fine-tuning transfer training. The pre-trained knowledge of this component has higher generalization robustness, which is exactly complementary to the generalization and adaptability of the large model after transfer. Through the Ensembling integrated learning mechanism, this component can further improve the generalization robustness of the attention mechanism target detection model after fine-tuning transfer.
[0069] There are two generalization robust improvement members in the ensemble learning mechanism, namely ensemble member 1 and ensemble member 2. Ensemble member 1 is a CLIP ViT model pre-trained with a large amount of Web data. After pre-training with Internet big data, this member has strong generalization robustness in category discrimination ability.
[0070] Ensemble member 2 is the initial attention mechanism target detection model obtained by pre-training. This member does not perform few-sample fine-tuning transfer adaptation for downstream new tasks, retaining the generalization robustness of the target localization capability that is complementary to the transfer-adapted model.
[0071] like Figure 2 As shown in the figure, after determining the new tasks of the downstream application scenario and meeting the above prerequisites, the attention mechanism target detection model needs to go through the fine-tuning migration preheating phase (LP) of the plastic fine-tuning migration and the steady fine-tuning migration adaptation phase (FR-FT) to obtain the migration adaptation model. In the inference phase, the migration adaptation model enters the ensemble inference configuration state (Ensembling) to output the inference results under the generalized robust ensemble improvement. The blue parameter components are the parameter components in the frozen state. The orange parameter components are the parameter components in the thawed state.
[0072] Specifically, each stage is described in detail as follows:
[0073] When the plastic fine-tuning migration warm-up phase is started, the attention mechanism target detection model is initialized and loaded from the parameter image obtained in the pre-training phase. Plastic fine-tuning refers to the fine-tuning phase in which the migration strategy gives priority to fully satisfying the plasticity requirements of the plastic modules involved in the migration under the complete freezing protection of the stability module.
[0074] like Figure 2 and Figure 3 As shown in the figure, the plastic fine-tuning migration warm-up stage not only fully meets the plastic migration adaptation requirements of the plasticity module, but also fully meets the pre-training knowledge protection requirements of the stability module through complete freezing. This stage decouples the requirements of a larger range of parameter adaptation migration and leaves it to the next stage of progressive migration, that is, the steady-state fine-tuning migration adaptation stage.
[0075] The main measures in this phase are:
[0076] (1) No freezing is performed on the plasticity module, giving it maximum plasticity;
[0077] (2) Completely freeze the stability module to provide maximum protection;
[0078] (3) Extend the training schedule to provide adequate migration warm-up;
[0079] Taking Sparse R-CNN as an example, the training round plan for plastic fine-tuning is as follows Figure 4 shown.
[0080] The training round plan for different small sample sizes Shot is different. The specific training rounds of other types of DETR target detectors may be different, but they need to be consistent with the measures of this stage and can be adjusted according to this round example. The main evaluation basis is the highest verification accuracy under the plastic configuration.
[0081] The steady-state fine-tuning migration adaptation phase refers to the fine-tuning phase that expands the range of parameters involved in migration adjustment and considers the needs of moderate fine-tuning migration under stability constraints in the design of migration strategy, based on the gradual inheritance of the plastic fine-tuning migration preheating parameters of the previous phase. When the steady-state fine-tuning adaptation migration phase is started, the model is initialized and loaded from the parameter image obtained in the plastic fine-tuning preheating migration phase.
[0082] like Figure 2 and Figure 5 As shown, the steady-state fine-tuning migration adaptation stage provides the plasticity and stability modules with a more harmonious and balanced migration adaptation strategy to achieve the best migration effect.
[0083] The main measures are:
[0084] (1) The plasticity module is not frozen;
[0085] (2) The stability module partially freezes the underlying backbone network to provide appropriate protection
[0086] (3) Appropriately shorten the training schedule to prevent the pre-training knowledge from being forgotten.
[0087] Taking Sparse R-CNN as an example, the training round plan for steady-state fine-tuning is as follows: Figure 6 shown.
[0088] The integrated reasoning stage includes centroid initialization and integrated reasoning. When entering the integrated reasoning stage for the first time, it is necessary to pre-initialize the available few-sample dataset S c Initialize and construct the category centroid groups of ensemble members 1 and ensemble members 2.
[0089] For ensemble member 1, the class centroid group r c The initialization construction algorithm is as follows:
[0090]
[0091] Where |S c | is the sample size Shot of the small sample data set. is the label of sample j. B represents the ViT model of CLIP, R represents the ROIAlign method, and b sp Represents the target box coordinates in the sample label. Formula (2) shows that the centroid of this group of categories is built based on vector u. Vector u is obtained by intercepting the activation map obtained after the image sample x is input into B according to the coordinates of b using the ROIAlign method.
[0092] For ensemble member 2, the class centroid group p c The initialization construction algorithm is as follows:
[0093]
[0094] Among them, |S c | is the sample size Shot of the small sample data set. is the label of sample j. Represents the original pre-trained DETR-like object detection model without transfer adaptation. N H Indicates the number of stages in the Decoder layer of the target detection model. sp Represents the target box coordinates in the sample label. Formula (4) shows that the group of category centroids is constructed based on vector v. Vector v is the image sample x input into S and its target box of each stage is fixed to the true value b sp The Query vector of the final Decoder layer is obtained.
[0095] During ensemble reasoning, for the reasoning sample x, vector u and vector v are obtained as shown in formula (2) and formula (4), and the cosine similarity between them and the category centroid is calculated to obtain the confidence s Im and The confidence of the target category in the result obtained by the target detection model after the transfer attention mechanism for the sample x is integrated and enhanced. Specifically, it is shown in formula (5):
[0096]
[0097] α+β+γ=1; (5)
[0098] It should be noted that the integration coefficients of different target detection models are different, but the method of integrating and aggregating the results of this stage is consistent. For different DETR-like target detection models, adjustments and configurations are made based on the verification accuracy of the integrated reasoning.
[0099] In order to further clarify the division of plasticity modules and stability modules, we Figure 7 Another typical partitioning scheme of the attention mechanism target detection model DINO is shown in Figure 2. Figure 7The figure shows the model structure diagram of the typical attention mechanism target detection model DINO. In the figure, LP corresponds to the plastic fine-tuning transfer warm-up stage, FR-FT corresponds to the steady-state fine-tuning transfer adaptation stage, and Ensembling corresponds to the integrated reasoning stage. The orange part in the figure represents the relevant components divided into plastic modules in the decoding Decoder layer module of each level in the DINO model. The blue part in the figure represents other components in the decoding Decoder layer of each level. Although they also need to be adjusted and adapted for downstream tasks, at the same time, these components need to prevent catastrophic forgetting and loss of pre-trained acquired knowledge during the fine-tuning transfer process. Therefore, they need to be protected for stability and divided into stability modules. In the plastic fine-tuning transfer warm-up stage (LP) and the steady-state fine-tuning transfer adaptation stage (FR-FT), the division of plasticity modules and stability modules is different, which constitutes the asymptotic transfer process of course learning. In the integrated reasoning stage (Ensembling), Figure 7 The serial reasoning relationship between the integrated learning calibrator (Calibrator) and the attention mechanism target detection model (Detector) is demonstrated. The target detection model infers the input to generate output results, and the input calibrator corrects and improves the output results of the target detection model.
[0100] Based on the above method, a small sample target detection device in this embodiment includes: at least one memory and at least one processor;
[0101] The at least one memory is used to store a machine-readable program;
[0102] The at least one processor is used to call the machine-readable program to execute a small sample target detection method.
[0103] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.
[0104] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A small sample target detection method, characterized in that: Reinitialize the parameters of the vector dimension reduction classifier in the cascade decoder layer of the attention mechanism target detection model. The classifier parameters are divided into a plasticity module and a stability module. A migration strategy for balancing and coordinating a plasticity module and a stability module, wherein the stability module includes a backbone network portion and a portion of components in a cascade decoder layer, and the plasticity module includes another portion of components in a cascade decoder layer; Through the Ensembling integrated learning mechanism, the generalization robustness of the attention mechanism target detection model after fine-tuning and migration is improved.
2. A small sample target detection method according to claim 1, characterized in that: There are two generalization robust improvement members in the ensemble learning mechanism, namely ensemble member 1 and ensemble member 2. Ensemble member 1 is a CLIP ViT model pre-trained with a large amount of Web data. After pre-training with Internet big data, the member has strong generalization robustness of category discrimination ability; The ensemble member 2 is an initial attention mechanism target detection model obtained by pre-training. This member does not perform a few-sample fine-tuning transfer adaptation for downstream new tasks, and retains the generalization robustness of the target positioning capability that is complementary to the model after transfer adaptation.
3. A small sample target detection method according to claim 2, characterized in that: The attention mechanism target detection model obtains the migration and adaptation model after the plastic fine-tuning migration warm-up stage and the steady-state fine-tuning migration adaptation stage. During the reasoning stage, the migration and adaptation model enters the integrated reasoning configuration state Ensembling to output the reasoning results under generalized robust integration improvement.
4. A small sample target detection method according to claim 3, characterized in that: When the plastic fine-tuning migration warm-up phase is started, the attention mechanism target detection model is initialized and loaded from the parameter image obtained in the pre-training phase. Plastic fine-tuning refers to the fine-tuning phase in which the migration strategy gives priority to fully meeting the plasticity requirements of the plastic modules involved in the migration under the complete freezing protection of the stability module.
5. A small sample target detection method according to claim 4, characterized in that: The measures taken in the plastic fine-tuning migration preheating stage are: (1) No freezing is performed on the plasticity module, giving it maximum plasticity; (2) Completely freeze the stability module to provide maximum protection; (3) Extend the training schedule to provide sufficient migration warm-up.
6. A small sample target detection method according to claim 5, characterized in that: Steady-state fine-tuning refers to the fine-tuning stage in which the parameters involved in migration adjustment are expanded on the basis of gradually inheriting the fine-tuning migration warm-up parameters in the previous stage, and the design of the migration strategy is considered to meet the needs of moderate fine-tuning migration under stability constraints; When the steady-state fine-tuning adaptation transfer phase starts, the attention mechanism target detection model is initialized and loaded from the parameter image obtained in the plastic fine-tuning transfer warm-up phase.
7. A small sample target detection method according to claim 6, characterized in that: In the steady-state fine-tuning migration adaptation stage, a migration adaptation strategy with relatively harmonious and balanced plasticity and stability is given to the plastic and stability modules. The measures are as follows: (1) The plasticity module is not frozen; (2) The stability module partially freezes the underlying backbone network to provide appropriate protection; (3) Appropriately shorten the training schedule to prevent the pre-training knowledge from being forgotten.
8. A small sample target detection method according to claim 7, characterized in that: In the integrated reasoning stage, the centroid is initialized first. When entering the integrated reasoning stage for the first time, it is necessary to initialize the centroid based on the available few-sample dataset S in advance. c Initialize and construct the category centroid groups of integration members 1 and 2; For ensemble member 2, the class centroid group p c The initialization construction algorithm is as follows: Among them, |S c | is the sample size Shot of the small sample data set, is the label of sample j, B represents the ViT model of CLIP, R represents the ROIAlign method, and b sp represents the target box coordinates in the sample label; Formula (2) shows that the centroid of this group of categories is constructed based on the vector u, which is obtained by intercepting the activation map obtained after the image sample x is input into B according to the coordinates of b using the ROIAlign method; For ensemble member 2, the class centroid group p c The initialization construction algorithm is as follows: Among them, |S c | is the sample size Shot of the small sample data set, is the label of sample j, N represents the original pre-trained DETR target detection model without migration adaptation. H Indicates the number of stages in the Decoder layer of the target detection model, b sp Represents the target box coordinates in the sample label; Formula (4) shows that the centroid of this group of categories is constructed based on the vector v, which is the image sample x input into S and the target box of each stage is fixed to the true value b sp The Query vector of the final Decoder layer is obtained.
9. A small sample target detection method according to claim 8, characterized in that: During ensemble reasoning, for the reasoning sample x, vector u and vector v are obtained as shown in formula (2) and formula (4), and the cosine similarity between them and the category centroid is calculated to obtain the confidence s Im and The target category confidence in the result obtained by the transfer attention mechanism target detection model for sample x reasoning is integrated and enhanced, as shown in the following formula: The integration coefficients of different target detection models are different, but they are consistent with the method of integrating and aggregating the results of this stage. For different DETR-like target detection models, they are adjusted and configured according to the verification accuracy of the integrated reasoning.
10. A small sample target detection device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 9.