Model-level incremental small sample infrared motion small target detection method

Abstract

The invention discloses a model-level incremental small sample infrared motion small target detection method. The method comprises the following steps: S1, constructing a model-level incremental small sample learning framework and an optical flow auxiliary detector; the model-level increment small sample learning framework comprises a task discriminator, a parameter selection strategy module and a distribution alignment prompt generator, and the optical flow auxiliary detector comprises a spatial-temporal feature extraction unit, an attention fusion unit and a detection head unit; s2, selecting a public infrared motion small target data set for task training; s3, training a basic task and an incremental task; s4, inputting a to-be-detected infrared image sequence into the model; matching an optimal optical flow auxiliary detector with a distribution alignment prompt generator through a task discriminator module; and inputting the image sequence after the key frame superposition prompt into an optical flow auxiliary detector. According to the invention, efficient and accurate infrared motion small target detection can be realized in a limited sample increment scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a model-level incremental small sample infrared moving small target detection method. BACKGROUND

[0002] Infrared moving small target detection is a research direction with great practical value and challenges in the field of computer vision, and has an irreplaceable application in key scenes such as military detection and remote sensing monitoring, so it has attracted more and more attention. However, the existing mainstream detection methods generally rely on static data sets containing sufficient and complete samples, and face many real bottlenecks in actual application. On the one hand, the cost of infrared imaging equipment is high, and the imaging conditions are harsh, which makes it extremely difficult to obtain a large number of high-quality infrared images, and new data needs to be continuously processed by the detector; on the other hand, the data in some application scenarios has a one-time use characteristic and cannot be retained for subsequent joint training, which requires the detector to adapt to new data while maintaining the detection performance of the original data.

[0003] In view of the above problems, researchers have carried out exploration around incremental small sample learning, trying to learn new tasks under limited data while reducing the catastrophic forgetting of previous task knowledge, and gradually introducing related methods into the target detection field. According to the differences in technical ideas, the existing incremental detection schemes can be divided into three categories as shown in (a)~(c): Figure 1

[0004] (a) Parameter fine-tuning scheme, which alleviates forgetting by imposing parameter constraints during incremental training, but as the number of training iterations increases, the cumulative parameter changes will cause the forgetting problem to become more serious;

[0005] (b) Parameter decoupling scheme, which separates model parameters into shared and task-specific parts, but still cannot avoid forgetting, and limited trainable parameters may lead to insufficient acquisition of subsequent task knowledge;

[0006] (c) Model internal parameter expansion scheme, which introduces additional trainable parameters for new tasks and freezes the original parameters, which can avoid forgetting to some extent, but task-specific parameters may interfere with each other during inference, and the shared backbone network is difficult to adapt to the differences of different task scenes in infrared small target detection, and cannot realize the cooperative optimization of all modules.

[0007] ​The infrared small target itself has the characteristics of limited and scattered effective information, so that the detection performance is extremely sensitive to parameter changes, and even a slight parameter adjustment can cause a significant decline in performance, which makes the traditional incremental detection method difficult to apply in the task of infrared small moving target detection. In addition, most of the existing methods do not fully consider the problems of cross-task data distribution difference, difficulty in feature extraction under a small amount of samples, etc. in infrared small target detection, lack of an incremental learning framework specially designed for this task, resulting in a significant decline in detection performance in the special scene of insufficient samples or dynamic increments. In summary, the existing scheme has obvious defects: first, it is difficult to balance the adaptation of new tasks and the retention of old task knowledge in incremental training, and is prone to catastrophic forgetting; second, the feature extraction capability for infrared small targets is insufficient, and the performance is poor in the scene of a small amount of samples; third, the adaptation problem caused by the distribution difference of cross-task data is not effectively solved, and there is no targeted optimization for the characteristics of the infrared small target detection task. SUMMARY

[0008] The purpose of the present application is to overcome the shortcomings of the prior art and provide a model-level incremental small sample infrared moving small target detection method. The present application relies on a publicly shared basic detection model and a pre-sequence task special model, identifies the task scene of the input image through a lightweight task discriminator, generates a distribution alignment prompt for the current task under the supervision of the pre-sequence task discriminator, realizes the incremental training of the new detection model by combining the parameter selection strategy, and almost does not produce catastrophic forgetting, finally realizes efficient and accurate infrared moving small target detection in the limited sample incremental scene.

[0009] The purpose of the present application is to overcome the shortcomings of the prior art and provide a model-level incremental small sample infrared moving small target detection method. The present application relies on a publicly shared basic detection model and a pre-sequence task special model, identifies the task scene of the input image through a lightweight task discriminator, generates a distribution alignment prompt for the current task under the supervision of the pre-sequence task discriminator, realizes the incremental training of the new detection model by combining the parameter selection strategy, and almost does not produce catastrophic forgetting, finally realizes efficient and accurate infrared moving small target detection in the limited sample incremental scene.

[0010] S1, constructing a model-level incremental small sample learning framework and an optical flow auxiliary detector;

[0011] The model-level incremental small sample learning framework includes a task discriminator, a parameter selection strategy module, and a distribution alignment prompt generator;

[0012] The task is divided into a basic task with sufficient samples and a single scene, and an incremental task with different scenes and limited samples and gradually added to the training; in the incremental task, the incremental task being trained or inferred is referred to as the current task, and the previous incremental task earlier than the current task in the training sequence is referred to as the pre-sequence task; a corresponding task discriminator, distribution alignment prompt generator and optical flow auxiliary detector are established for each basic task and incremental task respectively;

[0013] The task discriminator includes three convolutional layers and a fully connected layer connected in turn;

[0014] The parameter selection strategy module is configured to initialize parameters of the current optical flow assisted detector: by comparing the parameter difference of the optical flow assisted detector of the previous task and the base task, setting a threshold to screen general feature parameters, and initializing the optical flow assisted detector of the current task by using the general feature parameters;

[0015] The distribution alignment hint generator is configured to generate a distribution alignment hint by combining the optical flow image and the key frame feature; the specific implementation method is: respectively extracting the features of the key frame and the optical flow image by a 1x1 upsampling convolution module, and summing the two features, and the summed features are optimized by a normal convolution module and a 1x1 downsampling convolution module to generate the distribution alignment hint ;

[0016] The optical flow assisted detector comprises a space-time feature extraction unit, an attention fusion unit and a detection head unit.

[0017] The space-time feature extraction unit extracts spatial features by using a CSPDarkNet; and the time features are obtained by calculating the optical flow by using a Farneback algorithm.

[0018] The attention fusion unit is fused by using an attention mechanism.

[0019] The detection head unit realizes target positioning and classification by using a YOLOX.

[0020] S2, selecting a public infrared moving small target data set for task training, dividing the infrared image sequence in the data set into a base task data set and an incremental task data set according to the task scene, using complete sample data in the base task data set for base task training, and using three kinds of small sample data in the incremental task data set for incremental task training.

[0021] S3, training the task discriminator and the optical flow assisted detector of the base task by using the base task data set, training the task discriminator and the optical flow assisted detector of the incremental task by using the incremental data set; for each incremental task, initializing the optical flow assisted detector by using the parameter selection strategy module, and supervising the training by combining the distribution alignment hint generator and the task discriminator of the previous task.

[0022] S4, inputting the infrared image sequence to be detected into the model; matching the optimal optical flow assisted detector and the distribution alignment hint generator by using the task discriminator module; and inputting the image sequence after superimposing the key frame hint into the optical flow assisted detector.

[0023] The beneficial effects of the present application are: the present application first proposes a model-level incremental small sample learning framework for infrared moving small target detection, which almost completely avoids catastrophic forgetting through model-level expansion instead of parameter adjustment, solves the parameter sensitivity problem of traditional incremental methods in infrared small target detection; secondly, the combination of light weight task discriminator and distribution alignment prompt realizes efficient adaptation across task scenes, and improves the feature learning ability under small sample conditions; the parameter selection strategy effectively reuses the general knowledge of the previous task, while avoiding task-specific interference, laying a solid foundation for incremental training; the OFAD designed specially enhances target information through spatiotemporal feature fusion, further improving the detection performance. Experimental verification shows that the method performs excellent detection accuracy and incremental adaptability on DAUB, ITSDT and IRDST datasets, with a forgetting rate of only 7.30%, providing an efficient solution for infrared moving small target detection under insufficient sample or dynamic incremental scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 Figure 1 is a comparison chart of the incremental scheme of the present application and other different incremental schemes; wherein (a) is a parameter fine-tuning scheme, (b) is a parameter decoupling scheme, (c) is a model internal parameter expansion scheme, (d) is the model-level expansion scheme proposed by the present application, the same color represents the same parameter, and "+" represents parameter increment.

[0025] Figure 2 Figure 2 is a flowchart of a model-level incremental small sample infrared moving small target detection method of the present application.

[0026] Figure 3 Figure 3 is the MIFL framework architecture and workflow chart proposed by the present application; it shows the training logic of each task detector, discriminator and prompt generator in the training stage, as well as the complete process of input matching, prompt adjustment and detection of the discriminator in the inference stage.

[0027] Figure 4 Figure 4 is a task discriminator structure chart designed by the present application; it shows the process of the discriminator predicting the task feature vector through the convolutional network, and calculating the loss with the real statistical feature vector of the input frame to complete the optimization.

[0028] Figure 5 Figure 5 is a distribution alignment prompt generator structure chart of the present application; it shows the specific process of the generator generating distribution alignment prompts by combining optical flow images and key frame features through convolutional modules.

[0029] Figure 6 Figure 6 is an optical flow assisted detector (OFAD) structure chart of the present application; the red border and line represent the key frame and key frame input, and show the overall architecture of spatiotemporal feature extraction, fusion and detection head.

[0030] Figure 7 The figure is a visual comparison result of the present application and the existing incremental detection method. It respectively shows the detection effect of the present application and the DFP, G-FSOD and GMDP methods on the IRDST, DAUB and ITSDT three data sets. The red box is the model output, the green box is the real target, the blue box is the missed detection target, and the yellow box is the false detection target. DETAILED DESCRIPTION

[0031] The present application proposes a model-level incremental small sample learning framework for infrared moving small target detection by model-level expansion instead of parameter adjustment. Figure 1 The technical solutions of the present application are further described below in conjunction with the drawings.

[0032] As shown in Figure 2 The present application provides a model-level incremental small sample infrared moving small target detection method, which includes the following steps:

[0033] S1, an intelligent detection system specially used for infrared moving small target detection is constructed, including a model-level incremental small sample learning framework (MIFL) and an optical flow auxiliary detector (OFAD); the structures and functions of each core module are as follows:

[0034] (1) The model-level incremental small sample learning framework as the core learning framework includes a lightweight task discriminator, a parameter selection strategy module and a distribution alignment prompt generator.

[0035] The present application supports multiple task settings. The tasks are divided into basic tasks with sufficient samples and single scenes, and incremental tasks with different scenes, limited samples and gradually added training. In the incremental task, the incremental task being trained or inferred is referred to as the current task, and the previous incremental task earlier than the current task in the training sequence is referred to as the previous task. A corresponding task discriminator, distribution alignment prompt generator and optical flow auxiliary detector are established for each basic task and incremental task.

[0036] (1-1) In the task discriminator, in order to accurately identify the task scene of the input image and quickly match the corresponding special detector, a lightweight task discriminator is designed as shown in Figure 4 The task discriminator fully utilizes the statistical feature potential of the input frame and the advantages of the simple network structure, extracts a global feature vector composed of mean, variance and information entropy, and combines a three-layer convolution and a linear simplified network architecture to reduce the computational complexity to the maximum while ensuring the accuracy of task scene differentiation, thereby providing support for fast task matching in the inference stage.

[0037] The task discriminator includes three convolution layers and a fully connected layer connected in turn. After the image is input into the discriminator, features are extracted through three convolution operations ,Will The input layer is a fully connected layer, and the final output is the predicted feature vector. The calculation process is as follows:

[0038] (1);

[0039] in, Indicates input, Indicates a convolutional layer. This indicates a fully connected layer. Each convolutional layer also includes a corresponding ReLU activation layer and a max pooling layer.

[0040] The task discriminator is trained independently of the optical flow-assisted detector; during the training phase of the task discriminator, the goal is to minimize the predicted task feature vector of keyframes. With the true statistical feature vector of the keyframe of Loss is the optimization objective, and the loss function is... Represented as:

[0041] (2);

[0042] Here, the keyframe is the last frame of the input image; the true statistical feature vector of the keyframe This vector is composed of the mean of the keyframes. ,variance and information entropy Composed of spliced ​​features, this feature vector can effectively distinguish the differences in global distribution and local structure in different infrared mission scenarios. , The predictions of the task discriminator are respectively The three components represent its three dimensions, which are used to learn the mean, variance, and information entropy.

[0043] (1-2) The parameter selection strategy module is used to initialize the parameters of the current optical flow-assisted detector, fully reuse the general knowledge of the previous task and avoid overfitting interference.

[0044] The parameter selection strategy module compares the parameter differences between the optical flow-assisted detectors of the preceding task and the basic task, sets a threshold to filter common feature parameters, and uses the common feature parameters to initialize the optical flow-assisted detector of the current task.

[0045] The specific implementation method is as follows: Set up an optical flow-assisted detector for the basic task. The parameters are Optical flow-assisted detector for preceding tasks The parameters are The optical flow-assisted detector for the current task The parameters are , For the parameter index of the optical flow-assisted detector, The current task number; the threshold is set to... Parameter-by-parameter comparison and Difference: If the difference is less than Then it is believed We learned cross-task general features and used them to initialize... If the difference is greater than or equal to Then it is believed By adapting to the preceding tasks, use initialization The specific logic is as follows:

[0046] (3);

[0047] in This represents the total number of optical flow-assisted detectors.

[0048] (1-3) The distribution alignment cue generator, under the supervision of the task discriminator of the preceding task, combines optical flow images and keyframe features to generate distribution alignment cuees, adjusting the distribution of input data to better fit the parameter space of the optical flow-assisted detector of the current task. Its structure is as follows: Figure 5 As shown.

[0049] The specific implementation method is as follows: Input keyframes With optical flow image (Calculated by the Farneback algorithm), each passed through a 1×1 up-dimensional convolution module. Extract keyframes With optical flow image The features are analyzed and summed, and the summed features are then passed through a regular convolutional module. And a 1×1 dimensionality reduction convolution module Optimization to generate distribution alignment prompts The calculation process can be represented as follows:

[0050] (4);

[0051] During the training phase, keyframes with cues will be added. Input the preceding task discriminator to obtain the predicted feature vector. ,by and of loss function To optimize the objective, a distribution alignment cue generator is trained; the loss function is:

[0052] (5).

[0053] (2) The optical flow auxiliary detector (OFAD) is used as a special detection module to accurately extract the spatiotemporal features of the infrared small target and complete the detection. The OFAD includes a spatiotemporal feature extraction unit, an attention fusion unit, and a detection head unit, and has a structure as shown in FIG. 2. Figure 6

[0054] (2-1) First, spatial feature extraction is performed: the distribution alignment prompt is added to the key frame of the original image sequence, and then the input sequence is composed with other original images ( The new key frame after adding the original key frame and the prompt); the CSPDarkNet is used to extract the multi-scale feature sequence from the input sequence , and the spatial feature is obtained after the convolution layer fusion ;

[0055] (6);

[0056] represents the CSPDarkNet;

[0057] Temporal feature extraction: the temporal feature is obtained by calculating the optical flow through the Farneback algorithm: the optical flow of the key frame and other frames is calculated and summed to obtain the optical flow image , and then the temporal feature is obtained through the convolution layer processing:

[0058] (7);

[0059] wherein is the Farneback optical flow algorithm, the pyramid scaling factor is set to 0.5, and the window size of each layer is 15 pixels.

[0060] (2-2) Finally, spatiotemporal feature fusion is performed: the attention fusion unit adopts the attention mechanism; the attention weights of the spatial feature and the temporal feature are calculated through the attention module, the two features are fused, the interference is suppressed, and the target information is enhanced; the fused feature and the spatial feature extracted from the last frame of the image are subjected to attention fusion, the fused feature is processed through the fully connected layer, then the spatial feature extracted from the last frame of the image is added element by element, and then the obtained feature is subjected to layer normalization operation, further fusing the key frame feature extracted by the backbone network; the specific fusion method can be represented as:

[0061] (8);

[0062] wherein​ is the input feature dimension, represents the last frame of the image processed by the CSPDarkNet is the extracted spatial feature, represents the output feature fused with the spatial feature and the temporal feature, represents the output feature of the attention fusion unit, is the layer normalization, is the full connection layer.

[0063] (2-3) The detection head unit adopts YOLOX to realize target positioning and classification;

[0064] The total loss function of the optical flow assisted detector (OFAD) contains loss , classification loss and target loss , which is expressed as:

[0065] (9)

[0066] , , are the weights of loss , classification loss and target loss , respectively; wherein and are both binary cross-entropy losses.

[0067] S2, select a public infrared moving small target dataset for task training, divide the infrared image sequence in the dataset into a basic task dataset and an incremental task dataset according to the task scene, use complete sample data in the basic task dataset for basic task training, and use three kinds of small sample data in the incremental task dataset for incremental task training; and construct a mixed test set for model performance evaluation.

[0068] In the implementation, three public infrared moving small target datasets, IRDST, DAUB and ITSDT, are selected, which correspond to three typical scenes of ground-to-air, air-to-ground and military vehicle detection, respectively, covering different background complexity and target scale range.

[0069] IRDST is used as the basic task dataset, which contains 92 video clips with a total of 40657 frames, and is divided into a training set and a test set in a ratio of 5:5. Complete samples are used for training to ensure that the basic detector learns general infrared small target detection knowledge.

[0070] DAUB and ITSDT are used as incremental task datasets, with 4-shot, 8-shot, and 16-shot small sample settings. For each setting, a corresponding number of video clips (each containing 4 consecutive images) are randomly selected as the training set, and the remaining samples are used as the incremental task test set.

[0071] From the test set of each of the three datasets, 50 video clips (each containing 4 frames, a total of 2000 frames) are randomly selected to form a mixed test set for comprehensive evaluation of the incremental detection performance and knowledge retention ability of the model in a multi-task scenario.

[0072] The images of all datasets are uniformly resized to 512x512 pixels to maintain pixel scale consistency and avoid affecting model training and inference due to input size differences.

[0073] S3, the task discriminators and optical flow assisted detectors for the base task are trained using the base task dataset, and the task discriminators and optical flow assisted detectors for the incremental task are trained using the incremental data dataset. Each incremental task uses a phased training strategy, combining parameter selection, supervised training, and multi-index evaluation to iteratively optimize model performance. For each incremental task, the optical flow assisted detector is initialized using the parameter selection strategy module, and the task discriminators for the previous tasks are supervised trained using the distribution alignment prompt generator. The network parameters are iteratively optimized using the SGD optimizer and the CosineAnnealingLR scheduler, and the mixed test set is used to evaluate the performance of the model. 、 The optimal model is selected based on the value and forgetting rate.

[0074] In this embodiment, the IRDST complete training set is used to train the base task. During the training process, the detector uses the OFAD architecture to extract spatio-temporal features and complete target detection, and the task discriminator is independently optimized to learn the statistical features of the base task scenario, ensuring that the base model has excellent general detection capability.

[0075] For each incremental task, the current task detector is initialized based on the parameter selection strategy, and the differences between the previous task detector and the base detector are compared parameter by parameter. The threshold is set to 15%, and the universal feature parameters are selected for initialization to avoid interference caused by over-fitting of the previous task.

[0076] The distribution alignment prompt generator for the current task is trained using the task discriminator of the previous task as supervision. The distribution alignment prompt is generated to adapt to the parameter space of the current optical flow assisted detector. The input features after adjusting the prompt are combined with the small sample training data to jointly train the current detector. The parameters of the base detector and the previous task detector are fixed to avoid catastrophic forgetting.

[0077] ​The total number of model training epochs is set to 20, the batch size is 4, the SGD optimizer is used (with a momentum of 0.937), the initial learning rate is 0.001, the learning rate is dynamically adjusted by the CosineAnnealingLR scheduler, and the loss function weight is set to , , , balancing the classification, regression, and target loss in the detection task.

[0078] During training, the model parameters are saved every round, and the (IoU threshold is set to 0.5), value (harmonic mean of precision and recall) and forgetting rate (evaluating knowledge retention ability) are calculated on the mixed test set. The comprehensive performance of the three indicators is selected as the final detection model, and the forgetting rate is calculated as follows:

[0079] (10);

[0080] wherein, is the task ID, is the number of samples of the task in the mixed test set, is the total number of samples in the mixed test set, is the overall of the mixed test set, is the of the task.

[0081] The forgetting rate is calculated based on the of each task and the overall of the mixed test set, and is weighted by the corresponding sample size. It directly reflects the model's ability to retain old task knowledge.

[0082] S4, input the infrared image sequence to be detected into the model; match the optimal optical flow assisted detector and distribution alignment prompt generator through the task discriminator module; input the image sequence after superimposing the key frame prompt into the optical flow assisted detector.

[0083] Frame extraction is performed on the infrared image sequence to be detected, and the key frame is determined as the last frame of the sequence, and the real statistical feature vector of the key frame is calculated.

[0084] Input the key frame into all task discriminators in the matching pool, and each discriminator will output the corresponding predicted feature vector . By calculating each and the real statistical feature vector The Euclidean distance of the input sequence is calculated, the task discriminator with the minimum distance is selected as the optimal task discriminator, and the task scene to which the input sequence belongs is determined.

[0085] According to the matching result of the optimal task discriminator, the corresponding optical flow auxiliary detector is called from the detection pool The distribution alignment prompt generator , ensure that the detection model is accurately adapted to the current task scene, and lay a foundation for subsequent efficient detection.

[0086] The distribution alignment prompt generator generates a distribution alignment prompt , the distribution alignment prompt is superimposed on the key frame through element-wise addition, the distribution state of the input data is adjusted, and it is better adapted to the parameter space of the current task detector. The key frame superimposed with the distribution alignment prompt and other frames in the image sequence form a new image sequence, and the new image sequence is input into the optical flow auxiliary detector to output the target bounding box and the corresponding confidence, and the final detection result is obtained.

[0087] Non-maximum suppression (NMS) processing is applied to the prediction box output by the optical flow auxiliary detector, the IoU threshold is set to 0.65, only the prediction result with confidence not less than 0.001 is retained, and the prediction box with low confidence is removed, and finally the accurate infrared moving small target detection result is output.

[0088] Those skilled in the art should know Precision is a very important indicator in target detection, which is used to measure the average precision of the model when the IoU (intersection over union) threshold is 0.5.

[0089] The present application takes the disclosed infrared image sequence dataset DAUB, IRDST and ITSDT as the experimental object, and further verifies the effectiveness and universality of the method.

[0090] The DAUB dataset contains 17 video clips, of which the training set consists of 10 clips with a total of 8982 frames, the test set contains 7 clips with a total of 4795 frames, the target bounding box size is about 50 pixels, and the air-to-ground detection scene is selected; The IRDST dataset covers 92 video clips, totaling 40657 frames, which are divided into training set and test set in the ratio of 5:5, the target bounding box is about 100 pixels, the background is mainly sky, and is suitable for ground-to-air detection task; The ITSDT dataset contains 60 video clips with a total of 15000 frames, 10000 frames of 40 clips are selected as the training set, and 5000 frames of 20 clips are selected as the test set, the target bounding box size is between 100 and 300 pixels, the scene includes complex environments such as roads and military bases, and there are target occlusion, overlap and other challenges.

[0091] (1) Experimental initialization configuration

[0092] All images in both the training and test sets were resized to 512×512 pixels to ensure consistent input scale. Model training was configured for 20 epochs with a batch size of 4, using the SGD optimizer with a momentum of 0.937 and an initial learning rate of 0.001, dynamically adjusted using the CosineAnnealingLR scheduler. The loss function weights were configured as follows: , , These correspond to IOU loss, classification loss, and target loss, respectively. The threshold in the parameter selection strategy... The IoU threshold for nonmaximum suppression (NMS) during inference was set to 15%, with an IoU threshold of 0.65, retaining only predicted boxes with a confidence level higher than 0.001. All experiments were conducted on a hardware environment equipped with dual NVIDIA GeForce RTX 4090 GPUs.

[0093] (2) Basic task training phase

[0094] The basic detector is trained using the full training set of IRDST as the basic task data. With the corresponding lightweight task discriminator The detector employs an OFAD architecture, extracting multi-frame spatial features through CSPDarkNet and combining them with temporal features calculated using the Farneback optical flow algorithm. These features are then fused through attention and input into the detection head to complete target detection. The discriminator is trained independently, learning the statistical features of the basic task scene to support scene matching for subsequent incremental tasks.

[0095] (3) Incremental task training phase

[0096] For the incremental tasks corresponding to DAUB and ITSDT, three small sample settings—4-shot, 8-shot, and 16-shot—were used, respectively. A parameter selection strategy was employed to initialize the current task detector, comparing the differences between the previous task detector and the base detector parameter by parameter, and selecting general feature parameters to avoid overfitting interference. Using the previous task discriminator as supervision, a distribution-aligned cue generator was trained to generate cue information adapted to the parameter space of the current detector, adjusting the input data distribution to improve small sample learning performance. During training, the parameters of the base detector and the previous task detector were fixed, and only the detector, discriminator, and cue generator for the current task were optimized to avoid catastrophic forgetting.

[0097] (4) Feature adaptation and detection optimization stage

[0098] The distribution alignment cue generator combines keyframes and optical flow images to generate cues. , the input data is better adapted to the detector parameter space by element-wise addition and key frame superposition. The OFAD detector performs spatio-temporal fusion on the adapted features, enhances target information and suppresses background interference, and outputs the target bounding box and confidence through the detection head reasoning. During the training process, the prompt generation loss and detection loss are calculated, and the model parameters are iteratively optimized using the backpropagation gradient descent algorithm to improve detection accuracy.

[0099] (5) Model evaluation and optimal selection phase

[0100] During the training process, the model parameters are saved every round, and the , values and forgetting rate are calculated on the mixed test set. The forgetting rate is based on the of each task and the overall of the mixed test set, combined with the corresponding sample size weighted calculation, to comprehensively evaluate the incremental detection performance and knowledge retention ability of the model. Finally, the model with the best performance of the three indicators is selected as the final detection model.

[0101] To verify the effectiveness and superiority of the method (MIFL on OFAD) in the task of infrared moving small target detection, we carried out comparative experiments on three public infrared sequence datasets DAUB, IRDST and ITSDT, and compared their performance with other mainstream detection methods to fully verify the competitiveness of the invention under different technical routes. Experiment 1 is set as follows: Task 1 (Task1) is a benchmark task based on the IRDST dataset for full sample training; Task 2 (Task2) and Task 3 (Task3) are 4-shot tasks based on the DAUB and ITSDT datasets, respectively. It should be noted that in the comparison method, "Inf." represents a method specifically for infrared moving small target detection, "Few." represents a method specifically for small sample detection, and "Inc." represents a method specifically for incremental detection. In the experimental results, the optimal result is marked in bold, and the specific experimental results are shown in Table 1:

[0102] Table 1 Comparison of the effects of the method of the invention and traditional methods in Experiment 1

[0103]

[0104] Experiment 2 is set as follows: Task 1 (Task1) is a benchmark task based on the IRDST dataset for full sample training; Task 2 (Task2) and Task 3 (Task3) are 8-shot tasks based on the DAUB and ITSDT datasets, respectively, and the specific experimental results are shown in Table 2:

[0105] Table 2 Comparison of the effects of the method of the present application and the traditional method in Experiment 2

[0106]

[0107] Experiment 3 is set as follows: Task 1 (Task1) is a benchmark task based on the full sample training of IRDST dataset; Task 2 (Task2) and Task 3 (Task3) are 16-shot tasks based on the training of DAUB and ITSDT datasets respectively, and the specific experimental results are shown in Table 3:

[0108] Table 3 Comparison of the effects of the method of the present application and the traditional method in Experiment 3

[0109]

[0110] By comparing the results of Table 1, Table 2 and Table 3, three obvious findings can be observed:

[0111] 1) The method of the present application (MIFL on OFAD) is always significantly better than all other comparison methods in terms of For example, in Task 1 (IRDST full sample task) of Experiment 3, the of the method of the present application reaches 74.31%, which is equal to DFP on OFAD and leads 68.87% of ST-FSOD and other methods; in Task 1 of Experiment 2, the of the method of the present application is also 74.31%, which is far higher than 50.19% of Incrc-DETR and other comparison methods;

[0112] 2) In terms of of different tasks, the method of the present application is in the front row. For example, in Task 1 (IRDST full sample task) of Experiment 1, the of the method of the present application is 86.85%, which is equal to 86.85% of DFP on OFAD and higher than other methods; in Task 3 of Experiment 2, the of the method of the present application is 52.30%, which is the highest result among all comparison methods, exceeding 42.64% of DFP on OFAD;

[0113] 3) For each index of Mixed Task, the method of the present application is always better than all comparison methods. For example, in Mixed Task of Experiment 1, the of the method of the present application is 38.47%, which is better than 38.33% of the second highest DFP on OFAD; in Mixed Task of Experiment 3, the The value is 69.43%, which also exceeds the second highest DFP on OFAD of 65.17%, indicating that the method has a strong advantage in the comprehensive performance of complex mixed tasks.

[0114] In addition, in order to evaluate the influence of different modules in the application on the infrared moving small target detection performance and small sample learning ability, the IRDST dataset is selected as the training data of benchmark task 1 (Task1), and the DAUB and ITSDT datasets are selected as the training data of 8-shot task 2 (Task2) and task 3 (Task3), respectively, to carry out ablation experiments. Among them, OFA represents the optical flow auxiliary module, PT represents the strategy of loading the parameters of the previous task model as a pre-trained model, PS represents the parameter selector module, and DAP represents the distribution alignment prompt module. In the experimental results, the optimal result is marked in bold, and the specific experimental results are shown in Table 4:

[0115] Table 4 Effect comparison of different modules on the performance of small sample in ablation experiments

[0116]

[0117] By comparing the results in Table 4, three obvious findings can be observed:

[0118] 1) The introduction of the optical flow auxiliary module can significantly improve the infrared moving small target detection ability of the detector, increasing the benchmark task from 67.57% to 74.31%, and the from 81.08% to 86.85%.

[0119] 2) The parameter selection strategy of the application performs better than directly using the previous task parameters for initialization. For example, if the previous task parameters are directly used as a pre-trained model, the of Task2 is 16.31%; after using the parameter selection strategy of the application for initialization, the index is improved to 16.74%.

[0120] 3) With the help of distribution alignment prompts, the performance of small sample tasks is significantly improved. For example, the of Task2 is improved from 16.74% to 30.65%, from 40.08% to 50.84%.

[0121] ​In addition to numerical performance comparison, in order to more intuitively compare the performance of the method of the present application with that of the existing incremental detection method, this embodiment selects DFP, G-FSOD and GMDP, three representative methods, to carry out visual comparison experiments. In the experiment, we set IRDST as the benchmark task, DAUB and ITSDT as 16-shot tasks, train each method on these three tasks in turn, and select a sequence from each task to visualize the detection results of the key frames. Figure 7 The figure for visual comparison of the present application with the existing incremental detection method is shown in FIG. 1. It shows the detection results of the present application and the DFP, G-FSOD and GMDP methods on the IRDST, DAUB and ITSDT data sets. Among them, the red small box is the detection model output box, the green box is the correct detection target, the blue box is the missed detection target, and the yellow box is the false detection target. Some results are shown in FIG. 1. Figure 7

[0122] By visually comparing the results in FIG. 1, two obvious findings can be observed: Figure 7

[0123] 1) In the detection results of the IRDST task, the red output box of the method of the present application (Ours) completely matches the green correct box, there is no blue missed detection box and yellow false detection box, and the detection box is highly consistent with the target shape and closely surrounds the target. The detection results of DFP and G-FSOD tasks also do not have missed detection and false detection, but the fit degree of the output box is slightly inferior to that of the method of the present application. This result is not only due to the simple background and no occlusion of the sample, but also reflects the detection accuracy advantage of the method of the present application.

[0124] 2) In the DAUB task, GMDP has the problems of yellow false detection box (detecting non-existent target) and insufficient matching degree of green box, while the red output box of the method of the present application completely coincides with the green correct box without missed detection and false detection. In the ITSDT task, G-FSOD and GMDP both have blue missed detection box (not detecting real target) and low overlap degree of output box and green correct box, while the red box of the method of the present application can accurately match all green correct boxes without missed detection and false detection. This shows that in complex scenes, the detection accuracy and robustness of the method of the present application for infrared moving small targets are better.

[0125] In summary, the method of the present application has higher accuracy and robustness in infrared moving small target detection, and can effectively detect small targets appearing in the sequence images of infrared moving small targets collected by infrared imaging devices. This makes the method have wide application prospects in important application fields such as unmanned aerial vehicle supervision, security and military detection.

[0126] ​​Those skilled in the art will appreciate that the embodiments described herein are presented for purposes of illustration and that the inventive principles are not limited to these particular embodiments. Other variations and modifications can be made to the embodiments without departing from the spirit and scope of the inventive principles.

Claims

1. A model-level incremental small sample infrared moving small target detection method, characterized in that, The method comprises the following steps: S1, constructing a model-level incremental small sample learning framework and an optical flow auxiliary detector; The model-level incremental small sample learning framework comprises a task discriminator, a parameter selection strategy module, and a distribution alignment prompt generator; The tasks are divided into basic tasks with sufficient samples and single scenes, and incremental tasks with different scenes, limited samples, and step-by-step training; in the incremental tasks, the incremental task being trained or inferred is referred to as a current task, and a previous incremental task earlier than the current task in the training sequence is referred to as a previous task; a corresponding task discriminator, distribution alignment prompt generator, and optical flow auxiliary detector are established for each basic task and incremental task; The task discriminator comprises three convolutional layers and a fully connected layer connected in sequence; The parameter selection strategy module is used to initialize the parameters of the current optical flow auxiliary detector: by comparing the parameter differences of the optical flow auxiliary detectors of the previous task and the basic task, setting a threshold to screen general feature parameters, and using the general feature parameters to initialize the optical flow auxiliary detector of the current task; The distribution alignment hint generator is used to generate a distribution alignment hint in combination with the optical flow image and the key frame feature, and a specific implementation method is as follows: the key frame and the feature of the optical flow image are respectively extracted through a 1*1 dimension increasing convolution module, and the two features are summed, the summed feature is optimized through a common convolution module and a 1*1 dimension decreasing convolution module, and the distribution alignment hint is generated . The optical flow auxiliary detector comprises a spatio-temporal feature extraction unit, an attention fusion unit, and a detection head unit; The spatio-temporal feature extraction unit extracts spatial features by using a CSPDarkNet; and time features are obtained by calculating optical flow by using a Farneback algorithm; The attention fusion unit uses an attention mechanism for fusion; The detection head unit realizes target positioning and classification by using a YOLOX; S2, selecting a public infrared moving small target dataset for task training, dividing the infrared image sequence in the dataset into a basic task dataset and an incremental task dataset according to the task scenes, using complete sample data in the basic task dataset for basic task training, and using three kinds of small sample data in the incremental task dataset for incremental task training; S3, training the task discriminators and optical flow auxiliary detectors of the basic tasks by using the basic task dataset, and training the task discriminators and optical flow auxiliary detectors of the incremental tasks by using the incremental dataset; for each incremental task, the optical flow auxiliary detector is initialized by using the parameter selection strategy module, and supervised training is performed in combination with the distribution alignment prompt generator and the task discriminator of the previous task; S4, inputting a to-be-detected infrared image sequence into the model; matching the optimal optical flow auxiliary detector and the distribution alignment prompt generator by using the task discriminator module; Inputting the image sequence after superimposing the key frame prompt into the optical flow auxiliary detector.

2. The model-level incremental small sample infrared moving small target detection method according to claim 1, characterized in that, The specific implementation method of the parameter selection strategy module is as follows: Assume the optical flow-assisted detector of the basic task... The parameters are Optical flow-assisted detector for preceding tasks The parameters are The optical flow-assisted detector for the current task The parameters are , For the parameter index of the optical flow-assisted detector, The current task number; the threshold is set to... Parameter-by-parameter comparison and Difference: If the difference is less than Then it is believed We learned cross-task general features and used them to initialize... If the difference is greater than or equal to Then it is believed By adapting to the preceding tasks, use initialization The specific logic is as follows: （3）； wherein is the total number of optical flow assisted detectors.

3. The model-level incremental small sample infrared moving small target detection method according to claim 1, characterized in that, The step S4 specifically realizes the method as follows: frame extraction is performed on the infrared image sequence to be detected, a key frame is determined as the last frame of the sequence, and a real statistical feature vector of the key frame is calculated ; The key frame is input into all task discriminators in a matching pool, and each discriminator outputs a corresponding predicted feature vector ; by calculating the Euclidean distance between each and the real statistical feature vector , the task discriminator with the smallest distance is screened as the optimal task discriminator, so as to determine the task scene to which the input sequence belongs The distribution alignment prompt generator generates a distribution alignment prompt, superimposes the distribution alignment prompt on the key frame, and inputs the new image sequence composed of the key frame after superimposing the distribution alignment prompt and other frames in the image sequence into the optical flow auxiliary detector, to output a target bounding box and a corresponding confidence.

Images