An online learning method for airborne target detection with sparse parameter update

Abstract

The application discloses an online learning method for airborne target detection with sparse parameter updating, which comprises the following steps: firstly, using the prepared base class data set and label data, a deep learning target detection model is supervised trained; at the same time, using an airborne sensor, a new class data set corresponding to the target is actually shot; then, using the fine-tuning data set, the model is fine-tuned and trained online on the airborne end; after fine-tuning, the model is used to infer the fine-tuning data set again, and according to the confidence, the data set is screened again. After the training screening reaches the number of rounds, the fine-tuning is stopped, and the target detection model is output and updated. The method can realize online updating and recognition of the airborne target detection model, realize parameter training and fine-tuning of the model in the memory and display memory limited scene, and further improve the unsupervised target detection precision and reliability of the model.

Description

Technical Field

[0001] This invention belongs to the field of unmanned platform technology, specifically relating to an online learning method for airborne target detection with sparse parameter updates. Background Technology

[0002] Currently, unmanned platforms are widely used in military missions such as surveillance and reconnaissance, as well as in civilian fields such as land surveying and natural disaster prediction. At present, deep learning-based models are mainly used for autonomous target reconnaissance and identification in UAV reconnaissance missions. However, deep learning models require extensive pre-training with large amounts of data and can only identify specific target categories during reconnaissance, failing to recognize unknown target types. Furthermore, current model parameter update training for unknown target types can only be performed on offline platforms such as ground servers, which is insufficient to meet the rapid and urgent needs of reconnaissance missions. Therefore, the low adaptability and slow update efficiency of airborne target detection models are key challenges that urgently need to be addressed. In conclusion, there is an urgent need for an online learning algorithm that can achieve parameter updates for airborne target detection models. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention provides an online learning method for airborne target detection with sparse parameter updates. First, a deep learning target detection model is trained in a supervised manner using prepared base class datasets and label data. Simultaneously, new class datasets corresponding to the target are generated using actual images captured by airborne sensors. Then, the model is trained and fine-tuned online on the airborne platform using a fine-tuning dataset. The fine-tuned model then re-infers from the fine-tuned dataset, and the dataset is further filtered based on confidence levels. After a certain number of training and filtering rounds, fine-tuning stops, and the target detection model is output and updated. This invention enables online updating and recognition of airborne target detection models, achieving parameter training and fine-tuning even in memory- and GPU-constrained scenarios, further improving the unsupervised target detection accuracy and reliability.

[0004] The technical solution adopted by this invention to solve its technical problem is as follows:

[0005] Step 1: Collect a public dataset as the base dataset;

[0006] Step 2: Use integrated UAV-borne sensors to obtain infrared video data containing targets that do not exist in the base class, and decompose it into single-frame images to form a new class dataset; the base class refers to vehicles and people;

[0007] Step 3: Build an object detection model and train the object detection model parameters using the base class dataset;

[0008] Step 4: Perform graph optimization on the object detection model to improve its inference efficiency;

[0009] Step 5: Use the optimized object detection model to infer the new class dataset, obtain the object labels, and construct a fine-tuning dataset for model fine-tuning based on the detection confidence.

[0010] Step 6: Using the fine-tuning dataset, perform sparse training on the object detection model on the embedded processor;

[0011] Step 7: After sparse training is completed, the sparse trained model is obtained. The inference is performed again on the new class dataset. The results of the inference are used to update the fine-tuning dataset. Then, the sparse trained model is fine-tuned repeatedly until the set repetition threshold is reached, and then the fine-tuned model is output.

[0012] Preferably, the target detection model is a YOLOv8 neural network model.

[0013] Preferably, the sparse training is specifically as follows:

[0014] Step 6-1: Before deploying the object detection model, find the optimal parameter positions through the mAP optimization process; assuming that the convolution parameter w of the i-th layer in the model is adjusted individually, the resulting mAP change is: ΔmAP wi Similarly, by adjusting the bias parameter b of the i-th layer alone, the change in target detection mAP is: ΔmAP bi Based on the current peak memory constraint Mmax of the airborne embedded module, the following formula is optimized:

[0015]

[0016] stMemory(K,I,R)≤Mmax

[0017] Where I is the layer index of the weights, K is the layer index of the biases, and R is the ratio of learnable weights; K * ,I * ,R * These represent the optimal weight layer index, bias layer index, and learnable weight ratio after the search, respectively.

[0018] Step 6-2: Based on the found parameter optimization positions, mark these positions in advance, construct a training optimization graph, set the marked positions to 1 to represent parameter updates, and set other positions to false to represent no parameter updates during the parameter tuning process;

[0019] Step 6-3: Update parameters on the airborne end;

[0020] Based on the optimization graph settings, parameter updates are divided into three levels: first, updating the bias b; second, updating the detection head of the target detection model; and finally, updating some feature layer parameters of the feature extractor to address the model's generalization ability to targets of different scales.

[0021] All three levels of updates are performed using backward propagation, and the three updates are performed simultaneously.

[0022] Step 6-4: After constructing the backward propagation computation chain, the computation process is implemented in the embedded processor using the loss function, and the backward propagation parameters are calculated and updated.

[0023] Step 6-5: The updated parameters are used to update the optimization graph. During the parameter update process, parameters whose loss decreases below the set threshold are remarked as False, so that they no longer participate in the backward propagation calculation, thus reducing memory usage.

[0024] Step 6-6: After the update iteration is completed, evaluate the updated model using some test data.

[0025] Preferably, the loss function for model fine-tuning is as follows:

[0026] By employing a combined loss function, the transfer model is guided to learn the optimal parameters in each data domain:

[0027] The combined loss function includes the object detection loss for each data domain. The training weights between the two datasets are set as hyperparameters, preset with empirical values ​​before training and added as hyperparameters in the model's backpropagation during training. Regularization constraints are applied to the hyperparameters. The object loss for each data domain consists of a weighted location loss, a class loss, and a confidence loss.

[0028] L self =λ box L box +λ cls L cls +L obj +D λ (λ box ,λ cls )

[0029] Where λ box and λ cls These represent the position loss weights and class loss weights, respectively, and are also set as trainable hyperparameters that are updated during the backpropagation of model parameters.

[0030] By using a combined loss function to constrain parameter updates and effectively fusing multiple loss functions, the model is ultimately guided to learn the optimal parameters in various data domains.

[0031] The beneficial effects of this invention are as follows:

[0032] The method of this invention enables online updating and recognition of airborne target detection models. By utilizing sparse backpropagation capability, it achieves parameter training and fine-tuning of the model in scenarios with limited memory and GPU memory, thereby improving the generalization detection accuracy of the model in target datasets. Finally, by employing progressive self-learning technology and using new class datasets for self-training, it further improves the unsupervised target detection accuracy and reliability of the model. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the airborne target detection online learning process for sparse parameter updating as involved in the present invention.

[0034] Figure 2 This is a schematic diagram of the sparse parameter update process involved in the present invention;

[0035] Figure 3 This is a schematic diagram of the self-learning online learning model involved in the present invention. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] To effectively improve the ability of target detection models to adapt to new scenarios and targets on airborne platforms, this invention proposes an online learning method for airborne target detection with sparse parameter updates. This method introduces a sparse parameter backpropagation module to improve the efficiency of parameter fine-tuning and detection generalization of the model.

[0038] The technical method solution of the present invention includes the following steps:

[0039] Step 1: Collect a public dataset as the base dataset;

[0040] Step 2: Use UAV-borne sensors that integrate infrared and optical sensors to obtain infrared video data of targets such as pedestrians and cars that do not exist in the base class, and decompose them into single-frame images; thus forming a new class dataset;

[0041] Step 3: Build an object detection model and train the model parameters using the base class dataset;

[0042] Step 4: Perform graph optimization on the model to improve its inference efficiency;

[0043] Step 5: Use the optimized model to infer the new class dataset, obtain the target label, and construct a fine-tuning dataset for model fine-tuning based on the detection confidence.

[0044] Step 6: Use the fine-tuning dataset to perform sparse training on the model on the embedded processor;

[0045] Step 7: After the sparse training is completed, inference is performed again on the new class dataset. The results of the inference are used to update the fine-tuning dataset. Then, the sparse model is fine-tuned repeatedly until a certain repetition threshold is reached, and then the fine-tuned model is output.

[0046] Example:

[0047] Figure 1 The overall learning structure for airborne target detection with sparse parameter updates, as disclosed in this invention, firstly involves supervised training of a deep learning target detection model using prepared base class datasets and label data. Simultaneously, airborne sensors are used to generate new class datasets corresponding to the targets, where the target types in the base class data and the new class data differ. The new class dataset is then selected to construct a fine-tuning dataset. The specific selection process is as follows: First, the trained target detection model is used to infer the new class dataset, identifying the detected unknown categories. Proposal detection boxes are then sorted according to confidence levels, and the top k detection boxes are obtained based on a threshold. These top k detection boxes are clustered, and target boxes not belonging to the largest category are removed. A small amount of unknown slice data is transmitted to the ground via a drone channel. After manual judgment, the unknown targets are reclassified, and the reclassified category information is re-uploaded to the airborne terminal via a wireless channel. After class reclassification, the fine-tuning dataset is constructed.

[0048] The model was then trained and fine-tuned online on an in-flight environment using the fine-tuning dataset. After fine-tuning, the model re-inferred the fine-tuning dataset, and the dataset was further refined based on confidence levels. After a certain number of training and refinement rounds, fine-tuning was stopped, and the object detection model was output and updated.

[0049] Figure 2 The process of updating sparse parameters is described below:

[0050] Sparse inverse parameter propagation first optimizes the positions of the parameters to be tuned in the model. The optimization steps are as follows.

[0051] Before deploying the model, the optimal parameter positions are found through the mAP optimization process. Assuming that the convolution parameter w of the i-th layer in the model is adjusted individually, the resulting mAP change is: ΔmaP wi Similarly, by adjusting the bias parameter b of the i-th layer alone, the change in target detection mAP is: ΔmAP bi Based on the current peak memory constraint Mmax of the airborne embedded module, the following formula is optimized.

[0052]

[0053] stMemory(K,I,R)≤Mmax

[0054] Where I is the layer index of the weights, K is the layer index of the biases, and R is the ratio of learnable weights; K * ,I * ,R * These represent the optimal weight layer index, bias layer index, and learnable weight ratio after the search, respectively. The optimization objective is to find the best update configuration, where the total mAP contribution is maximized while memory usage does not exceed hardware constraints. The accuracy contribution of each mAP change can be summarized, thus the problem can be effectively solved through evolutionary search.

[0055] Based on the identified parameter optimization locations, these locations are marked in advance to construct a training optimization graph. The marked locations are set to 1 to indicate that parameter updates will be performed, while other locations are set to false to indicate that parameter updates will not be performed during the parameter tuning process.

[0056] Parameter updates are performed on the airborne end. Based on the optimization graph settings, parameter updates are mainly divided into three levels: First, updating the bias b. Compared to the weight w, the backward propagation of the bias b involves fewer computations, making parameter updates more efficient. Second, updating the detection head of the object detection model, only updating the classification layer of the detection head in the model, thereby improving the model's ability to identify unknown categories. Finally, updating some feature layer parameters of the feature extractor to address the model's generalization ability to targets of different scales.

[0057] All three updates are performed using traditional backward propagation, and are carried out simultaneously, with flexible configuration based on the hardware capabilities of the airborne sensors.

[0058] After constructing the backward propagation computation chain, the computation process is then implemented in the embedded processor using a loss function, completing the backward propagation parameter calculation and update.

[0059] The updated parameters also update the optimization graph. Parameters that show a small decrease in loss during the update process are remarked as False, so they no longer participate in the backward propagation calculation, thus reducing memory usage.

[0060] After the update iteration is completed, the updated model is evaluated using a small amount of test data to ensure the correctness of the update process.

[0061] Figure 3This is a flowchart of the self-learning fine-tuning process proposed in this invention. The fine-tuned model is used again to predict the fine-tuned dataset, generating labels and confidence scores. The dataset is then further filtered using the labels and confidence scores to remove erroneous and unrecognized samples. Since there are feature differences between the base class and new class datasets under unsupervised conditions, a mechanism for fusing and balancing loss functions needs to be designed. A combined loss function for the new and base classes is designed to comprehensively evaluate the model's detection and transfer performance, ensuring that the updated model achieves similar results for both the new and base classes. Furthermore, during training on each dataset, the target location loss, class loss, and confidence loss need to be further optimized. For self-learning multiple loss functions, this invention proposes a combined loss function to guide the transfer model to learn the optimal parameters in each data domain.

[0062] The combined loss function includes object detection losses from each data domain. The training weights between the two datasets are set as hyperparameters, preset with empirical values ​​before training and added as hyperparameters in the model's backpropagation during training. Regularization constraints are applied to the hyperparameters to prevent them from becoming too large after weight updates. The target loss for each data domain consists of a weighted location loss, a class loss, and a confidence loss.

[0063] L self =λ box L box +λ cls L cls +L obj +D λ (λ box ,λ cls )

[0064] Where λ box and λ cls These represent the position loss weights and class loss weights, respectively. They are also set as trainable hyperparameters and updated during the backward propagation of model parameters. D... λ (λ box ,λ cls The ) item is used to ensure that the weight is not too large.

[0065] By using a combined loss function to constrain parameter updates and effectively fusing multiple loss functions, the model is ultimately guided to learn the optimal parameters in each data domain, enabling the network to achieve balanced performance in both the target and source domains.

Claims

1. An online learning method for airborne target detection with sparse parameter updates, characterized in that, Includes the following steps: Step 1: Collect a public dataset as the base dataset; Step 2: Use integrated UAV-borne sensors to obtain infrared video data containing targets that do not exist in the base class, and decompose it into single-frame images to form a new class dataset; the base class refers to vehicles and people; Step 3: Build an object detection model and train the object detection model parameters using the base class dataset; Step 4: Perform graph optimization on the object detection model to improve its inference efficiency; Step 5: Use the optimized object detection model to infer the new class dataset, obtain the object labels, and construct a fine-tuning dataset for model fine-tuning based on the detection confidence. Step 6: Using the fine-tuning dataset, perform sparse training on the object detection model on the embedded processor; The sparse training is described in the following details: Step 6-1: Before deploying the object detection model, find the optimal parameter positions through the mAP optimization process; assuming that the convolution parameter w of the i-th layer in the model is adjusted individually, the resulting mAP change is as follows: Similarly, by adjusting the bias parameter b of the i-th layer alone, the change in target detection mAP is as follows: Based on the current peak memory constraint Mmax of the airborne embedded module, the following formula is optimized: Where I is the layer index of the weights, K is the layer index of the biases, and R is the ratio of learnable weights. These represent the optimal weight layer index, bias layer index, and learnable weight ratio after the search, respectively. Step 6-2: Based on the found parameter optimization positions, mark these positions in advance, construct a training optimization graph, set the marked positions to 1 to represent parameter updates, and set other positions to false to represent no parameter updates during the parameter tuning process; Step 6-3: Update parameters on the airborne end; Based on the optimization graph settings, parameter updates are divided into three levels: first, updating the bias b; second, updating the detection head of the target detection model; and finally, updating some feature layer parameters of the feature extractor to address the model's generalization ability to targets of different scales. All three levels of updates are performed using backward propagation, and the three updates are performed simultaneously. Step 6-4: After constructing the backward propagation computation chain, the computation process is implemented in the embedded processor using the loss function, and the backward propagation parameters are calculated and updated. Step 6-5: The updated parameters are used to update the optimization graph. During the parameter update process, parameters whose loss decreases below the set threshold are remarked as False, so that they no longer participate in the backward propagation calculation, thus reducing memory usage. Step 6-6: After the update iteration is completed, evaluate the updated model using some test data; Step 7: After sparse training is completed, the sparse trained model is obtained. The inference is performed again on the new class dataset. The results of the inference are used to update the fine-tuning dataset. Then, the sparse trained model is fine-tuned repeatedly until the set repetition threshold is reached, and then the fine-tuned model is output.

2. The online learning method for airborne target detection with sparse parameter updating according to claim 1, characterized in that, The target detection model is a YOLOV8 neural network model.

3. The online learning method for airborne target detection with sparse parameter updating according to claim 1, characterized in that, The loss function for model fine-tuning is as follows: By employing a combined loss function, the transfer model is guided to learn the optimal parameters in each data domain: The combined loss function includes the object detection loss for each data domain. The training weights between the two datasets are set as hyperparameters, preset with empirical values ​​before training and added as hyperparameters in the model's backpropagation during training. Regularization constraints are applied to the hyperparameters. The object loss for each data domain consists of a weighted location loss, a class loss, and a confidence loss. in These represent the position loss weights and class loss weights, respectively, and are also set as trainable hyperparameters that are updated during the backpropagation of model parameters. By using a combined loss function to constrain parameter updates and effectively fusing multiple loss functions, the model is ultimately guided to learn the optimal parameters in various data domains.

Images