Remote sensing target detection sample sampling method and device based on active learning
By generating labeled and unlabeled datasets and using pre-trained networks for feature extraction and Mean Shift clustering, the problem of low sample sampling efficiency in remote sensing target detection is solved, achieving efficient sample acquisition and model training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2024-05-21
- Publication Date
- 2026-05-19
AI Technical Summary
Existing remote sensing target detection sample collection methods based on active learning suffer from high time costs, difficult model training, and low sample acquisition efficiency.
By acquiring unlabeled remote sensing samples, labeled and unlabeled datasets are generated. Feature extraction is performed using a pre-trained object detection network. The uncertainty of the dataset is evaluated based on feature difference mechanism and sample consistency mechanism. Sample clustering is performed using Mean Shift clustering algorithm to obtain high-quality sampled samples.
It effectively improves the limitations of a single evaluation method, increases the efficiency of mining high-quality samples in unlabeled datasets, reduces the difficulty and time cost of model training, and improves the efficiency of sample acquisition.
Smart Images
Figure CN118537537B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, specifically relating to a remote sensing target detection sample sampling method and apparatus based on active learning. Background Technology
[0002] In recent years, with the continuous development of artificial intelligence technology, remote sensing target detection and recognition technology based on deep learning has gradually attracted more attention. It is currently widely used in many fields such as resource exploration, environmental disaster detection, and military reconnaissance, having a significant impact on national economy, defense, and social development. However, commonly used target detection methods are all based on deep learning, which requires a large amount of manually labeled data. In real-world scenarios, acquiring much of this data is very costly, and the manual cost of labeling data is a considerable expense. Therefore, how to utilize limited data to improve the accuracy and performance of remote sensing target detection has become a research hotspot.
[0003] Active learning is a semi-supervised learning method. Its core idea is to use a small number of highly informative unlabeled samples to evaluate unlabeled data on a labeled dataset, thereby assisting in the learning of supervised tasks. Compared to randomly selecting samples, active learning only requires a small number of labeled samples to achieve considerable model performance. Therefore, using active learning for object detection data sampling has become a mainstream approach.
[0004] For example, some scholars have proposed a "Key Data Screening Method for UAV Target Detection Based on Active Learning" (Application No.: 202410252652.8, Publication No.: CN117876910A, Publication Date: 2024.04.12), which improves the efficiency and reliability of UAV target detection models. However, this method requires training multiple target detection models and constructing a model committee composed of different models, resulting in high time costs and low efficiency in practical applications. Other scholars have proposed an "Active Learning Framework with Both Uncertainty and Representativeness" (Application No.: 202410018254.X, Publication No.: CN117852617A, Publication Date: 2024.04.09), which solves the cold start problem of active learning. However, this framework uses the K-nearest neighbor algorithm to calculate outliers, resulting in low computational efficiency and sensitivity to hyperparameters, making it unsuitable for direct application in real-world scenarios. Some scholars have also proposed an "uncertainty-guided adversarial active learning target detection method" (Computer Measurement & Control: 1-19 [2024-04-26], Publication date: 2024.3.23). However, this method requires the additional construction of an adversarial network for sample evaluation, resulting in high model complexity and training difficulty, and it cannot be adapted to different detection networks.
[0005] Therefore, existing sample sampling methods based on active learning suffer from problems such as high time cost, difficulty in model training, and low sample acquisition efficiency. Summary of the Invention
[0006] To address the aforementioned problems in the existing technology, this invention provides a remote sensing target detection sample sampling method and apparatus based on active learning.
[0007] The technical problem to be solved by this invention is achieved through the following technical solution:
[0008] In a first aspect, the present invention provides a remote sensing target detection sample sampling method based on active learning, comprising:
[0009] Unlabeled remote sensing samples are acquired, and a first dataset and a second dataset are generated based on the unlabeled remote sensing samples; the first dataset is a labeled dataset; the second dataset is an unlabeled dataset.
[0010] The initial object detection network is trained using the first dataset to obtain a pre-trained object detection network;
[0011] The first and second datasets were extracted using a pre-trained object detection network, resulting in multiple first-level multi-layer features and multiple second-level multi-layer features.
[0012] Based on the feature difference mechanism, the differences between multiple first-level and multiple second-level features are judged to obtain difference distance information;
[0013] Stability results were obtained based on the sample consistency mechanism and the second dataset;
[0014] The difference distance information and stability results are weighted and summed to obtain the uncertainty results of the second dataset;
[0015] Obtain the target categories and the number of target categories in the first dataset, and then determine the category sampling weights based on the target categories and the number of target categories.
[0016] A sampling score is generated for each second sample in the second dataset based on the category sampling weights and the uncertainty results;
[0017] The second sample was clustered using the Mean Shift clustering algorithm to obtain clusters.
[0018] In each cluster, the second sample whose sampling score meets the preset requirements is used as the target remote sensing sampling sample.
[0019] Optionally, unlabeled remote sensing samples are obtained, and a first dataset and a second dataset are generated based on the unlabeled remote sensing samples, including:
[0020] Acquire unlabeled remote sensing samples;
[0021] A portion of unlabeled remote sensing samples were randomly selected from the unlabeled remote sensing samples as initial data, and the initial data after data labeling was used as the first dataset.
[0022] The unlabeled remote sensing samples other than the initial data are used as the second dataset.
[0023] Optionally, a pre-trained object detection network is used to extract features from the first dataset and the second dataset, respectively, resulting in multiple first multi-level features and multiple second multi-level features, including:
[0024] The first and second datasets are respectively input into the feature extraction layer of the pre-trained object detection network for feature extraction.
[0025] For each first sample in the first dataset, multiple first multi-level features are obtained;
[0026] For each second sample in the second dataset, multiple second multi-level features are obtained.
[0027] Optionally, based on a feature difference mechanism, difference judgment is performed on multiple first-level features and multiple second-level features to obtain difference distance information, including:
[0028] The average value of multiple first-level features is calculated to obtain the first mean feature;
[0029] The average value of multiple second-level features is calculated to obtain the second mean feature;
[0030] The difference between the first mean feature and the second mean feature is calculated using Euclidean distance to obtain the difference distance information.
[0031] Optionally, stability results are obtained based on a sample consistency mechanism and a second dataset, including:
[0032] The second dataset is augmented based on the sample consistency mechanism to obtain the second augmented dataset.
[0033] The stability of the second augmented dataset and the second dataset is assessed, and the stability results are obtained.
[0034] Optionally, stability is assessed on the second augmented dataset and the second dataset to obtain stability results, including:
[0035] The second augmented dataset is input into the pre-trained object detection network to obtain multiple first detection results, and the information entropy of the multiple first detection results is calculated to obtain the first information entropy data.
[0036] The second dataset is input into the pre-trained object detection network to obtain multiple second detection results, and the information entropy of the multiple second detection results is calculated to obtain the second information entropy data.
[0037] The variance between the first information entropy data and the second information entropy data is calculated to obtain the stability result.
[0038] Optionally, obtain the target categories and the number of target categories in the first dataset, and obtain the category sampling weights based on the target categories and the number of target categories, including:
[0039] Obtain the target categories and the number of target categories in the first dataset;
[0040] Sum the number of target categories to get the total number of categories;
[0041] Divide the number of target categories by the total number of categories to obtain the category sampling weights corresponding to the target categories.
[0042] Optionally, a sampling score is generated for each second sample in the second dataset based on the category sampling weights and the uncertainty result, including:
[0043] Input the second augmented dataset into the pre-trained object detection network to obtain the first detection result;
[0044] Based on the first detection result, find the corresponding target detection weight coefficient in the category sampling weight;
[0045] Input the target detection weight coefficient and uncertainty result into the sampling score formula to calculate the sampling score;
[0046] The sampling score formula is expressed as:
[0047]
[0048] S x U represents the sampling score. x W represents an uncertain outcome. y N represents the target detection weight coefficient corresponding to the target category y. CL This indicates the total number of target categories included in the first detection result.
[0049] Optionally, within each cluster, the second sample whose sampling score meets a preset requirement is used as the target remote sensing sampling sample, including:
[0050] Within each cluster, the sampling scores are sorted by size to obtain the sampling ranking score;
[0051] The second sample whose sampling ranking score is greater than a preset threshold is used as the target remote sensing sampling sample.
[0052] Secondly, the present invention provides a remote sensing target detection sample sampling device based on active learning, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the remote sensing target detection sample sampling device based on active learning is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the remote sensing target detection sample sampling method based on active learning described in the first aspect above.
[0053] This invention provides a remote sensing target detection sample sampling method and apparatus based on active learning. One method for sampling remote sensing target detection samples based on active learning includes: acquiring unlabeled remote sensing samples; generating a first dataset and a second dataset based on the unlabeled remote sensing samples; the first dataset is a labeled dataset; the second dataset is an unlabeled dataset; training an initial target detection network using the first dataset to obtain a pre-trained target detection network; extracting features from the first dataset and the second dataset using the pre-trained target detection network to obtain multiple first multi-level features and multiple second multi-level features; performing difference judgment on the multiple first multi-level features and multiple second multi-level features based on a feature difference mechanism to obtain difference distance information; obtaining stability results based on a sample consistency mechanism and the second dataset; performing weighted summation processing on the difference distance information and stability results to obtain uncertainty results for the second dataset; acquiring the target categories and the number of target categories in the first dataset, and obtaining category sampling weights based on the target categories and the number of target categories; generating a sampling score for each second sample in the second dataset based on the category sampling weights and uncertainty results; clustering the second samples using the Mean Shift clustering algorithm to obtain clusters; and using the second samples in each cluster whose sampling scores meet preset requirements as target remote sensing sampling samples. In this invention, by employing a feature difference mechanism and a sample consistency mechanism to jointly evaluate the uncertainty results of unlabeled datasets, the limitations of a single evaluation method are effectively improved, and the mining of high-quality second samples in unlabeled datasets is achieved. Secondly, by integrating the MeanShift clustering algorithm, the clustering speed is greatly improved, thereby improving the sample acquisition efficiency. Finally, because the model structure of this invention is simple, the training of additional modules is avoided, reducing the difficulty and time cost of model training.
[0054] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0055] Figure 1 A flowchart illustrating a remote sensing target detection sample sampling method based on active learning, provided in an embodiment of the present invention;
[0056] Figure 2 A schematic diagram of the overall process of the remote sensing target detection sample sampling method based on active learning provided in an embodiment of the present invention;
[0057] Figure 3 The category distribution of randomly selected samples and samples sampled using the method of the present invention is provided for embodiments of the present invention;
[0058] Figure 4 This is a schematic diagram of a remote sensing target detection sample sampling device based on active learning provided in an embodiment of the present invention. Detailed Implementation
[0059] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.
[0060] To achieve the mining of high-quality second samples in unlabeled datasets while reducing the difficulty and time cost of model training, this invention provides a remote sensing target detection sample sampling method based on active learning. Figure 1 This is a flowchart illustrating a remote sensing target detection sample sampling method based on active learning, provided as an embodiment of the present invention. Figure 1 As shown, it includes:
[0061] S101. Obtain unlabeled remote sensing samples, and generate the first dataset and the second dataset based on the unlabeled remote sensing samples.
[0062] The first dataset is a labeled dataset; the second dataset is an unlabeled dataset.
[0063] Optionally, step S101 may specifically include:
[0064] Acquire unlabeled remote sensing samples;
[0065] A portion of unlabeled remote sensing samples were randomly selected from the unlabeled remote sensing samples as initial data, and the initial data after data labeling was used as the first dataset.
[0066] The unlabeled remote sensing samples other than the initial data are used as the second dataset.
[0067] In addition, the first dataset consists of a small subset of randomly selected samples from unlabeled remote sensing data, which have then been labeled. The second dataset consists of unlabeled remote sensing samples other than the initial dataset.
[0068] S102. Train the initial object detection network using the first dataset to obtain the pre-trained object detection network.
[0069] It should be noted that, in the embodiments of the present invention, the initial target detection network can be a TOLOv9 network.
[0070] S103. Use a pre-trained object detection network to extract features from the first dataset and the second dataset respectively, and obtain multiple first multi-level features and multiple second multi-level features.
[0071] It should be noted that each first sample in the first dataset corresponds to multiple first multi-level features, and each second sample in the second dataset corresponds to multiple second multi-level features.
[0072] Specifically, the current second sample from the second dataset is input into a pre-trained object detection network for feature extraction. For each feature extraction layer of the pre-trained object detection network, multiple second-level features are output. Similarly, the current first sample from the first dataset is input into a pre-trained object detection network for feature extraction. For each feature extraction layer of the pre-trained object detection network, multiple first-level features are output.
[0073] Optionally, step S103 may specifically include:
[0074] The first and second datasets are respectively input into the feature extraction layer of the pre-trained object detection network for feature extraction.
[0075] For each first sample in the first dataset, multiple first multi-level features are obtained;
[0076] For each second sample in the second dataset, multiple second multi-level features are obtained.
[0077] It should be noted that both the first and second multi-level features are feature outputs of the High Efficiency Layer Aggregation Network (GELAN) module in the pre-trained object detection network.
[0078] For example, in this embodiment of the invention, the feature vectors of the last three layers of GELAN can be taken as the first multi-level feature vector. This includes: the first multi-level feature vector... First multi-layer feature vector two and the first multi-layer feature vector three The corresponding number of channels can be 128, 256, and 512.
[0079] Alternatively, the feature vectors from the last three layers of GELAN can be used as the second-level multi-layer features. This includes: the second-level feature vector... Second multi-layer feature vector Second and third layer feature vectors The corresponding number of channels can be 128, 256, and 512.
[0080] S104. Based on the feature difference mechanism, perform difference judgment on multiple first-level features and multiple second-level features to obtain difference distance information.
[0081] Optionally, step S104 may specifically include:
[0082] The average value of multiple first-level features is calculated to obtain the first mean feature;
[0083] The average value of multiple second-level features is calculated to obtain the second mean feature;
[0084] The difference between the first mean feature and the second mean feature is calculated using Euclidean distance to obtain the difference distance information.
[0085] The Euclidean distance formula is as follows:
[0086]
[0087] Among them, D px This represents the difference distance information, where i represents the i-th level, j represents the j-th channel of the mean feature, and N i This represents the number of channels for the mean feature of the i-th level. This represents the first mean feature of the j-th channel at the i-th level. This represents the second mean feature of the j-th channel at the i-th level.
[0088] It should be emphasized that in this embodiment, the average of multiple first multi-level features corresponding to the current first sample is calculated to obtain the first mean feature; and the average of multiple second multi-level features corresponding to the current second sample is calculated to obtain the second mean feature.
[0089] Therefore, after this step is performed, each first sample will have a corresponding first mean feature, and each second sample will have a corresponding second mean feature.
[0090] S105. Stability results are obtained based on the sample consistency mechanism and the second dataset.
[0091] Optionally, step S105 may specifically include:
[0092] The second dataset is augmented based on the sample consistency mechanism to obtain the second augmented dataset.
[0093] The stability of the second augmented dataset and the second dataset is assessed, and the stability results are obtained.
[0094] Optionally, stability is assessed on the second augmented dataset and the second dataset to obtain stability results, including:
[0095] The second augmented dataset is input into the pre-trained object detection network to obtain multiple first detection results, and the information entropy of the multiple first detection results is calculated to obtain the first information entropy data.
[0096] The second dataset is input into the pre-trained object detection network to obtain multiple second detection results, and the information entropy of the multiple second detection results is calculated to obtain the second information entropy data.
[0097] The variance between the first information entropy data and the second information entropy data is calculated to obtain the stability result.
[0098] S106. Perform a weighted summation of the difference distance information and stability results to obtain the uncertainty results of the second dataset.
[0099] The formula for calculating uncertain results is as follows:
[0100] U x =α·D Px +β·V Px ;
[0101] Among them, U x D represents the uncertainty outcome of the second sample in the second dataset. Px V represents the difference distance information of the second sample in the second dataset. Px This represents the stability result of the second sample in the second dataset. α and β are both hyperparameters, and their values are specified based on human experience.
[0102] S107. Obtain the target category and the number of target categories in the first dataset, and obtain the category sampling weights based on the target category and the number of target categories.
[0103] It should be noted that, in this embodiment of the invention, by incorporating Mean Shift clustering and a class-balanced sampling mechanism with class sampling weights, the problem of imbalanced sample classes in remote sensing target detection tasks is effectively solved.
[0104] Optionally, step S107 may specifically include:
[0105] Obtain the target categories and the number of target categories in the first dataset;
[0106] Sum the number of target categories to get the total number of categories;
[0107] Divide the number of target categories by the total number of categories to obtain the category sampling weights corresponding to the target categories.
[0108] It should be noted that, in this embodiment, obtaining the target category and the number of target categories in the first dataset can specifically involve determining the types of bounding boxes contained in the first dataset and the number of bounding boxes under each bounding box type.
[0109] S108. Generate the sampling score for each second sample in the second dataset based on the category sampling weights and the uncertainty results.
[0110] Optionally, step S108 may specifically include:
[0111] Input the second augmented dataset into the pre-trained object detection network to obtain the first detection result;
[0112] Based on the first detection result, find the corresponding target detection weight coefficient in the category sampling weight;
[0113] Input the target detection weight coefficient and uncertainty result into the sampling score formula to calculate the sampling score;
[0114] The sampling score formula is expressed as:
[0115]
[0116] S x U represents the sampling score. x W represents an uncertain outcome. y N represents the target detection weight coefficient corresponding to the target category y. CL This indicates the total number of target categories included in the first detection result.
[0117] S109. The second sample is clustered using the Mean Shift clustering algorithm to obtain clusters.
[0118] It should be noted that, in this embodiment, clustering the second sample can specifically involve clustering the second multi-level features corresponding to the second sample to obtain clusters.
[0119] S110. In each cluster, the second sample whose sampling score meets the preset requirements is used as the target remote sensing sampling sample.
[0120] Optionally, step S110 may specifically include:
[0121] Within each cluster, the sampling scores are sorted by size to obtain the sampling ranking score;
[0122] The second sample whose sampling ranking score is greater than a preset threshold is used as the target remote sensing sampling sample.
[0123] Additionally, it should be noted that steps S101-S110 can be repeated until the number of target remote sensing samples reaches the set total requirement, at which point sampling should be stopped. All target remote sensing samples are then submitted to experts for annotation, constructing the final sample data used to train the model.
[0124] This invention provides a remote sensing target detection sample sampling method based on active learning, comprising: acquiring unlabeled remote sensing samples; generating a first dataset and a second dataset based on the unlabeled remote sensing samples; the first dataset being a labeled dataset; and the second dataset being an unlabeled dataset; training an initial target detection network using the first dataset to obtain a pre-trained target detection network; extracting features from the first dataset and the second dataset using the pre-trained target detection network to obtain multiple first multi-level features and multiple second multi-level features respectively; performing difference judgment on the multiple first multi-level features and multiple second multi-level features based on a feature difference mechanism to obtain difference distance information; obtaining stability results based on a sample consistency mechanism and the second dataset; performing weighted summation processing on the difference distance information and the stability results to obtain the uncertainty result of the second dataset; acquiring the target categories and the number of target categories in the first dataset, and obtaining category sampling weights based on the target categories and the number of target categories; generating a sampling score for each second sample in the second dataset based on the category sampling weights and the uncertainty result; performing clustering processing on the second samples based on the Mean Shift clustering algorithm to obtain clusters; and using the second samples whose sampling scores meet preset requirements in each cluster as target remote sensing sampling samples. In this embodiment of the invention, by employing a feature difference mechanism and a sample consistency mechanism to jointly evaluate the uncertainty results of the unlabeled dataset, the limitations of a single evaluation method are effectively improved, and the mining of high-quality second samples in the unlabeled dataset is realized. Secondly, by integrating the Mean Shift clustering algorithm, the clustering speed is greatly improved, thereby improving the sample acquisition efficiency. Finally, since the model structure of this invention is simple, the training of other additional modules is avoided, reducing the difficulty and time cost of model training.
[0125] This embodiment provides an overall description of the remote sensing target detection sample sampling method based on active learning. Figure 2 This is a schematic diagram illustrating the overall process of a remote sensing target detection sample sampling method based on active learning, provided in an embodiment of the present invention. Figure 2As shown, after acquiring a large amount of unlabeled data, the initially labeled data is first removed to obtain the data to be screened. An object detection model is then trained using the initially labeled data. This model is then used to extract features from the data to be screened. Based on the feature extraction results, a consistency judgment is made on the data to be screened to calculate the uncertainty of the samples. Next, based on the sample uncertainty, the feature vector clustering results of the data to be screened, and the category distribution of the data to be screened, category-balanced sampling is performed to evaluate the data value and obtain low-value and high-value data.
[0126] To verify the accuracy of the remote sensing target detection sample sampling method based on active learning provided in this embodiment of the invention, simulation experiments were also conducted.
[0127] 1. Simulation experimental conditions:
[0128] The hardware platform for the simulation experiment is as follows: the processor is an Intel(R) Core(TM) CPU i9-13900K with a main frequency of 5.80GHz, the memory is 64GB, and the GPU is an RTX 4090.
[0129] The software platform for the simulation experiment is: Windows 10 operating system, Python 3.8, PyTorch 1.7.0 and PyCharm 2017.
[0130] Experiments were conducted to validate a remote sensing target detection sample collection method based on active learning for a small-sample UAV remote sensing target recognition task. The dataset used in this experiment was the Visdrone2019 dataset, which contains 10 target classes: pedestrians, stationary people, bicycles, vehicles, vans, trucks, tricycles, awning tricycles, buses, and motorcycles. The training set contains 6471 images, and the validation set contains 548 images. The minimum image resolution is 1360×765×3 pixels, and the maximum resolution is 1920×1080×3 pixels. The image format is JPEG.
[0131] 2. Simulation content and result analysis:
[0132] The simulation experiment compared the method of this invention with three existing technologies: Entropy, CALD, and PPAL.
[0133] The three existing technologies used in the simulation experiment are:
[0134] Entropy refers to the uncertainty sampling method of information entropy proposed in "Entropy-based active learning for object detection with progressive diversity constraint. In Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition, pages 9397-9406, 2022."
[0135] CALD refers to a sampling method based on consistency-based active learning, proposed in "Consistency-based active learning for object detection. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 3951-3960, 2022."
[0136] PPAL refers to a sampling method based on uncertainty and class imbalance proposed in "Plug and play active learning for object detection.arXiv preprint arXiv:2211.11612,2022."
[0137] In the simulation experiment, YOLOv9 was used as the detection model. Initially, 500 images were randomly selected for training the initial model. In subsequent steps, sampling methods were used to select 100, 200, 300, 400, and 500 samples, which were then mixed with the original samples to train the detection model. The precision, recall, and mean average precision (mAP) of the detection models trained using different sampling methods on the validation set were compared.
[0138] The experimental results are shown in Table 1:
[0139] Table 1. Quantitative Analysis of Detection Results of the Invention Method and Existing Technologies in Simulation Experiments
[0140]
[0141] As shown in Table 1, the samples sampled using the method of this invention result in higher precision, recall, and mAP on the validation set compared to other methods. After sampling 500 samples, compared to the best-performing existing PPAL sampling method, precision is improved by 3.5%, recall by 2.7%, and mAP by 3.0%, demonstrating the superiority of the method of this invention under small sample conditions.
[0142] The effects of the present invention will be further described below with reference to simulation diagrams:
[0143] Figure 3 The category distribution of randomly selected samples and samples sampled using the method of the present invention is provided for embodiments of the present invention. Figure 3 Figure (a) shows the category distribution of 100 randomly selected samples and 100 samples sampled using the method of this invention in the simulation experiment of this invention. Figure 3 Figure (b) shows the category distribution of 500 randomly selected samples and 500 samples sampled using the method of this invention in the simulation experiment of this invention. Figure 3 As can be seen, the samples collected by this method can achieve a more balanced distribution of different target categories, effectively improving the imbalance of sample categories in remote sensing target detection.
[0144] The simulation experiments above demonstrate that the active learning-based remote sensing target detection sample sampling method of this invention can effectively solve the problems of small number of training samples from UAV-collected data, high sampling difficulty, and imbalanced sample categories. By selecting key data that contributes to improving the performance of the detection model, the annotation burden and resource consumption are reduced, and the detection model can achieve good detection results with a small amount of data.
[0145] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.
[0146] Based on the same inventive concept, embodiments of the present invention also provide a remote sensing target detection sample sampling device based on active learning. Figure 4This is a schematic diagram of a remote sensing target detection sample sampling device based on active learning provided in an embodiment of the present invention. It includes a processor 710, a storage medium 720, and a bus 730. The storage medium 720 stores machine-readable instructions executable by the processor 710. When the remote sensing target detection sample sampling device based on active learning is running, the processor 710 communicates with the storage medium 720 via the bus 730. The processor 710 executes the machine-readable instructions to perform the steps of the aforementioned remote sensing target detection sample sampling method based on active learning. The specific implementation and technical effects are similar and will not be described in detail here.
[0147] The storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the storage medium may also be at least one storage device located remotely from the aforementioned processor.
[0148] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0149] The present invention also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of any of the above-described active learning-based remote sensing target detection sample sampling methods.
[0150] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention.
[0151] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.
[0152] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.
[0153] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. A remote sensing target detection sample sampling method based on active learning, characterized in that, include: Obtain unlabeled remote sensing samples, and generate a first dataset and a second dataset based on the unlabeled remote sensing samples; The first dataset is a labeled dataset; The second dataset is an unlabeled dataset; The initial object detection network is trained using the first dataset to obtain a pre-trained object detection network; The pre-trained object detection network is used to extract features from the first dataset and the second dataset respectively, resulting in multiple first multi-level features and multiple second multi-level features. Based on the feature difference mechanism, the differences between multiple first multi-level features and multiple second multi-level features are judged to obtain difference distance information; Stability results were obtained based on the sample consistency mechanism and the second dataset; The difference distance information and the stability result are weighted and summed to obtain the uncertainty result of the second dataset; Obtain the target categories and the number of target categories in the first dataset, and obtain the category sampling weights based on the target categories and the number of target categories; A sampling score is generated for each second sample in the second dataset based on the category sampling weights and the uncertainty results; The second sample is clustered using the Mean Shift clustering algorithm to obtain clusters; In each cluster, the second sample whose sampling score meets the preset requirements is used as the target remote sensing sampling sample.
2. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The step of acquiring unlabeled remote sensing samples and generating a first dataset and a second dataset based on the unlabeled remote sensing samples includes: Obtain the unlabeled remote sensing samples; A portion of the unlabeled remote sensing samples is randomly selected as initial data, and the initial data after data labeling is used as the first dataset. The unlabeled remote sensing samples other than the initial data are used as the second dataset.
3. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The pre-trained object detection network is used to extract features from the first dataset and the second dataset respectively, resulting in multiple first multi-level features and multiple second multi-level features, including: The first dataset and the second dataset are respectively input into the feature extraction layer of the pre-trained object detection network for feature extraction; For each first sample in the first dataset, multiple first multi-level features are obtained; For each second sample in the second dataset, multiple second multi-level features are obtained.
4. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The feature difference mechanism is used to determine the differences between multiple first multi-level features and multiple second multi-level features, and to obtain difference distance information, including: The average value of multiple first multi-level features is calculated to obtain the first mean feature; The average value of multiple second-level features is calculated to obtain the second mean feature; The difference between the first mean feature and the second mean feature is calculated using Euclidean distance to obtain the difference distance information.
5. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The stability results obtained based on the sample consistency mechanism and the second dataset include: The second dataset is augmented based on the sample consistency mechanism to obtain the second augmented dataset. The stability of the second augmented dataset and the second dataset is assessed to obtain the stability result.
6. The remote sensing target detection sample sampling method based on active learning according to claim 5, characterized in that, The stability assessment of the second augmented dataset and the second dataset to obtain the stability result includes: The second augmented dataset is input into the pre-trained target detection network to obtain multiple first detection results, and information entropy is calculated on the multiple first detection results to obtain first information entropy data; The second dataset is input into the pre-trained object detection network to obtain multiple second detection results, and information entropy is calculated on the multiple second detection results to obtain second information entropy data; The variance between the first information entropy data and the second information entropy data is calculated to obtain the stability result.
7. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The step of obtaining the target categories and the number of target categories in the first dataset, and obtaining the category sampling weights based on the target categories and the number of target categories, includes: Obtain the target categories and the number of target categories in the first dataset; The total number of categories is obtained by summing the number of target categories. Divide the number of target categories by the total number of categories to obtain the category sampling weight corresponding to the target category.
8. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The step of generating a sampling score for each second sample in the second dataset based on the category sampling weights and the uncertainty result includes: The second augmented dataset is input into the pre-trained object detection network to obtain the first detection result; Based on the first detection result, the corresponding target detection weight coefficient is found in the category sampling weight; The target detection weight coefficient and the uncertainty result are input into the sampling score formula to calculate the sampling score. The sampling score formula is expressed as follows: S x U represents the sampling score. x W represents an uncertain outcome. y N represents the target detection weight coefficient corresponding to the target category y. CL This indicates the total number of target categories included in the first detection result.
9. The remote sensing target detection sample sampling method based on active learning according to claim 1, characterized in that, The step of using the second sample whose sampling score meets the preset requirements in each of the clusters as the target remote sensing sampling sample includes: Within each cluster, the sampling scores are sorted by size to obtain a sampling ranking score; The second sample whose sampling ranking score is greater than a preset threshold is used as the target remote sensing sampling sample.
10. A remote sensing target detection sample sampling device based on active learning, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the active learning-based remote sensing target detection sample sampling device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the active learning-based remote sensing target detection sample sampling method as described in any one of claims 1-9.