A target detection method, equipment, and medium for a remote sensing image-based system for detecting dense small targets using density estimation.

Abstract

This invention discloses a target detection method, device, and medium based on density estimation in a remote sensing image-based dense small target detection system. The system includes a self-supervised density estimation module, a density peak optimization module, and a dynamic cue segmentation module. The system takes a remote sensing image as input and generates a small target density heatmap through self-supervised learning. It then uses the density peak to dynamically optimize the density heatmap, obtains small target extraction cue points, and drives a SAM model based on these cue points to obtain segmentation results. This invention, by combining self-supervised pre-training with a few-shot learning framework, improves detection accuracy in new categories of small targets without prior training, effectively overcoming the category limitation problem in open scenes.

Description

Technical Field

[0001] This invention relates to the field of dense small target detection technology in remote sensing images, and in particular to a target detection method, device and medium based on a density estimation-based dense small target detection system in remote sensing images. Background Technology

[0002] In the field of remote sensing image analysis, the detection of dense small targets in open scenes faces significant technical bottlenecks. Traditional deep learning methods rely on large-scale labeled datasets, but retraining the model is necessary for newly emerging target categories, resulting in high time costs and limited transferability. Existing solutions have the following shortcomings:

[0003] (1) Insufficient generalization ability of segmentation models: Mainstream instance segmentation models (such as MaskR-CNN) require fully supervised training for specific categories, and cannot achieve zero-sample transfer for unseen small target categories, making it difficult to adapt to the needs of dynamic expansion of target categories in remote sensing scenes. (2) Cue segmentation relies on manual labeling density: Although interactive segmentation technology based on Segment Anything Model (SAM) has the ability to segment open scenes, it requires multiple high-quality cue points to be provided manually. When facing densely distributed small targets (such as port container groups) in remote sensing images, manually labeling cue points for each target is inefficient, and the spatial overlap between dense small targets can easily lead to cue point positioning errors. (3) Poor scene adaptability of density estimation methods: Although existing density estimation models (such as CounTR) can predict target distribution, they have the following problems when directly applied to small targets: the detection of local density peaks is easily interfered with by background noise, resulting in false alarms. In open scenes, the ability to capture the features of untrained categories is insufficient, and accurate instance contours cannot be generated when the boundaries of dense small targets are blurred. (4) Inadequate few-shot learning schemes: Current few-shot detection methods (such as FamNet) rely on category semantic priors, which have poor discrimination for small targets (such as containers of different sizes) without obvious semantic features, and cannot effectively cooperate with segmentation models.

[0004] For example, application number 202311866441.5 discloses a method, apparatus, device, and storage medium for target segmentation of remote sensing images. The target segmentation model in this application is constructed based on a target detection model and a SAM model. Only a small amount of remote sensing image training data is needed to fine-tune the model parameters of the initial target detection model and the initial SAM model to obtain a target segmentation model that can accurately segment remote sensing images, thereby greatly reducing the cost of target segmentation of remote sensing images. However, its solution also has the following problems: (1) the prompt segmentation depends on the density of manual annotation; (2) it is not suitable for dense small target scenes, and the density estimation method has poor scene adaptability.

[0005] To address the aforementioned issues, there is an urgent need for a technical solution that integrates density estimation and cue segmentation. This solution can achieve automatic localization and accurate segmentation of dense small targets through single-sample cues, eliminating the need for pre-training of small target categories. This approach can overcome the efficiency and accuracy bottlenecks in remote sensing small target detection in open scenarios. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a target detection method, device, and medium for a remote sensing image dense small target detection system based on density estimation, thereby achieving automatic positioning and accurate segmentation of dense small targets in remote sensing images.

[0007] This invention provides a target detection method, device, and medium for a remote sensing image dense small target detection system based on density estimation.

[0008] First aspect: A target detection method based on a density estimation-based remote sensing image dense small target detection system, including the following steps:

[0009] S1. Input remote sensing images and generate a small target density heat map through self-supervised learning;

[0010] S2. Optimize the density thermogram dynamically using density peaks;

[0011] S3. Obtain small target extraction prompts based on the optimized density heatmap, and drive the SAM model to obtain segmentation results based on the small target extraction prompts;

[0012] S1 includes the following steps:

[0013] S11. The ViT-Base model is pre-trained on unlabeled remote sensing images using MAE to learn the edge texture features of small targets.

[0014] S12. Freeze the underlying parameters of the ViT-Base model and add a cross-scale feature interaction module to the CounTR network to enable attention fusion of multi-scale feature maps and improve sensitivity to small targets.

[0015] S2 includes the following steps:

[0016] S21. Traverse the density heatmap using a sliding window. Based on the dynamic threshold segmentation algorithm, automatically adjust the peak detection threshold according to the local density distribution. The formula is as follows:

[0017]

[0018] Where, μ local This represents the local mean of the sliding window. Let α and β be the local variance of the sliding window, and α and β be the adaptive weighting coefficients.

[0019] S22. Based on the improved DBSCAN clustering algorithm, clustering is performed based on the spatial distance and density similarity of small target points. The formula is expressed as:

[0020] d(p i ,p j )=λ·‖p i -p j ‖2+(1-λ)·|D(p i )-D(p j )|

[0021] Where D(p) is the density value of the small target point p, and λ is the weighting coefficient that balances spatial distance and density difference.

[0022] The system includes:

[0023] The self-supervised density estimation module generates a small target density heatmap from the input remote sensing image through self-supervised learning.

[0024] The density peak optimization module, based on dynamic threshold filtering algorithm and DBSCAN clustering, optimizes the density heatmap to eliminate background noise points;

[0025] The dynamic prompting segmentation module, based on an optimized density heatmap, extracts prompt points for small targets as prompts for the SAM model, and obtains the SAM segmentation results;

[0026] Specifically, the SAM segmentation results are back-projected to the self-supervised density estimation module, and the small target extraction cue points are iteratively optimized to construct a two-way feedback mechanism.

[0027] Optionally, step S3 includes the following steps:

[0028] S31. Generate one small target center prompt point for each cluster center based on the optimized density heatmap:

[0029] S32. Input the small target center cue point as a cue point into the SAM model to obtain the SAM segmentation result.

[0030] Optionally, the SAM model generates a multi-scale mask for the center cue point of each clustered small target, and the SAM segmentation result is obtained by fusing the multi-scale masks.

[0031] Second aspect: An electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the steps of the method provided in the first aspect.

[0032] Third aspect: A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect.

[0033] The beneficial effects of this invention are:

[0034] 1. This invention combines self-supervised pre-training with a few-shot learning framework to improve the detection accuracy of new categories of small targets (such as containers, streetlights, cars, etc.) that have not been trained by traditional methods by 15%-25%, effectively overcoming the category limitation problem in open scenarios and improving the system's zero-shot generalization ability.

[0035] 2. This invention addresses extreme scenarios where each image contains multiple dense small targets (e.g., more than 1000). By employing density clustering optimization and multi-scale segmentation strategies, the annotation efficiency reaches more than 20 times that of traditional manual annotation, while the false negative rate is controlled within 5%, achieving a breakthrough in the efficiency of processing dense targets.

[0036] 3. Under challenging conditions such as low light, target occlusion, and background interference, this invention reduces the false detection rate by 60%-80% based on dynamic threshold correction and feedback optimization mechanisms. It is particularly outstanding in scenarios where traditional solutions fail, such as vegetation cover and shaded areas, thus enhancing robustness in complex scenarios.

[0037] 4. This invention adopts a two-stage framework design, which reduces the number of model parameters by 30%, increases the inference speed by more than 2 times, and supports real-time processing of 4K resolution remote sensing images. A single GPU server can meet the needs of large-scale industrial deployment, effectively reducing the consumption of computing resources. It can be applied to fields such as smart port management and border security monitoring. It achieves a counting accuracy of more than 98% in automatic annotation tasks with tens of thousands of targets, providing efficient and reliable technical support for intelligent analysis of remote sensing images. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the principle and structure of a target detection method in a remote sensing image dense small target detection system based on density estimation, according to the present invention.

[0039] Figure 2 This is a schematic diagram of the structure of a remote sensing image dense small target detection system based on density estimation according to the present invention;

[0040] Figure 3 This is a flowchart illustrating a target detection method for a remote sensing image-based dense small target detection system based on density estimation, according to the present invention.

[0041] Figure 4 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0042] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0043] Existing segmentation models suffer from the following problems: insufficient generalization ability, making it difficult to adapt to the dynamic expansion of target categories in remote sensing scenes; reliance on manually labeled density for cue segmentation, which can easily lead to cue point positioning errors due to spatial overlap between dense small targets; susceptibility of local density peak detection to background noise interference, and insufficient ability to capture untrained category target features; and poor discriminative ability for small targets without obvious semantic features.

[0044] To address the aforementioned problems, this invention provides a target detection method for a remote sensing image-based dense small target detection system based on density estimation. Figure 1 This is a flowchart illustrating the principle of the target detection method in the remote sensing image dense small target detection system based on density estimation provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the system structure, which includes a self-supervised density estimation module, a density peak optimization module, and a dynamic cueing segmentation module.

[0045] The self-supervised density estimation module is used to generate a small target density heatmap from the input remote sensing image through self-supervised learning.

[0046] Existing segmentation models (such as Mask R-CNN) suffer from the problem of relying on fully supervised training and being unable to generalize to untrained categories.

[0047] This invention employs a visual ViT-Base model pre-training strategy based on MAE (Masked Autoencoder) to capture general texture features of remote sensing images through self-supervised learning.

[0048] The self-supervised density estimation module constructs a two-stage training framework. First, it pre-trains the ViT-Base model and the CounTR network using unlabeled remote sensing data. Then, it fine-tunes the model using single-sample prompts, achieving zero-sample transfer capability for unknown small target categories (e.g., direct generalization to the untrained container category). This builds a few-shot learning framework for self-supervised pre-training and density estimation.

[0049] The density peak optimization module optimizes the density heatmap by using a dynamic threshold filtering algorithm and DBSCAN clustering to eliminate background noise points.

[0050] Existing density estimation methods are susceptible to noise interference and generate false alarms in dense small target scenarios.

[0051] This invention introduces a dynamic threshold segmentation algorithm, which automatically adjusts the peak detection threshold based on the local density distribution. The formula is as follows:

[0052]

[0053] Where, μ local This represents the local mean of the sliding window. Let α be the local variance of the sliding window, and β be the adaptive weighting coefficients.

[0054] Then, an improved DBSCAN clustering algorithm is adopted to integrate spatial distance and density similarity measures, eliminate discrete noise points, and suppress false detections in the background area (e.g., misjudging vegetation texture as small targets) by designing density gradient constraints, thereby realizing density peak clustering and adaptive threshold mechanism.

[0055] The dynamic prompting segmentation module uses an optimized density heatmap to extract prompt points from small targets as prompts for the SAM model, and then obtains the SAM segmentation results.

[0056] Existing SAM models suffer from insufficient segmentation accuracy on small targets with blurred boundaries.

[0057] This invention uses the center point of a small target as a point cue input to the SAM model to obtain the SAM segmentation result. Simultaneously, the SAM segmentation result can be back-projected to a self-supervised density estimation module to iteratively optimize the coordinates of the small target extraction cue points, constructing a two-way feedback mechanism. The SAM generates multi-scale masks (1×1, 3×3, 5×5 pixels) for each cluster center, which can integrate segmentation results from different receptive fields to design a multi-scale cue enhancement strategy. Furthermore, a contour continuity correction algorithm is introduced, based on Hough transform to complete the fracture boundaries, realizing dynamic cue correction and a multi-scale segmentation mechanism.

[0058] like Figure 3 As shown, based on the above-mentioned small target detection system, this invention discloses a target detection method based on a density estimation-based remote sensing image dense small target detection system, comprising the following steps:

[0059] S1. Input remote sensing imagery and generate a small target density heatmap through self-supervised learning. Specifically:

[0060] First, the ViT-Base model is pre-trained on unlabeled remote sensing images using MAE to learn the edge texture features of small targets.

[0061] For example, in the pre-training stage: the ViT-Base model is pre-trained on 2 million unlabeled remote sensing images using MAE, with the mask ratio set to 60%, focusing on learning the edge texture features of small targets such as ships and vehicles.

[0062] Then, the underlying parameters of the ViT-Base model are frozen, and a cross-scale feature interaction module is added to the CounTR network to enable attention fusion of multi-scale feature maps and improve sensitivity to small targets.

[0063] For example, freeze the underlying parameters of the ViT-Base model and add a cross-scale attention module to the CounTR network, such as performing attention fusion between 16×16 and 32×32 scale feature maps, to fine-tune and improve the sensitivity to small targets.

[0064] S2. Optimize the density thermogram dynamically using density peak values, specifically:

[0065] First, the density heatmap is traversed using a sliding window. Based on a dynamic threshold filtering algorithm, the peak detection threshold is automatically adjusted according to the local density distribution to suppress false detections in low-contrast areas of the density heatmap. The formula is expressed as:

[0066]

[0067] Where, μ local This represents the local mean of the sliding window. Let α be the local variance of the sliding window, and β be the adaptive weighting coefficients.

[0068] For example, by traversing the density heatmap with a sliding window (e.g., 32×32 pixels), the local mean μ and variance σ of each sliding window are calculated, and α and β in the formula are dynamically adjusted, for example, α = 0.8 and β = 0.2, to suppress false detections in low-contrast areas and achieve dynamic threshold calculation.

[0069] Then, based on the improved DBSCAN clustering algorithm, clustering is performed based on the spatial distance and density similarity of small target points, as expressed in the formula:

[0070] d(p i ,p j )=λ·‖p i -p j ‖2+(1-λ)·|D(p i )-D(p j )|

[0071] Where D(p) is the density value of the small target point p, and λ is the weighting coefficient that balances spatial distance and density difference.

[0072] The improved DBSCAN algorithm enhances the traditional Euclidean distance metric to achieve spatially proximate clustering with similar densities.

[0073] S3. Obtain small target extraction cue points based on the optimized density heatmap, and drive the SAM model to obtain segmentation results based on the small target extraction cue points. Specifically:

[0074] First, a small target center cue point is generated for each cluster center based on the optimized density heatmap.

[0075] Then, the small target center cue point is used as a cue point input into the SAM model to obtain the SAM segmentation result.

[0076] One central cue point is generated for each cluster center, serving as a point cue. The point cue uses single-point coordinates and is input into the SAM model to generate multi-scale masks (e.g., 1×1, 3×3, 5×5 pixels). Segmentation results from different receptive fields are then combined. A weighted voting method is used to calculate the IoU consistency score for the multi-scale segmentation results, retaining reliable regions with scores > 0.7. Simultaneously, the SAM segmentation results can be back-projected to a self-supervised density estimation module to iteratively optimize the coordinates of small target cue points, constructing a two-way feedback mechanism. Furthermore, a contour continuity correction algorithm is introduced to complete fracture boundaries based on Hough transform.

[0077] Examples, such as Figure 1 As shown, automatic labeling is performed on port containers:

[0078] Input a drone aerial image containing 356 containers, with a target size of 15-20 pixels.

[0079] First, density estimation was performed, and a heatmap was generated based on the CounTR network, initially detecting 382 candidate peaks.

[0080] Then, peak optimization was performed, and after dynamic threshold filtering and DBSCAN clustering, 32 background noise points were eliminated, resulting in 324 effective cue points.

[0081] Finally, the segmentation was performed based on SAM, and a multi-scale mask was generated for each cue point. After fusion, an accurate instance outline was obtained, and the final counting accuracy reached 98.2% (349 / 356 correct).

[0082] By locating the target through density estimation and generating cue points through cluster optimization, this invention achieves high counting accuracy after SAM segmentation.

[0083] The innovative effect of this invention is verified: When inspecting containers in a port, the method of this invention improves the annotation efficiency by 12 times compared with traditional solutions (e.g., FamNet+SAM) by inputting drone aerial images (the time taken per image is reduced from 6 minutes to 30 seconds).

[0084] Zero-shot testing on the COWC vehicle dataset shows that mAP@0.5 reaches 76.3%, surpassing the BMNet+ solution by 9.2 percentage points.

[0085] The present invention also provides an electronic device, Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory, for example, to execute the following method:

[0086] S1. Input remote sensing images and generate a small target density heat map through self-supervised learning;

[0087] S2. Optimize the density thermogram dynamically using density peaks;

[0088] S3. Obtain small target extraction prompts based on the optimized density heatmap, and drive the SAM model to obtain segmentation results based on the small target extraction prompts.

[0089] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, including, for example:

[0091] S1. Input remote sensing images and generate a small target density heat map through self-supervised learning;

[0092] S2. Optimize the density thermogram dynamically using density peaks;

[0093] S3. Obtain small target extraction prompts based on the optimized density heatmap, and drive the SAM model to obtain segmentation results based on the small target extraction prompts.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method for a remote sensing image dense small target detection system based on density estimation, characterized in that, Including the following steps: S1. Input remote sensing images and generate a small target density heat map through self-supervised learning; S2. Optimize the density thermogram dynamically using density peaks; S3. Obtain small target extraction prompts based on the optimized density heatmap, and drive the SAM model to obtain segmentation results based on the small target extraction prompts; S1 includes the following steps: S11. The ViT-Base model is pre-trained on unlabeled remote sensing images using MAE to learn the edge texture features of small targets. S12. Freeze the underlying parameters of the ViT-Base model and add a cross-scale feature interaction module to the CounTR network to enable attention fusion of multi-scale feature maps and improve sensitivity to small targets. S2 includes the following steps: S21. Traverse the density heatmap using a sliding window. Based on the dynamic threshold segmentation algorithm, automatically adjust the peak detection threshold according to the local density distribution. The formula is as follows: Where, μ local This represents the local mean of the sliding window. Let α and β be the local variance of the sliding window, and α and β be the adaptive weighting coefficients. S22. Based on the improved DBSCAN clustering algorithm, clustering is performed based on the spatial distance and density similarity of small target points, expressed by the following formula: d(p i ,p j )=λ·‖p i -p j ‖2+(1-λ)·|D(p i )-D(p j )| Where D(p) is the density value of the small target point p, and λ is the weighting coefficient that balances spatial distance and density difference; The system includes: The self-supervised density estimation module generates a small target density heatmap from the input remote sensing image through self-supervised learning. The density peak optimization module, based on dynamic threshold filtering algorithm and DBSCAN clustering, optimizes the density heatmap to eliminate background noise points; The dynamic prompting segmentation module, based on an optimized density heatmap, extracts prompt points for small targets as prompts for the SAM model, and obtains the SAM segmentation results; Specifically, the SAM segmentation results are back-projected to the self-supervised density estimation module, and the small target extraction cue points are iteratively optimized to construct a two-way feedback mechanism.

2. The method according to claim 1, characterized in that, S3 includes the following steps: S31. Generate one small target center prompt point for each cluster center based on the optimized density heatmap: S32. Input the small target center cue point as a cue point into the SAM model to obtain the SAM segmentation result.

3. The method according to claim 2, characterized in that, The SAM model generates a multi-scale mask for the center cue point of each clustered small target, and the SAM segmentation result is obtained by fusing the multi-scale masks.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 3.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 3.

Images