Remote sensing image dense small target detection system, method and equipment based on density estimation, and medium

Abstract

The invention discloses a remote sensing image dense small target detection system, method and device based on density estimation and a medium, the system comprises a self-supervised density estimation module, a density peak value optimization module and a dynamic prompt segmentation module, the system generates a small target density thermodynamic diagram through self-supervised learning by inputting a remote sensing image; and dynamically optimizing the density thermodynamic diagram by using the density peak value, obtaining a small target extraction prompt point, and driving the SAM model to obtain a segmentation result according to the small target extraction prompt point. According to the invention, through combination of self-supervised pre-training and a few-sample learning framework, in an untrained new category small target detection task, the detection accuracy is improved, and the category limitation problem in an open scene is effectively overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image dense small target detection, and in particular to a remote sensing image dense small target detection system, method, equipment and medium based on density estimation. Background Art

[0002] In the field of remote sensing image analysis, dense small object detection in open scenes faces significant technical bottlenecks. Traditional deep learning methods rely on large-scale annotated datasets, and new target categories require retraining models, resulting in high time costs and limited transferability. Existing solutions have the following shortcomings:

[0003] (1) Insufficient generalization ability of segmentation models: Mainstream instance segmentation models (such as Mask R-CNN) require fully supervised training for specific categories, and cannot achieve zero-sample transfer for unseen small target categories, making it difficult to adapt to the needs of dynamic expansion of target categories in remote sensing scenes. (2) Hint segmentation relies on manual annotation density: Although the interactive segmentation technology based on Segment Anything Model (SAM) has the ability to segment open scenes, it requires manual provision of multiple high-quality hint points. Faced with densely distributed small targets in remote sensing images (such as port container clusters), manually annotating the hint points of each target is inefficient, and the spatial overlap between dense small targets can easily lead to hint point positioning errors. (3) Poor scene adaptability of density estimation methods: Although existing density estimation models (such as CounTR) can predict target distribution, they have the following problems when directly applied to small targets: local density peak detection is easily interfered by background noise, resulting in false alarms; insufficient feature capture of untrained category targets in open scenes; and inability to generate accurate instance contours when the boundaries of dense small targets are blurred. (4) Imperfect few-shot learning scheme: Current few-shot detection methods (such as FamNet) rely on category semantic priors, have poor discrimination for small targets without obvious semantic features (such as containers of different specifications), and cannot effectively cooperate with segmentation models.

[0004] For example, the invention application with application number 202311866441.5 discloses a method, device, equipment, and storage medium for target segmentation of remote sensing images. The target segmentation model in this application is constructed based on a target detection model and a SAM model. Only a small amount of remote sensing image training data is needed to fine-tune the model parameters of the initial target detection model and the initial SAM model to obtain a target segmentation model that can accurately segment remote sensing image targets, thereby greatly reducing the target segmentation cost of remote sensing images. However, this solution also has the following problems: (1) the segmentation depends on the density of manual annotations; (2) it is not suitable for dense small target scenes, and the density estimation method has poor scene adaptability.

[0005] To address the above problems, there is an urgent need for a technical solution that integrates density estimation and prompt segmentation. Without the need for pre-training of small target categories, the automatic positioning and precise segmentation of dense small targets can be achieved through single-sample prompts, which can break through the efficiency and accuracy bottleneck of remote sensing small target detection in open scenes. Summary of the Invention

[0006] In response to the above-mentioned problems, the purpose of the present invention is to provide a remote sensing image dense small target detection system, method, equipment and medium based on density estimation, so as to realize automatic positioning and precise segmentation of dense small targets in remote sensing images.

[0007] Embodiments of the present invention provide a system, method, device and medium for detecting dense small targets in remote sensing images based on density estimation.

[0008] Aspect 1: A remote sensing image dense small target detection system based on density estimation, comprising:

[0009] The self-supervised density estimation module generates a small target density heat map based on the input remote sensing image through self-supervised learning;

[0010] Density peak optimization module, based on dynamic threshold filtering algorithm and DBSCAN clustering, optimizes density heat map to eliminate background noise points;

[0011] The dynamic prompt segmentation module obtains small target extraction prompt points as SAM model point prompts based on the optimized density heat map to obtain SAM segmentation results.

[0012] Optionally, the SAM segmentation results are back-projected to the self-supervised density estimation module, and the small object extraction prompt points are iteratively optimized to build a bidirectional feedback mechanism.

[0013] The second aspect: A method for detecting dense small targets in remote sensing images based on density estimation, comprising the following steps:

[0014] S1, input remote sensing images and generate small target density heat map through self-supervised learning;

[0015] S2. Dynamically optimize the density heat map using density peaks;

[0016] S3. Obtain small target extraction prompt points based on the optimized density heat map, and drive the SAM model based on the small target extraction prompt points to obtain the segmentation results.

[0017] Optionally, the S1 includes the steps of:

[0018] S11. Use MAE to pre-train the ViT-Base model on unlabeled remote sensing images to learn the edge texture features of small objects;

[0019] S12. Freeze the underlying parameters of the ViT-Base model and add a cross-scale feature interaction module to the CounTR network to enable attention fusion of multi-scale feature maps and improve sensitivity to small targets.

[0020] Optionally, the S2 includes the steps of:

[0021] S21. Use a sliding window to traverse the density heat map and automatically adjust the peak detection threshold according to the local density distribution based on the dynamic threshold segmentation algorithm. The formula is expressed as:

[0022]

[0023] Among them, μ local is the local mean of the sliding window, is the local variance of the sliding window, α and β are adaptive weight coefficients;

[0024] S22, based on the improved DBSCAN clustering algorithm, clustering is based on the spatial distance and density similarity of small target points. The formula is expressed as:

[0025] d(p i ,p j )=λ·‖p i -p j ‖2+(1-λ)·|D(p i )-D(p j )|

[0026] Where D(p) is the density value of the small target point p, and λ is the weight coefficient that balances the spatial distance and density difference.

[0027] Optionally, the S3 includes the steps of:

[0028] S31. Generate a small target center prompt point for each cluster center based on the optimized density heat map:

[0029] S32. Input the small target center cue point as a point cue into the SAM model to obtain the SAM segmentation result.

[0030] Optionally, the SAM model generates a multi-scale mask for each clustered small target center cue point, and the multi-scale masks are fused to obtain the SAM segmentation result.

[0031] A third aspect: An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method provided in the second aspect are implemented.

[0032] A fourth aspect: A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the second aspect.

[0033] Beneficial effects of the present invention:

[0034] 1. By combining self-supervised pre-training with a few-shot learning framework, the present invention improves the detection accuracy by 15%-25% compared with traditional solutions in the detection task of untrained new categories of small targets (such as containers, street lights, cars, etc.), effectively overcoming the category limitation problem in open scenarios and improving the system's zero-shot generalization capability.

[0035] 2. For extreme scenarios where each image contains multiple dense small targets (for example, more than 1,000), the present invention adopts density clustering optimization and multi-scale segmentation strategies. The labeling efficiency is more than 20 times that of traditional manual labeling, and the missed detection rate is controlled within 5%, achieving a breakthrough in the efficiency of processing dense targets.

[0036] 3. Under challenging conditions such as low light, target occlusion, and background interference, the present invention reduces the false detection rate by 60%-80% based on dynamic threshold correction and feedback optimization mechanism. It performs particularly well in scenarios where traditional solutions fail, such as vegetation cover and shadow areas, and enhances the robustness of complex scenarios.

[0037] 4. The present invention adopts a two-stage framework design to reduce the number of model parameters by 30%, increase the inference speed by more than 2 times, and support real-time processing of 4K resolution remote sensing images. A single GPU server can meet the needs of large-scale industrial deployment, effectively reducing the consumption and optimization of computing resources. It can be applied to smart port management, border security monitoring and other fields, and achieve a counting accuracy of more than 98% in the automatic labeling task of tens of thousands of targets, providing efficient and reliable technical support for the intelligent analysis of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the principle structure of a remote sensing image dense small target detection system based on density estimation of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of a remote sensing image dense small target detection system based on density estimation according to the present invention;

[0040] Figure 3 This is a flow chart of a method for detecting dense small targets in remote sensing images based on density estimation according to the present invention;

[0041] Figure 4 Schematic diagram of the structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0042] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0043] Existing segmentation models have the following problems: insufficient generalization ability, making it difficult to adapt to the needs of dynamic expansion of target categories in remote sensing scenes; cue segmentation relies on manual annotation density, which can easily lead to cue point positioning deviations due to spatial overlap between dense small targets; local density peak detection is easily interfered by background noise, and the ability to capture features of untrained category targets is insufficient; and there is a problem of poor discrimination of small targets without obvious semantic features.

[0044] In order to solve the above problems, the present invention provides a remote sensing image dense small target detection system based on density estimation. Figure 1 The principle flow chart of the remote sensing image dense small target detection system based on density estimation provided by the embodiment of the present invention is as follows: Figure 2 This is a structural diagram of the system, which includes a self-supervised density estimation module, a density peak optimization module, and a dynamic prompt segmentation module.

[0045] Among them, the self-supervised density estimation module is used to generate small target density heat maps through self-supervised learning for the input remote sensing image.

[0046] Existing segmentation models (such as Mask R-CNN) rely on fully supervised training and cannot generalize to untrained categories.

[0047] The present invention adopts a visual ViT-Base model pre-training strategy based on MAE (Masked Autoencoder) to capture the universal texture features of remote sensing images through self-supervised learning.

[0048] The self-supervised density estimation module constructs a two-stage training framework. First, the ViT-Base model and CounTR network are pre-trained with unlabeled remote sensing data. Then, they are fine-tuned with single-sample prompts to achieve zero-shot migration capabilities for unknown small object categories (for example, direct generalization from untrained container categories), thereby building a few-shot learning framework for self-supervised pre-training and density estimation.

[0049] The density peak optimization module optimizes the density heat map and eliminates background noise points based on the dynamic threshold filtering algorithm and DBSCAN clustering.

[0050] Existing density estimation methods are susceptible to noise interference and produce false alarms in dense small target scenes.

[0051] The present invention introduces a dynamic threshold segmentation algorithm to automatically adjust the peak detection threshold according to the local density distribution. The formula is expressed as:

[0052]

[0053] Among them, μ local is the local mean of the sliding window, is the local variance of the sliding window, and α and β are adaptive weight coefficients.

[0054] Then, an improved DBSCAN clustering algorithm is used to integrate spatial distance and density similarity metrics to eliminate discrete noise points. By designing density gradient constraints, false detection of background areas (for example, vegetation texture is misjudged as small targets) is suppressed, and density peak clustering and adaptive threshold mechanism are realized.

[0055] The dynamic prompt segmentation module obtains small target extraction prompt points as SAM model point prompts based on the optimized density heat map to obtain SAM segmentation results.

[0056] The existing SAM model has the problem of insufficient segmentation accuracy for small targets with fuzzy boundaries.

[0057] The present invention inputs the small target center cue point as a point cue into the SAM model to obtain the SAM segmentation result; at the same time, the SAM segmentation result can be back-projected into the self-supervised density estimation module, and the coordinates of the small target extraction cue point can be iteratively optimized to build a two-way feedback mechanism; SAM generates a multi-scale mask (1×1, 3×3, 5×5 pixels) for each cluster center, which can integrate the segmentation results of different receptive fields and design a multi-scale cue enhancement strategy; at the same time, a contour continuity correction algorithm is introduced to complete the broken boundaries based on the Hough transform, realizing dynamic cue correction and multi-scale segmentation mechanism.

[0058] like Figure 3 As shown, based on the above-mentioned small target detection system, the present invention discloses a remote sensing image dense small target detection method based on density estimation, comprising the steps of:

[0059] S1. Input remote sensing images and generate small target density heat maps through self-supervised learning. Specifically:

[0060] First, MAE is used to pre-train the ViT-Base model on unlabeled remote sensing images to learn the edge texture features of small objects.

[0061] For example, in the pre-training stage: MAE is used to pre-train the ViT-Base model on 2 million unlabeled remote sensing images, with the mask ratio set to 60%, focusing on learning the edge texture features of small targets such as ships and vehicles.

[0062] Then, the underlying parameters of the ViT-Base model are frozen, and a cross-scale feature interaction module is added to the CounTR network to enable attention fusion of multi-scale feature maps and improve sensitivity to small targets.

[0063] For example, the underlying parameters of the ViT-Base model are frozen, and a cross-scale feature interaction module (Cross-Scale Attention) is added to the CounTR network. For example, attention is fused between 16×16 and 32×32 scale feature maps, and fine-tuning is performed to improve sensitivity to small targets.

[0064] S2. Use density peaks to dynamically optimize the density heat map. Specifically:

[0065] First, the density heat map is traversed with a sliding window. Based on the dynamic threshold filtering algorithm, the peak detection threshold is automatically adjusted according to the local density distribution to suppress false detection in low-contrast areas of the density heat map. The formula is expressed as:

[0066]

[0067] Among them, μ local is the local mean of the sliding window, is the local variance of the sliding window, and α and β are adaptive weight coefficients.

[0068] For example, a sliding window (e.g., 32×32 pixels) is used to traverse the density heat map, calculate the local mean μ and variance σ of each sliding window, and dynamically adjust α and β in the formula, such as α = 0.8 and β = 0.2, to suppress false detection in low-contrast areas and implement dynamic threshold calculation.

[0069] Then, based on the improved DBSCAN clustering algorithm, clustering is performed based on the spatial distance and density similarity of small target points. The formula is expressed as:

[0070] d(p i ,p j )=λ·‖p i -p j ‖2+(1-λ)·|D(p i )-D(p j )|

[0071] Where D(p) is the density value of the small target point p, and λ is the weight coefficient that balances the spatial distance and density difference.

[0072] The improved DBSCAN algorithm improves the traditional Euclidean distance metric to achieve clustering of spatial proximity and similar density.

[0073] S3. Obtain small target extraction prompt points based on the optimized density heat map, and drive the SAM model based on the small target extraction prompt points to obtain segmentation results. Specifically:

[0074] First, a small target center cue point is generated for each cluster center based on the optimized density heat map.

[0075] Then, the small target center cue point is input into the SAM model as a point cue to obtain the SAM segmentation result.

[0076] A central cue point is generated for each cluster center. This cue point serves as a point cue, using single-point coordinates. This cue is fed into the SAM model to generate multi-scale masks (e.g., 1×1, 3×3, and 5×5 pixels). Segmentation results from different receptive fields are then combined. A weighted voting method is then used to calculate the IoU consistency score for the multi-scale segmentation results, retaining reliable regions with a score greater than 0.7. The SAM segmentation results are then back-projected into a self-supervised density estimation module, which iteratively optimizes the cue point coordinates for small object extraction. A bidirectional feedback mechanism is constructed, and a contour continuity correction algorithm is introduced to complete broken boundaries based on the Hough transform.

[0077] Examples, such as Figure 1 As shown, the port containers are automatically marked:

[0078] Input a drone aerial image of 356 containers with a target size of 15-20 pixels.

[0079] First, density estimation is performed, and a heat map is generated based on the CounTR network, and 382 candidate peaks are preliminarily detected.

[0080] Then, peak optimization was performed, dynamic threshold filtering and DBSCAN clustering were performed to eliminate 32 background noise points and obtain 324 valid prompt points.

[0081] Finally, segmentation is performed based on SAM, and multi-scale masks are generated for each hint point. After fusion, the precise instance contour is obtained, and the final counting accuracy reaches 98.2% (349 / 356 correct).

[0082] The target is located by density estimation, prompt points are generated by cluster optimization, and higher counting accuracy is achieved after SAM segmentation.

[0083] The innovative effect of the present invention is verified: for port container inspection, drone aerial images are input and the labeling efficiency of the method of the present invention is improved by 12 times compared with traditional solutions (for example: FamNet+SAM) (the time consumption for a single image is reduced from 6 minutes to 30 seconds).

[0084] Zero-shot testing on the COWC vehicle dataset shows that mAP@0.5 reaches 76.3%, surpassing the BMNet+ solution by 9.2 percentage points.

[0085] The present invention also provides an electronic device, Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor may call logic instructions in the memory, for example, to execute the following method:

[0086] S1, input remote sensing images and generate small target density heat map through self-supervised learning;

[0087] S2. Dynamically optimize the density heat map using density peaks;

[0088] S3. Obtain small target extraction prompt points based on the optimized density heat map, and drive the SAM model based on the small target extraction prompt points to obtain the segmentation results.

[0089] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0090] An embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in each of the above embodiments is implemented, for example, including:

[0091] S1, input remote sensing images and generate small target density heat map through self-supervised learning;

[0092] S2. Dynamically optimize the density heat map using density peaks;

[0093] S3. Obtain small target extraction prompt points based on the optimized density heat map, and drive the SAM model based on the small target extraction prompt points to obtain the segmentation results.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A remote sensing image dense small target detection system based on density estimation, characterized by: include: The self-supervised density estimation module generates a small target density heat map based on the input remote sensing image through self-supervised learning; Density peak optimization module, based on dynamic threshold filtering algorithm and DBSCAN clustering, optimizes density heat map to eliminate background noise points; The dynamic prompt segmentation module obtains small target extraction prompt points as SAM model point prompts based on the optimized density heat map to obtain SAM segmentation results.

2. The small target detection system according to claim 1, characterized in that: The SAM segmentation results are back-projected into the self-supervised density estimation module, and the small target extraction prompt points are iteratively optimized to build a bidirectional feedback mechanism.

3. The small target detection system according to claim 1 or 2, characterized in that: The target detection method applied to the small target detection system includes the following steps: S1, input remote sensing images and generate small target density heat map through self-supervised learning; S2. Dynamically optimize the density heat map using density peaks; S3. Obtain small target extraction prompt points based on the optimized density heat map, and drive the SAM model based on the small target extraction prompt points to obtain the segmentation results.

4. The target detection method according to claim 3, wherein: Said S1 comprises the steps of: S11. Use MAE to pre-train the ViT-Base model on unlabeled remote sensing images to learn the edge texture features of small objects; S12. Freeze the underlying parameters of the ViT-Base model and add a cross-scale feature interaction module to the CounTR network to enable attention fusion of multi-scale feature maps and improve sensitivity to small targets.

5. The target detection method according to claim 4, characterized in that: Said S2 comprises the steps of: S21. Use a sliding window to traverse the density heat map and automatically adjust the peak detection threshold according to the local density distribution based on the dynamic threshold segmentation algorithm. The formula is expressed as: Among them, μ local is the local mean of the sliding window, is the local variance of the sliding window, α and β are adaptive weight coefficients; S22, based on the improved DBSCAN clustering algorithm, clustering is based on the spatial distance and density similarity of small target points. The formula is expressed as: d(p i ,p j )=λ·‖p i -p j ‖2+(1-λ)·|D(p i )-D(p j )| Where D(p) is the density value of the small target point p, and λ is the weight coefficient that balances the spatial distance and density difference.

6. The target detection method according to claim 5, characterized in that: The S3 comprises the steps of: S31. Generate a small target center prompt point for each cluster center based on the optimized density heat map: S32. Input the small target center cue point as a point cue into the SAM model to obtain the SAM segmentation result.

7. The target detection method according to claim 6, characterized in that: The SAM model generates a multi-scale mask for each clustered small target center cue point, and the SAM segmentation result is obtained by fusing the multi-scale masks.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the target detection method according to any one of claims 3 to 7 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target detection method according to any one of claims 3 to 7 are implemented.

Images