Small target detection method and device based on high-resolution fused feature map

Abstract

The application provides a small target detection method and device based on a high-resolution fusion feature map, which comprises the following steps: dividing a to-be-detected image into a plurality of overlapping sub-regions and recording position information of the sub-regions; performing feature extraction on each sub-region and generating a high-resolution fusion feature map through multi-scale fusion; generating geometric parameters and initial confidence of a candidate box based on each spatial position on the map, and obtaining a candidate box set of the sub-regions through preliminary screening; mapping the candidate boxes in all candidate box sets to a global coordinate system according to the position information, merging repeated candidate boxes pointing to the same small target, and forming a global candidate box set; performing fine discrimination on each candidate box in the global candidate box set by cutting a local image region in the to-be-detected image, and obtaining a fine confidence; fusing the initial confidence and the fine confidence to obtain a comprehensive confidence, and outputting a final small target detection result according to a comparison result of the comprehensive confidence and an adaptive threshold. The application can accurately capture a small target in an image.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for small target detection based on high-resolution fused feature maps. Background Technology

[0002] With the rapid development of fields such as intelligent security, autonomous driving, and remote sensing analysis, target detection technology has become a key support. However, in complex scenes, targets often appear in the form of small size, low resolution, and weak contrast, posing a unique challenge to small target detection.

[0003] Existing general detection methods have significant limitations in this scenario: multiple downsampling operations in deep feature extraction easily lose subtle features of small targets; candidate box generation mechanisms based on anchor points or sparse sampling have insufficient coverage density, leading to high false negative rates; when processing multi-view or slice detection results, traditional non-maximum suppression strategies struggle to stably integrate repeated detections and are prone to mistakenly deleting valid targets; furthermore, small targets have high similarity to background noise, and single-stage classification mechanisms lack fine-grained discrimination capabilities, resulting in persistently high false positive rates. These shortcomings severely restrict the accuracy and reliability of small target detection in practical applications.

[0004] Therefore, existing technologies lack a small target detection method and device based on high-resolution fusion feature maps that can achieve both high recall and high accuracy. Summary of the Invention

[0005] Therefore, it is necessary to provide a small target detection method and apparatus based on high-resolution fused feature maps to address the aforementioned technical problems.

[0006] In a first aspect, the present invention provides a small target detection method based on high-resolution fused feature maps, comprising:

[0007] The image to be detected is divided into regions to obtain multiple overlapping sub-regions, and the position information of each sub-region in the image to be detected is recorded.

[0008] For each sub-region, the following processing is performed: feature extraction and multi-scale fusion are performed on the sub-region to generate a high-resolution fused feature map; based on each spatial location of the high-resolution fused feature map, the geometric parameters of candidate boxes for small targets and their corresponding initial confidence scores are generated; the geometric parameters and corresponding initial confidence scores of each candidate box are initially screened to obtain the candidate box set for that sub-region.

[0009] Based on the location information of each sub-region, the candidate boxes in the candidate box set of all sub-regions are mapped to the global coordinate system of the image to be detected, and duplicate candidate boxes that point to the same small target after mapping are merged to form a global candidate box set.

[0010] For each candidate box in the global candidate box set, the corresponding local image region is cropped in the image to be detected for fine discrimination to obtain the refined confidence score;

[0011] The initial and refined confidence scores of each candidate box are combined to obtain a comprehensive confidence score. Based on the comparison between the comprehensive confidence score and an adaptive threshold related to the target size, the final small target detection result is output.

[0012] Optionally, the method further includes:

[0013] Obtain the size information of the original image;

[0014] If the ratio of the width to the height of the original image is greater than the first preset ratio, the original image is cropped into multiple sub-images, wherein the ratio of the width to the height of each sub-image is less than or equal to the second preset ratio.

[0015] For each sub-image obtained from the original image or the cropped image, determine whether the maximum side length of the original image or each sub-image obtained from the cropped image is greater than a preset size threshold.

[0016] If the maximum side length of the original image or the cropped sub-image is greater than the preset size threshold, the original image or the cropped sub-image is scaled proportionally so that the maximum side length of the scaled original image or the cropped sub-image is equal to the preset size threshold, and the scaled original image or the cropped sub-image is used as the image to be detected.

[0017] If the maximum side length of the original image or the cropped sub-image is less than or equal to the preset size threshold, it will be directly used as the image to be detected.

[0018] For video stream input, the preset size threshold is determined based on the maximum side length of the first frame of the original image in the video stream, and the same scaling ratio is applied to the original images of subsequent frames.

[0019] Optionally, the image to be detected is divided into regions to obtain multiple overlapping sub-regions, and the position information of each sub-region in the image to be detected is recorded, including:

[0020] The sub-region division size is determined based on the preset maximum size of the small target;

[0021] The sliding step size is determined based on the preset overlap rate and division size;

[0022] A sliding window is used to divide the image to be detected by a sliding step size, generating multiple overlapping sub-regions. The coordinates of the upper left corner of each sub-region in the image to be detected are recorded as the position information of the corresponding sub-region. The sliding window is a square region with a side length equal to the division size.

[0023] When the defined sub-region exceeds the boundary of the image to be detected, the actual size of the sub-region is determined with the boundary of the image to be detected as the limit.

[0024] Optionally, feature extraction and multi-scale fusion are performed on the sub-regions to generate a high-resolution fused feature map, including:

[0025] A pre-defined feature extraction network is used to extract multi-scale features from the sub-region. The multi-scale features include at least the first scale feature with the largest spatial size, the second scale feature with the middle spatial size, and the third scale feature with the smallest spatial size.

[0026] Upsample the second-scale and third-scale features so that the spatial dimensions of the second-scale and third-scale features are aligned with the spatial dimensions of the first-scale features.

[0027] The first-scale features are fused with the upsampled and aligned second-scale and third-scale features according to preset weights to generate a high-resolution fused feature map of the sub-region.

[0028] Optionally, based on each spatial location of the high-resolution fused feature map, the geometric parameters of the candidate bounding box for the small target and the corresponding initial confidence score are generated, including:

[0029] Each spatial location of the high-resolution fused feature map is mapped to the image coordinate system of the sub-region to obtain the corresponding reference center point coordinates;

[0030] Using a pre-defined detection network, a set of target parameters are predicted based on the features of each spatial location. These target parameters include the probability of target presence, center point offset, width adjustment, and height adjustment.

[0031] Based on the reference center point coordinates and target parameters, generate the geometric parameters of the candidate bounding box for each small target at each spatial location. The geometric parameters include the center point coordinates, width, and height of the candidate bounding box in the sub-region.

[0032] The predicted probability of the target's existence is used as the initial confidence level for the candidate box.

[0033] Optionally, the geometric parameters and corresponding initial confidence scores of each candidate box are initially screened to obtain a set of candidate boxes for the sub-region, including:

[0034] Select geometric parameters from the candidate bounding boxes whose width and height both conform to the preset range of small target sizes;

[0035] From the geometric parameters filtered by the small target size range, select the geometric parameters whose initial confidence is greater than or equal to the preset initial confidence threshold.

[0036] The geometric parameters and their corresponding initial confidence scores that simultaneously meet both the small target size range and the initial confidence threshold are used to form the candidate box set for the sub-region.

[0037] Optionally, based on the location information of each sub-region, the candidate boxes in the candidate box set of all sub-regions are mapped to the global coordinate system of the image to be detected, and duplicate candidate boxes pointing to the same small target after mapping are merged to form a global candidate box set, including:

[0038] For each candidate box set in a sub-region, obtain the coordinates of the top-left corner of the sub-region in the image to be detected, and based on the coordinates, convert the center point coordinates of each candidate box in the candidate box set in the sub-region into the global center point coordinates in the global coordinate system of the image to be detected;

[0039] For all candidate boxes transformed to the global coordinate system, if the intersection-union ratio between any two candidate boxes is greater than or equal to the first preset threshold, and / or the normalized center distance between any two candidate boxes is less than or equal to the dynamic distance threshold, the two candidate boxes are determined to point to the same small target.

[0040] For candidate boxes identified as pointing to the same small target, the global center point coordinates, width, and height of each box are weighted and averaged according to their initial confidence scores to calculate the merged geometric parameters. The highest initial confidence score among the merged geometric parameters is then used as the initial confidence score of the merged candidate box.

[0041] All candidate boxes after merging and their corresponding initial confidence scores are combined to form a global candidate box set.

[0042] Optionally, for each candidate box in the global candidate box set, a corresponding local image region is cropped in the image to be detected for refined discrimination to obtain a refined confidence score, including:

[0043] Using the global center point coordinates of each candidate box in the global candidate box set as the center, a local image region including context information is cropped from the image to be detected according to a preset expansion coefficient.

[0044] The cropped local image region is resampled to a preset uniform size;

[0045] A local image region of uniform size is input into a preset lightweight discriminant network, which outputs a refined confidence score representing the probability that the local image region contains a real small target.

[0046] Optionally, the initial confidence and refined confidence of each candidate box are fused to obtain a comprehensive confidence score. Based on the comparison between the comprehensive confidence score and an adaptive threshold related to the target size, the final small target detection result is output, including:

[0047] For each candidate box, a dynamic weighted fusion formula is used. Calculate the overall confidence level, where, To assess the overall confidence level, As the initial confidence level, To refine the confidence level, and These are the first and second weighting coefficients;

[0048] Calculate the adaptive decision threshold for each candidate box, where the adaptive decision threshold is calculated using the following formula: ,in, For adaptive decision threshold, and The final width and final height of the candidate box. The downsampling step size of the high-resolution fused feature map relative to the sub-region. Indicates rounding down;

[0049] Determine whether the overall confidence score is greater than or equal to the adaptive decision threshold; if the overall confidence score is greater than or equal to the adaptive decision threshold, output the candidate box and its overall confidence score as the final small object detection result; if the overall confidence score is less than the adaptive decision threshold, filter out the candidate box.

[0050] Secondly, the present invention provides a small target detection device based on high-resolution fused feature maps, comprising:

[0051] The region segmentation module is used to segment the image to be detected into multiple overlapping sub-regions and record the position information of each sub-region in the image to be detected.

[0052] The sub-region processing module, connected to the region partitioning module, performs the following processing on each sub-region: feature extraction and multi-scale fusion of the sub-region to generate a high-resolution fused feature map; based on each spatial location of the high-resolution fused feature map, generating the geometric parameters and corresponding initial confidence scores of candidate boxes for small targets; and performing preliminary screening on the geometric parameters and corresponding initial confidence scores of each candidate box to obtain a set of candidate boxes for that sub-region.

[0053] The global fusion module, connected to the sub-region processing module, is used to map the candidate boxes in the candidate box set of all sub-regions to the global coordinate system of the image to be detected based on the position information of each sub-region, and merge the duplicate candidate boxes that point to the same small target after mapping to form a global candidate box set.

[0054] The fine discrimination module, connected to the global fusion module, is used to perform fine discrimination on each candidate box in the global candidate box set, cropping the corresponding local image region in the image to be detected, and obtaining the fine-tuned confidence score.

[0055] The results output module, connected to the fine discrimination module, is used to fuse the initial confidence and refined confidence of each candidate box to obtain the comprehensive confidence. Based on the comparison between the comprehensive confidence and the adaptive threshold related to the target size, the final small target detection result is output.

[0056] The small target detection method and apparatus based on high-resolution fused feature maps provided by this invention significantly improves the perception capability and coverage density of small target features by introducing high-resolution fused feature maps to generate candidate boxes point by point and combining them with an overlapping sub-region segmentation strategy. This effectively overcomes the problem of missed detection caused by small target size and weak features. Furthermore, through global coordinate system mapping and spatial consistency fusion mechanism, it can stably merge duplicate detection results from different sub-regions, avoiding the false deletion of valid targets that may be caused by traditional non-maximum suppression strategies, and enhancing the robustness of the detection results. In addition, it also uses local resampling and a lightweight discriminant network for fine verification, and combines dynamic fusion of initial confidence and refined confidence with scale-adaptive decision-making to significantly reduce the false alarm rate in complex backgrounds. Attached Figure Description

[0057] Figure 1a This is a schematic diagram of a small target detection method based on high-resolution fused feature maps provided in an embodiment of the present invention;

[0058] Figure 1b This is another flowchart illustrating the small target detection method based on high-resolution fused feature maps provided in this embodiment of the invention;

[0059] Figure 1c This is another flowchart illustrating the small target detection method based on high-resolution fused feature maps provided in this embodiment of the invention.

[0060] Figure 2 This is a schematic diagram of the circuit module structure of a small target detection device based on high-resolution fused feature maps provided in an embodiment of the present invention;

[0061] Figure 3 This is an internal structural diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0063] like Figure 1a As shown, this invention provides a small target detection method based on high-resolution fused feature maps, including:

[0064] Step S11: Divide the image to be detected into regions to obtain multiple overlapping sub-regions, and record the position information of each sub-region in the image to be detected;

[0065] Optionally, such as Figure 1b As shown, step S11 specifically includes:

[0066] Step S111: Determine the sub-region division size based on the preset maximum size of the small target;

[0067] In this invention, the preset maximum size of the small target refers to the upper limit of the size of the "small target", which is expressed in pixels as the width or height of the bounding box of the small target. This maximum size of the small target needs to be determined comprehensively based on the actual physical size of the target and the image resolution in the specific business scenario. Generally, it can be determined by statistically analyzing the target size distribution of the training set or typical business images, and selecting the size that can cover the vast majority of small targets (such as more than 90%) as this value. Preferably, the width of the maximum size of the small target is [20, 64] and / or the height of the maximum size of the small target is [20, 64].

[0068] After determining the maximum size of the small target, the formula P=min(1024,max(384,2×max) can be used. STS ×10), calculate the sub-region partitioning size; where P is the partitioning size, max STS This represents the maximum size of the small target. Specifically, if max... STS =32, then P=min(1024,max(384,2×32×10))=min(1024,max(384,640))=min(1024,640)=640. It should be noted that the division size must be an integer multiple of 32.

[0069] Step S112: Determine the sliding step length based on the preset overlap rate and division size;

[0070] To ensure the continuity of features between adjacent sub-regions and to avoid incomplete or lost features due to small targets being located exactly at the boundary, the overlap rate is usually set in the range of [30%, 50%], for example, 40%. This empirical setting ensures that the same small target has a high probability of being completely covered by at least one sub-region, while avoiding the generation of too many redundant sub-regions due to excessive overlap, thus achieving a good balance between detection recall and computational efficiency.

[0071] The sliding step size can be calculated using the formula S'=P×(1-C), where S' is the sliding step size, P is the division size, and C is the overlap ratio. For example, if C=40%, S'=P×(1-40%)=60%P=0.6P.

[0072] Step S113: Divide the image to be detected into a sliding window with a sliding step size to generate multiple overlapping sub-regions, and record the coordinates of the upper left corner of each sub-region in the image to be detected as the position information of the corresponding sub-region; wherein, the sliding window adopts a square region with a side length equal to the division size;

[0073] For example, suppose the dimensions of the image to be detected are: width W = 1000 pixels, height H = 800 pixels, P = 640 pixels (i.e., the sub-region is a square with a side length equal to P), C = 40%, and S' = P × (1 - C) = 640 × (1 - 40%) = 384 pixels. ① Starting from the top left corner (0,0) of the image to be detected, slide in the horizontal and vertical directions at a distance of S' = 384 pixels. ② Calculate the coordinates of the top left corner of all possible sliding windows: The starting X coordinates in the horizontal direction are 0, 384, and 768 in sequence (because when the starting X=768, the sliding window theoretically covers the X range [768,1407], which exceeds the image width W=1000); The starting Y coordinates in the vertical direction are 0, 384, and 768 in sequence (similarly, when the starting Y=768, the sliding window theoretically covers the Y range [768,1407], which exceeds the image height H=800); The coordinates of the top left corner of all generated sub-regions (3×3=9 in total) are: (0,0), (384,0), (768,0), (0,384), (384,384), (768,384), (0,768), (384,768), (768,768), and each sub-region is theoretically a square region of 640×640 pixels.

[0074] Step S114: When the divided sub-region exceeds the boundary of the image to be detected, the actual size of the sub-region is determined with the boundary of the image to be detected as the limit.

[0075] Continuing with the previous example, the sub-region with the top left corner at (768,0) has the following theoretical coverage: X∈[768,1407], Y∈[0,639]; image boundaries: width W=1000 pixels, height H=800 pixels; actual effective range: the maximum value in the X direction is 999, so the actual width is 1000-768=232 pixels; the Y direction is completely within the image (639<800), and the height remains at 640 pixels; therefore, the actual size of this sub-region is 232×640 pixels.

[0076] The sub-region with the top left corner at (0, 768): Theoretical coverage: X∈[0, 639], Y∈[768, 1407]; Actual effective range: The X direction is completely within the image, with a width of 640 pixels; The Y direction reaches a maximum of 799, with an actual height of 800-768=32 pixels; Therefore, the actual size of this sub-region is 640×32 pixels.

[0077] The sub-region with the top left corner at (768, 768): Theoretical coverage: X∈[768, 1407], Y∈[768, 1407]; Actual effective range: width is 232 pixels, height is 32 pixels; Therefore, the actual size of this sub-region is 232×32 pixels.

[0078] The remaining six sub-regions, being entirely located within the image, retain an actual size of 640×640 pixels.

[0079] In this invention, multiple sub-regions are generated by a sliding window, and adaptive cropping is performed on the sub-regions at the boundaries to ensure that all sub-regions are effectively aligned with the image to be detected, avoiding the introduction of invalid filling areas, thereby maintaining processing efficiency while ensuring the integrity of small target coverage.

[0080] Step S12: For each sub-region, perform the following processing: extract features and perform multi-scale fusion on the sub-region to generate a high-resolution fused feature map; based on each spatial location of the high-resolution fused feature map, generate the geometric parameters of the candidate boxes for small targets and the corresponding initial confidence scores; perform preliminary screening on the geometric parameters and corresponding initial confidence scores of each candidate box to obtain the candidate box set for that sub-region.

[0081] Step S12 involves feature extraction and multi-scale fusion of the sub-regions to generate a high-resolution fused feature map, including:

[0082] Step S120: Using a preset feature extraction network, extract multi-scale features from the sub-region, wherein the multi-scale features include at least the first scale feature with the largest spatial size, the second scale feature with the middle spatial size, and the third scale feature with the smallest spatial size.

[0083] To adapt to the trade-offs between computing resources and detection accuracy required in different application scenarios, the feature extraction network can be flexibly configured with a suitable backbone network architecture based on actual deployment conditions, without any limitations here. Specifically, in lightweight or edge computing scenarios, a backbone network with high computational efficiency and few parameters (such as MobileNetV3) can be selected to ensure real-time detection; while in scenarios with high requirements for detection accuracy, a deeper backbone network with stronger feature representation capabilities (such as ResNet-50) can be selected to improve the ability to capture subtle features of small targets.

[0084] In a preferred embodiment of the present invention, the multi-scale features include at least a first-scale feature downsampled by 4 times, a second-scale feature downsampled by 8 times, and a third-scale feature downsampled by 16 times using a preset feature extraction network.

[0085] Step S121: Upsample the second-scale features and the third-scale features so that the spatial dimensions of the second-scale features and the third-scale features are aligned with the spatial dimensions of the first-scale features;

[0086] Step S122: The first scale feature is fused with the second and third scale features after upsampling and alignment according to the preset weights to generate a high-resolution fused feature map of the sub-region.

[0087] Optionally, those skilled in the art can flexibly set the weights corresponding to features at each scale according to actual needs, and no limitation is made here. Specifically, a predefined fixed weight combination can be used (the first scale feature, the second scale feature, and the third scale feature are weighted in a ratio of 0.5:0.3:0.2), or a lightweight attention mechanism module (such as a channel attention module) can be introduced, so that the network can dynamically learn and generate adaptive weights for features at each scale based on the input content, thereby achieving better feature selection and fusion.

[0088] For example, assuming the input sub-region is 640×640 pixels in size, after processing by a pre-defined feature extraction network, feature maps at three scales are obtained: First scale feature map (downsampled by 4 times): spatial size 160×160 pixels, number of channels 256. This feature map retains relatively rich spatial details; Second scale feature map (downsampled by 8 times): spatial size 80×80 pixels, number of channels 512; Third scale feature map (downsampled by 16 times): spatial size 40×40 pixels, number of channels 1024.

[0089] To perform the fusion, the second-scale features and the third-scale features are first upsampled: the second-scale features (80×80 pixels) are upsampled by 2 times through bilinear interpolation or transposed convolution, so that their spatial size becomes 160×160 pixels; the third-scale features (40×40 pixels) are upsampled by 4 times, so that their spatial size also becomes 160×160 pixels.

[0090] Next, the three spatially aligned features (all 160×160) are weighted and fused according to preset weights. For example, a typical weighting setting is: 0.5 for the first-scale feature, 0.3 for the second-scale feature, and 0.2 for the third-scale feature. The fusion process can be expressed as: F = (0.5 × F1) + (0.3 × F2) / (0.5 × F1) = 0.5 × F1 / (0.3 × F2 ... 2up )+(0.2×F 3up ); where F is the high-resolution fused feature map, F1 is the first-scale feature, F 2up and F 3up These are the second and third scale features after upsampling. After weighted summation, a high-resolution fused feature map with a spatial size of 160×160 pixels is finally generated. This feature map simultaneously gathers detailed information from the shallow layer, structural information from the middle layer, and semantic information from the deep layer, providing a crucial feature foundation for subsequent prediction of small target candidate boxes at fine spatial locations.

[0091] In step S12, for each spatial location based on the high-resolution fused feature map, the geometric parameters of the candidate bounding box for the small target and the corresponding initial confidence score are generated, including:

[0092] Step S123: Map each spatial location of the high-resolution fused feature map to the image coordinate system of the sub-region to obtain the corresponding reference center point coordinates;

[0093] Optionally, the coordinates of the reference center point are calculated using the following formula: cx base =(j+0.5)×stride, cy base =(i+0.5)×stride, where cx base Let cy be the x-coordinate of the reference center point. base Let i be the ordinate of the reference center point, and i and j be the row and column indices of a spatial location in the high-resolution fused feature map (where the row index corresponds to the ordinate direction and the column index corresponds to the abscissa direction, in pixels). stride is the downsampling step size of the high-resolution fused feature map relative to the sub-region (in pixels).

[0094] Preferably, the downsampling step size (i.e., stride) of the high-resolution fused feature map relative to the sub-region is 4 or 8, etc. Those skilled in the art can choose flexibly according to actual needs, and there is no limitation here.

[0095] Step S124: Using a preset detection network, predict a set of target parameters based on the features of each spatial location, wherein the target parameters include the probability of target presence, center point offset, width adjustment, and height adjustment.

[0096] The detection network can be an existing YOLOv8 Detection Head, and those skilled in the art can choose flexibly according to actual needs; no limitation is made here. Specifically, if a YOLOv8 Detection Head is used, the processing procedure is as follows: First, channel adaptation is performed, using 1×1 convolutions to convert the number of channels in the high-resolution fused feature map into the number of channels required for "probability prediction" and "spatial parameter prediction," respectively; then, probability prediction is performed, and the output of this part is activated by the Sigmoid function to obtain the probability of the target's existence, which belongs to [0,1]; simultaneously, spatial parameter prediction is performed, directly outputting the center point offset, width adjustment, and height adjustment of the candidate box for small targets. It should be noted that probability prediction and spatial parameter prediction are two parallel network branches.

[0097] Step S125: Based on the reference center point coordinates and target parameters, generate the geometric parameters of the candidate box for each small target at each spatial location, wherein the geometric parameters include the center point coordinates, width and height of the candidate box in the sub-region;

[0098] Optionally, the geometric parameters of the candidate bounding box for each small target at each spatial location are calculated using the following formula: cx local =cx base +dx×stride, cy local =cy base +dy×stride,w local =exp(dw)×w base h local =exp(dh)×h base ; where cx local cx is the x-coordinate of the center point of the candidate box in the sub-region. base Let cy be the x-coordinate of the reference center point. local cy is the y-coordinate of the center point of the candidate box in the sub-region. base Let dx and dy be the ordinate of the reference center point, dy be the center point offset (both are dimensionless relative offsets), stride be the downsampling step size of the high-resolution fused feature map relative to the sub-region, and w be the center point offset. local and h local w represents the width and height of the candidate box within the sub-region, respectively. base and h baseThese are the preset baseline width and height, respectively, where dw is the width adjustment amount and dh is the height adjustment amount.

[0099] Step S126: Use the predicted probability of the target's existence as the initial confidence level of the candidate box.

[0100] Assuming the input sub-region has a size of 640×640 pixels, the high-resolution fused feature map generated after step S122 has a size of 160×160 pixels (i.e., stride=4). For a spatial location (i,j) on this high-resolution fused feature map, such as (10,20), the preset baseline width and height w... base =h base =16 pixels (preset according to the small target size range).

[0101] First, the spatial location (i,j) of the high-resolution fused feature map is mapped to the image coordinate system of the sub-region to obtain the corresponding reference center point coordinates: cx base =(20+0.5)×4=82.0 (pixels), cy base =(10+0.5)×4=42.0 (pixels).

[0102] Next, the preset detection network predicts a set of target parameters [pobj,dx,dy,dw,dh]=[0.85,0.1,−0.2,0.05,0.03] based on the feature vector at that location, where pobj is the probability of the target's presence, dx and dy are the center point offsets, and dw and dh are the width and height adjustments.

[0103] Then, based on the coordinates of the reference center point and the predicted target parameters, the geometric parameters of the candidate bounding boxes of the small targets corresponding to this spatial location are generated. It should be noted that if the geometric parameters of the candidate bounding boxes of the small targets corresponding to this spatial location are decimals, they need to be rounded to the nearest integer.

[0104] cx local =cx base +dx×stride=82.0+0.1×4=82.4 (pixels), rounded to cx local =82 (pixels);

[0105] cy local =cy base +dy×stride=42.0+(-0.2)×4=41.2 (pixels), rounded to cy local =41 (pixels);

[0106] w local =exp(dw)×w base=exp(0.05)×16≈16.82 (pixels), rounded to the nearest integer w local =17 (pixels);

[0107] h local =exp(dh)×h base =exp(0.03)×16≈16.48 (pixels), rounded to h local =16 (pixels).

[0108] Finally, the probability of the target's presence predicted by the network (pobj=0.85) is directly used as the initial confidence score for this candidate box. Thus, for the spatial location (10,20) in the high-resolution fused feature map, a small target candidate box with a center point located within the sub-region (82,41) pixels, a width of approximately 17 pixels, a height of approximately 16 pixels, and an initial confidence score of 0.85 is generated. This process is performed in parallel at each spatial location in the high-resolution fused feature map, thereby obtaining the geometric parameters of multiple candidate boxes and their corresponding initial confidence scores.

[0109] Step S12 involves preliminary screening of the geometric parameters and corresponding initial confidence scores of each candidate box to obtain a set of candidate boxes for the sub-region, including:

[0110] Step S127: Select geometric parameters from the candidate boxes whose width and height both conform to the preset range of small target size;

[0111] The range of small target sizes can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the range of small target sizes includes a range of small target width and a range of small target height, with the small target width range being [20, 64] and the small target height range being [20, 64].

[0112] Step S128: From the geometric parameters filtered by the small target size range, select the geometric parameters whose initial confidence level is greater than or equal to the preset initial confidence level threshold.

[0113] The initial confidence threshold can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the initial confidence threshold is 0.2.

[0114] Step S129: Combine the geometric parameters and corresponding initial confidence scores that simultaneously meet the two screening conditions of small target size range and initial confidence threshold to form a candidate box set for the sub-region.

[0115] Suppose that within a sub-region (e.g., sub-region A), there are four candidate bounding boxes (A1, A2, A3, A4) generated based on the high-resolution fused feature map. The geometric parameters of the four candidate bounding boxes, including their width and height within the sub-region and their corresponding initial confidence scores, are as follows:

[0116] A1: w A1 =25 pixels, h A1 =28 pixels, S A1 =0.75;

[0117] A2: w A2 =45 pixels, h A2 =50 pixels, S A2 =0.45;

[0118] A3: w A3 =15 pixels, h A3 =18 pixels, S A3 =0.55;

[0119] A4: w A4 =60 pixels, h A4 =65 pixels, S A4 =0.60.

[0120] The preset small target size range includes the small target width range and the small target height range. The small target width range is [20, 64] and the small target height range is [20, 64]. The preset initial confidence threshold is: θ1 = 0.2.

[0121] First, from the geometric parameters of the four candidate boxes A1, A2, A3, and A4, those whose width and height both conform to the small target size range are selected. Among them, candidate boxes A1 and A2 have both width and height conforming to the small target size range and proceed to the next step of selection; candidate boxes A3 and A4 have both width and height not conforming to the small target size range and are therefore eliminated.

[0122] Next, from the geometric parameters filtered by the small target size range, candidate boxes with an initial confidence level greater than or equal to a preset initial confidence threshold (θ1=0.2) are selected. Among them, the initial confidence levels of candidate boxes A1 and A2 are both greater than the initial confidence threshold (i.e., S). A1 =0.75>0.2、S A2 =0.45>0.2), therefore, both candidate boxes A1 and A2 are retained.

[0123] Finally, the geometric parameters and corresponding initial confidence scores of the two candidate boxes A1 and A2 that simultaneously meet the two screening conditions of small target size range and initial confidence threshold are used to form the candidate box set for this sub-region.

[0124] Through the above dual screening mechanism, candidate boxes with incorrect sizes or low confidence are effectively filtered out, ensuring that the candidate boxes output by each sub-region have high quality and reliability, laying a good foundation for subsequent global mapping and fusion.

[0125] Step S13: Based on the location information of each sub-region, map the candidate boxes in the candidate box set of all sub-regions to the global coordinate system of the image to be detected, and merge the duplicate candidate boxes that point to the same small target after mapping to form a global candidate box set;

[0126] Optionally, such as Figure 1c As shown, step S13 specifically includes:

[0127] Step S131: For each candidate box set of a sub-region, obtain the coordinates of the upper left corner of the sub-region in the image to be detected, and convert the center point coordinates of each candidate box in the candidate box set in the sub-region into the global center point coordinates in the global coordinate system of the image to be detected based on the coordinates.

[0128] Assuming the image to be detected has dimensions W = 1000 pixels wide and H = 800 pixels high, there are two overlapping sub-regions: the coordinates of the top-left corner of sub-region A in the image to be detected are (tile...). xA ,tile yA = (0,0); The coordinates of the top left corner of sub-region B in the image to be detected are (tile) xB ,tile yB = (320, 0). After processing in step S12, the candidate box sets of the two sub-regions respectively contain the following candidate boxes:

[0129] Candidate bounding boxes for sub-region A:

[0130] Candidate box A1: cx localA1 =350, cy localA1 =120, w A1 =25, h A1 =28, S A1 =0.75;

[0131] Candidate box A2: cx localA2 =300, cy localA2 =200, w A2 =45, h A2 =50, S A2 =0.45.

[0132] The candidate bounding box set for sub-region B:

[0133] B1 candidate box: cx localB1 =30, cy localB1=115, w B1 =24, h B1 =30, S B1 =0.80 (Candidate box B1 and candidate box A1 actually point to the same small target);

[0134] B2 candidate box: cx localB2 =400, cy localB2 =300, w B2 =30, h B2 =35, S B2 =0.60.

[0135] Add the coordinates of the center point of each candidate box within a sub-region to the coordinates of the top-left corner of that sub-region, and then transform them to the global center point coordinates in the global coordinate system of the image to be detected:

[0136] Global center coordinates of candidate box A1: cx globalA1 =cx localA1 +tile xA =350+0=350, cy globalA1 =cy localA1 +tile yA =120+0=120.

[0137] Global center coordinates of candidate box A2: cx globalA2 =300+0=300, cy globalA2 =200+0=200.

[0138] Global center coordinates of candidate box B1: cx globalB1 =cx locaB1 +tile xB =30+320=350, cy globalB1 =cy locaB1 +tile yB =115+0=115.

[0139] Global center coordinates of candidate box B2: cx globalB2 =400+320=720, cy globalB2 =300+0=300.

[0140] Step S132: For all candidate boxes transformed to the global coordinate system, if the intersection-union ratio between any two candidate boxes is greater than or equal to the first preset threshold, and / or the normalized center distance between any two candidate boxes is less than or equal to the dynamic distance threshold, it is determined that the two candidate boxes point to the same small target.

[0141] The first preset threshold can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the first preset threshold is 0.3.

[0142] The dynamic distance threshold can be calculated using the following formula: ;in, This is a dynamic distance threshold; The second preset threshold can be flexibly set by those skilled in the art according to actual needs, such as... ; and These represent the width and height of the first candidate box in any two candidate boxes within the sub-region, respectively. and These are the width and height of the second candidate box in any two candidate boxes within the sub-region, respectively.

[0143] Calculate the metrics between candidate boxes A1 and B1:

[0144] ① Intersection over Union (IoU): The global center point coordinates of candidate box A1 are (350, 120), width is 25, and height is 28; the global center point coordinates of candidate box B1 are (350, 115), width is 24, and height is 30. The ratio of the intersection to the union area of ​​the two candidate boxes is calculated. The calculated IoU(A1,B1)≈0.68. Since 0.68≥0.3, the IoU condition is satisfied.

[0145] ② Normalized central distance:

[0146] Normalized center distance between candidate boxes A1 and B1: ; Calculate the dynamic distance threshold: ; Therefore, the normalized center distance condition is satisfied.

[0147] Since candidate boxes A1 and B1 satisfy the IoU condition (and also the normalized center distance condition), it is determined that candidate boxes A1 and B1 point to the same small target.

[0148] The cross-union ratios between candidate boxes A2 and B1 and B2, as well as between candidate box B2 and A1, are all less than 0.3 after calculation, and the normalized center distances also exceed the dynamic distance threshold. Therefore, they are determined to be small targets pointing to different targets.

[0149] It should be noted that if either the intersection-union ratio (IU) or the normalized center distance (NDM) condition is met, the two candidate boxes can be determined to point to the same small target. In other words, the two candidate boxes will only be determined to point to different small targets if neither condition is met.

[0150] Step S133: For candidate boxes that are determined to point to the same small target, calculate the weighted average of their respective global center point coordinates, width and height according to their initial confidence, and use the highest initial confidence among the merged geometric parameters as the initial confidence of the merged candidate box.

[0151] Optionally, the merged geometric parameters in step S133 are calculated using the following formula: , , , ;in, and The coordinates of the global center point of the merged candidate M boxes. and To determine if they are pointing to the same small target The global center coordinates of the candidate boxes, n is the number of candidate boxes that are determined to point to the same small target, and S is the global center coordinates of the candidate boxes. k To determine if they are pointing to the same small target Initial confidence level of candidate boxes. This is the width of the merged candidate box M. To determine if they are pointing to the same small target The width of the candidate box within the sub-region. The height of the merged candidate box M. To determine if they are pointing to the same small target The height of the candidate box within the sub-region.

[0152] Candidate boxes A1 and B1, which are determined to point to the same small target, are merged using a weighted average. The weights are their respective initial confidence scores. The merged global center point coordinates are: cx mergedM1 =(350×0.75+350×0.80) / (0.75+0.80)≈349.68; cy mergedM1= (120×0.75+115×0.80) / (0.75+0.80)≈117.42. The combined width and height: w mergedM1 =(25×0.75+24×0.80) / (0.75+0.80)≈24.48; h mergedM1 =(28×0.75+30×0.80) / (0.75+0.80)≈29.03. Initial confidence level after merging: Take the highest initial confidence level between candidate boxes A1 and B1, i.e., S. mergedM1 =0.80. After merging, a new candidate box is formed, namely M1.

[0153] Step S134: Combine all candidate boxes after merging and their corresponding initial confidence scores to form a global candidate box set.

[0154] The candidate boxes A2 and B2 that were not merged, along with the newly merged candidate box M1, are combined to form the global candidate box set:

[0155] Candidate box M1 (merged from A1 and B1): cx mergedM1 ≈349.68, cy mergedM1 ≈117.42, w mergedM1 ≈24.48, h mergedM1 ≈29.03, S mergedM1 =0.80;

[0156] Candidate box A2: cx localA2 =300, cy localA2 =200, w A2 =45, h A2 =50, S A2 =0.45;

[0157] B2 candidate box: cx localB2 =720, cy localB2 =300, w B2 =30, h B2 =35, S B2 =0.60.

[0158] Step S14: For each candidate box in the global candidate box set, perform fine-grained discrimination by cropping the corresponding local image region in the image to be detected, and obtain the refined confidence score;

[0159] Optionally, step S14 specifically includes:

[0160] Step S141: Using the global center point coordinates of each candidate box in the global candidate box set as the center, and according to the preset expansion coefficient, crop out the local image region including context information from the image to be detected.

[0161] Optionally, when the final size of the candidate box is less than or equal to 32×32 (i.e. When k=3.0; when the final size of the candidate box is greater than 32×32 (i.e., ... When k=1.5, (where k is the maximum value). and These are the final width and final height of the candidate box.

[0162] It should be noted that if two candidate boxes are determined to point to the same small target, the final width and final height of the candidate boxes are merged to form the height and width of the new candidate box. Taking candidate box M1 as an example, the final width and final height of candidate box M1 are w. mergedM1 and h mergedM1If the candidate boxes are not merged, the final width and final height of the candidate boxes are the width and height of the candidate boxes in the sub-regions, respectively.

[0163] Continuing the previous example, using the global center point coordinates (349.68, 117.42) of the candidate box M1 as the center, and based on the preset expansion coefficient k=3.0 (used for small targets to include richer contextual information), calculate the local image region including contextual information to be cropped from the image to be detected; where the width of the local image region is: w crop =w mergedM1 ×k = 24.48 × 3.0 = 73.44 (pixels), take 73 pixels; Height of the local image region: h crop =h mergedM1 ×k = 29.03 × 3.0 = 87.09 (pixels), take 87 pixels. Using the global center point coordinates (349.68, 117.42) of the candidate box as the center, directly crop a rectangular local image region with a width of 73 pixels and a height of 87 pixels from the image to be detected. This region completely covers the candidate box itself and includes its surrounding context information.

[0164] Step S142: Resample the cropped local image region to a preset uniform size;

[0165] The preset uniform size can be 64×64, and those skilled in the art can choose flexibly according to actual needs, without limitation here.

[0166] Step S143: Input a local image region of uniform size into a preset lightweight discriminant network, and have the lightweight discriminant network output a refined confidence score representing the probability that the local image region contains a real small target.

[0167] The preset lightweight discrimination network can be the existing MobileNetV3. Of course, those skilled in the art can flexibly choose other networks according to actual needs, which is not limited here.

[0168] Step S15: Fuse the initial confidence and refined confidence of each candidate box to obtain the comprehensive confidence, and output the final small target detection result based on the comparison result of the comprehensive confidence and the adaptive threshold related to the target size.

[0169] Optionally, step S15 specifically includes:

[0170] Step S151: For each candidate box, apply a dynamic weighted fusion formula. Calculate the overall confidence level, where, To assess the overall confidence level, As the initial confidence level, To refine the confidence level, and These are the first and second weighting coefficients;

[0171] It should be noted that when the final size of the candidate box is less than or equal to 32×32 (i.e., ... )hour, , When the final size of the candidate box is greater than 32×32 (i.e. )hour, , .in, and These are the final width and final height of the candidate box.

[0172] Step S152: Calculate the adaptive decision threshold for each candidate box, wherein the adaptive decision threshold is calculated using the following formula: ,in, For adaptive decision threshold, and The final width and final height of the candidate box (in pixels). This represents the downsampling step size (in pixels) of the high-resolution fused feature map relative to the sub-region. Indicates rounding down;

[0173] Step S153: Determine whether the overall confidence score is greater than or equal to the adaptive decision threshold; if the overall confidence score is greater than or equal to the adaptive decision threshold, output the candidate box and its overall confidence score as the final small object detection result; if the overall confidence score is less than the adaptive decision threshold, filter out the candidate box.

[0174] In an optional embodiment of the present invention, the method further includes:

[0175] Step S16: Obtain the size information of the original image;

[0176] Step S17: If the ratio of the width to the height of the original image is greater than the first preset ratio, the original image is cropped into multiple sub-images, wherein the ratio of the width to the height of each sub-image is less than or equal to the second preset ratio.

[0177] The first preset ratio and the second preset ratio can be flexibly selected by those skilled in the art according to actual needs, and are not limited here. Preferably, the first preset ratio is 5 and the second preset ratio is 3.

[0178] Step S18: For the original image or each sub-image obtained after cropping, determine whether the maximum side length of the original image or each sub-image obtained after cropping is greater than a preset size threshold.

[0179] The preset size threshold can be flexibly set by those skilled in the art according to actual needs, and is not limited here, such as 1280.

[0180] Step S19: If the maximum side length of the original image or the cropped sub-image is greater than the preset size threshold, scale the original image or the cropped sub-image proportionally so that the maximum side length of the scaled original image or the cropped sub-image is equal to the preset size threshold, and use the scaled original image or the cropped sub-image as the image to be detected.

[0181] The scaling factor used for proportional scaling can be calculated using the following formula: ,in, Scaling factor The width of the original image or the cropped sub-image. The height of the original image or the cropped sub-image. This is a preset size threshold.

[0182] Step S20: If the maximum side length of the original image or the cropped sub-image is less than or equal to the preset size threshold, it is directly used as the image to be detected.

[0183] For video stream input, the preset size threshold is determined based on the maximum side length of the first frame of the original image in the video stream, and the same scaling ratio is applied to the original images of subsequent frames.

[0184] It should be noted that steps S121 to S129, S141 to S143, and S16 to S20 are for ease of description only and are not shown in the figure.

[0185] The small target detection method based on high-resolution fused feature maps provided by this invention significantly improves the perception capability and coverage density of small target features by introducing high-resolution fused feature maps to generate candidate boxes point by point and combining them with an overlapping sub-region segmentation strategy. This effectively overcomes the problem of missed detection caused by small target size and weak features. Furthermore, through global coordinate system mapping and spatial consistency fusion mechanism, it can stably merge duplicate detection results from different sub-regions, avoiding the false deletion of valid targets that may be caused by traditional non-maximum suppression strategies, and enhancing the robustness of the detection results. In addition, it also uses local resampling and a lightweight discriminant network for fine verification, and combines dynamic fusion of initial confidence and refined confidence with scale-adaptive decision-making to significantly reduce the false alarm rate in complex backgrounds.

[0186] Based on the same inventive concept, embodiments of the present invention also provide a small target detection device based on high-resolution fusion feature maps for implementing the small target detection method based on high-resolution fusion feature maps described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more embodiments of the small target detection device based on high-resolution fusion feature maps provided below can be found in the limitations of the small target detection method based on high-resolution fusion feature maps described above, and will not be repeated here.

[0187] like Figure 2 As shown, this invention provides a small target detection device based on high-resolution fused feature maps, comprising: a region segmentation module 21, a sub-region processing module 22, a global fusion module 23, a fine discrimination module 24, and a result output module 25; wherein,

[0188] The region segmentation module 21 is used to segment the image to be detected into multiple overlapping sub-regions and record the position information of each sub-region in the image to be detected.

[0189] The sub-region processing module 22, connected to the region partitioning module 21, is used to perform the following processing on each sub-region: extract features and perform multi-scale fusion on the sub-region to generate a high-resolution fused feature map; generate the geometric parameters and corresponding initial confidence scores of candidate boxes for small targets based on each spatial location of the high-resolution fused feature map; and perform preliminary screening on the geometric parameters and corresponding initial confidence scores of each candidate box to obtain a set of candidate boxes for the sub-region.

[0190] The global fusion module 23, connected to the sub-region processing module 22, is used to map the candidate boxes in the candidate box set of all sub-regions to the global coordinate system of the image to be detected based on the position information of each sub-region, and merge the duplicate candidate boxes that point to the same small target after mapping to form a global candidate box set.

[0191] The fine discrimination module 24, connected to the global fusion module 23, is used to perform fine discrimination on each candidate box in the global candidate box set, cropping the corresponding local image region in the image to be detected, and obtaining the fine-tuned confidence score.

[0192] The result output module 25, connected to the fine discrimination module 24, is used to fuse the initial confidence and refined confidence of each candidate box to obtain the comprehensive confidence. Based on the comparison result of the comprehensive confidence and the adaptive threshold related to the target size, the final small target detection result is output.

[0193] Optionally, the device further includes: an image acquisition module (not shown in the figure), connected to the region division module 21, used to acquire the size information of the original image; if the ratio of the width to the height of the original image is greater than a first preset ratio, the original image is cropped into multiple sub-images, wherein the ratio of the width to the height of each sub-image is less than or equal to a second preset ratio; for the original image or each sub-image obtained after cropping, it is determined whether the maximum side length of the original image or each sub-image obtained after cropping is greater than a preset size threshold; if the maximum side length of the original image or the sub-image obtained after cropping is greater than the preset size threshold, the original image or the sub-image obtained after cropping is scaled proportionally so that the maximum side length of the scaled original image or the sub-image obtained after cropping is equal to the preset size threshold, and the scaled original image or the sub-image obtained after cropping is used as the image to be detected; if the maximum side length of the original image or the sub-image obtained after cropping is less than or equal to the preset size threshold, it is directly used as the image to be detected; wherein, for input in the form of a video stream, the preset size threshold is determined according to the maximum side length of the first frame of the original image in the video stream, and the same scaling ratio is applied to the subsequent frames of the original image for processing.

[0194] Optionally, the region segmentation module 21 is specifically used for: determining the segmentation size of the sub-region based on the preset maximum size of the small target; determining the sliding step size based on the preset overlap rate and segmentation size; performing sliding window segmentation on the image to be detected with the sliding step size to generate multiple overlapping sub-regions, and recording the coordinates of the upper left corner of each sub-region in the image to be detected as the position information of the corresponding sub-region; wherein, the sliding window adopts a square region with a side length equal to the segmentation size; when the segmented sub-region exceeds the boundary of the image to be detected, the actual size of the sub-region is determined with the boundary of the image to be detected as the limit.

[0195] Optionally, the sub-region processing module 22 performs feature extraction and multi-scale fusion on the sub-region to generate a high-resolution fused feature map, including: using a preset feature extraction network to extract multi-scale features from the sub-region, wherein the multi-scale features include at least a first-scale feature with the largest spatial size, a second-scale feature with a medium spatial size, and a third-scale feature with the smallest spatial size; upsampling the second-scale feature and the third-scale feature so that the spatial sizes of the second-scale feature and the third-scale feature are aligned with the spatial size of the first-scale feature; and fusing the first-scale feature with the upsampled and aligned second-scale feature and third-scale feature according to preset weights to generate a high-resolution fused feature map of the sub-region.

[0196] Optionally, in the sub-region processing module 22, for each spatial location based on the high-resolution fused feature map, the geometric parameters of the candidate bounding box for the small target and the corresponding initial confidence score are generated, including: mapping each spatial location of the high-resolution fused feature map to the image coordinate system of the sub-region to obtain the corresponding reference center point coordinates; predicting a set of target parameters based on the features of each spatial location through a preset detection network, wherein the target parameters include the target existence probability, center point offset, width adjustment, and height adjustment; generating the geometric parameters of the candidate bounding box for the small target corresponding to each spatial location based on the reference center point coordinates and the target parameters, wherein the geometric parameters include the center point coordinates, width, and height of the candidate bounding box in the sub-region; and using the predicted target existence probability as the initial confidence score of the candidate bounding box.

[0197] Optionally, the sub-region processing module 22 performs preliminary screening of the geometric parameters and corresponding initial confidence scores of each candidate box to obtain a candidate box set for the sub-region. This includes: selecting geometric parameters whose width and height both conform to a preset small target size range from the geometric parameters of each candidate box; selecting geometric parameters whose corresponding initial confidence scores are greater than or equal to a preset initial confidence threshold from the geometric parameters after small target size range screening; and forming the candidate box set for the sub-region with the geometric parameters that simultaneously meet both the small target size range and the initial confidence threshold screening conditions and their corresponding initial confidence scores.

[0198] Optionally, the global fusion module 23 is specifically used for: for each sub-region's candidate box set, obtaining the coordinates of the upper left corner of the sub-region in the image to be detected, and converting the center point coordinates of each candidate box in the candidate box set in the sub-region into global center point coordinates in the global coordinate system of the image to be detected based on the coordinates; for all candidate boxes converted to the global coordinate system, if the intersection-union ratio between any two candidate boxes is greater than or equal to a first preset threshold, and / or the normalized center distance between any two candidate boxes is less than or equal to a dynamic distance threshold, it is determined that the two candidate boxes point to the same small target; for candidate boxes determined to point to the same small target, the weighted average of their respective global center point coordinates, width, and height is calculated according to their initial confidence, and the highest initial confidence among the merged geometric parameters is used as the initial confidence of the merged candidate box; and all candidate boxes after merging and their corresponding initial confidence are combined to form a global candidate box set.

[0199] Optionally, the fine discrimination module 24 is specifically used to: crop out a local image region including contextual information from the image to be detected, centered on the global center point coordinates of each candidate box in the global candidate box set, according to a preset expansion coefficient; resample the cropped local image region to a preset uniform size; input the uniform-sized local image region into a preset lightweight discrimination network, and have the lightweight discrimination network output a refined confidence score representing the probability that the local image region includes a real small target.

[0200] Optionally, the result output module 25 is specifically used to: for each candidate box, employ a dynamically weighted fusion formula. Calculate the overall confidence level, where, To assess the overall confidence level, As the initial confidence level, To refine the confidence level, and Let be the first weight coefficient and the second weight coefficient; calculate the adaptive decision threshold for each candidate box, where the adaptive decision threshold is calculated using the following formula: ,in, For adaptive decision threshold, and The final width and final height of the candidate box. The downsampling step size of the high-resolution fused feature map relative to the sub-region. This indicates rounding down; it checks if the overall confidence score is greater than or equal to the adaptive decision threshold; if the overall confidence score is greater than or equal to the adaptive decision threshold, the candidate box and its overall confidence score are output as the final small object detection result; if the overall confidence score is less than the adaptive decision threshold, the candidate box is filtered out.

[0201] The small target detection device based on high-resolution fused feature maps provided by this invention significantly improves the perception capability and coverage density of small target features by introducing high-resolution fused feature maps to generate candidate boxes point by point and combining them with an overlapping sub-region segmentation strategy. This effectively overcomes the problem of missed detection caused by small target size and weak features. Furthermore, through a global coordinate system mapping and spatial consistency fusion mechanism, it can stably merge duplicate detection results from different sub-regions, avoiding the false deletion of valid targets that may be caused by traditional non-maximum suppression strategies, thus enhancing the robustness of the detection results. In addition, it uses local resampling and a lightweight discriminant network for fine verification, and combines dynamic fusion of initial confidence and refined confidence with scale-adaptive decision-making to significantly reduce the false alarm rate in complex backgrounds.

[0202] It should be noted that "multiple" in this invention includes two or more.

[0203] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0204] Each module in the devices of this invention can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0205] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data required for or generated by the aforementioned small target detection method based on high-resolution fused feature maps. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a small target detection method based on high-resolution fused feature maps.

[0206] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a small target detection method based on high-resolution fused feature maps. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0207] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0208] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0209] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0210] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0211] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties.

[0212] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0213] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A small target detection method based on high-resolution fused feature maps, characterized in that, include: The image to be detected is divided into regions to obtain multiple overlapping sub-regions, and the position information of each sub-region in the image to be detected is recorded. For each sub-region, the following processing is performed: feature extraction and multi-scale fusion are performed on the sub-region to generate a high-resolution fused feature map; based on each spatial location of the high-resolution fused feature map, the geometric parameters of candidate boxes for small targets and their corresponding initial confidence scores are generated; the geometric parameters and corresponding initial confidence scores of each candidate box are initially screened to obtain the candidate box set for that sub-region. Based on the location information of each sub-region, the candidate boxes in the candidate box set of all sub-regions are mapped to the global coordinate system of the image to be detected, and duplicate candidate boxes that point to the same small target after mapping are merged to form a global candidate box set; For each candidate box in the global candidate box set, the corresponding local image region is cropped in the image to be detected for fine discrimination to obtain the refined confidence score; The initial and refined confidence scores of each candidate box are combined to obtain a comprehensive confidence score. Based on the comparison between the comprehensive confidence score and an adaptive threshold related to the target size, the final small target detection result is output. The method further includes: Obtain the size information of the original image; If the ratio of the width to the height of the original image is greater than the first preset ratio, the original image is cropped into multiple sub-images, wherein the ratio of the width to the height of each sub-image is less than or equal to the second preset ratio. For each sub-image obtained from the original image or the cropped image, determine whether the maximum side length of the original image or each sub-image obtained from the cropped image is greater than a preset size threshold. If the maximum side length of the original image or the cropped sub-image is greater than the preset size threshold, the original image or the cropped sub-image is scaled proportionally so that the maximum side length of the scaled original image or the cropped sub-image is equal to the preset size threshold, and the scaled original image or the cropped sub-image is used as the image to be detected. If the maximum side length of the original image or the cropped sub-image is less than or equal to the preset size threshold, it will be directly used as the image to be detected. For video stream input, the preset size threshold is determined based on the maximum side length of the first frame of the original image in the video stream, and the same scaling ratio is applied to the original images of subsequent frames.

2. The method according to claim 1, characterized in that, The process involves dividing the image to be detected into multiple overlapping sub-regions and recording the positional information of each sub-region within the image, including: The sub-region division size is determined based on the preset maximum size of the small target; The sliding step size is determined based on the preset overlap rate and division size; A sliding window is used to divide the image to be detected by a sliding step size, generating multiple overlapping sub-regions. The coordinates of the upper left corner of each sub-region in the image to be detected are recorded as the position information of the corresponding sub-region. The sliding window is a square region with a side length equal to the division size. When the defined sub-region exceeds the boundary of the image to be detected, the actual size of the sub-region is determined with the boundary of the image to be detected as the limit.

3. The method according to claim 1, characterized in that, The process of extracting features from sub-regions and fusing them at multiple scales to generate a high-resolution fused feature map includes: A pre-defined feature extraction network is used to extract multi-scale features from the sub-region. The multi-scale features include at least the first scale feature with the largest spatial size, the second scale feature with the middle spatial size, and the third scale feature with the smallest spatial size. Upsample the second-scale and third-scale features so that the spatial dimensions of the second-scale and third-scale features are aligned with the spatial dimensions of the first-scale features. The first-scale features are fused with the upsampled and aligned second-scale and third-scale features according to preset weights to generate a high-resolution fused feature map of the sub-region.

4. The method according to claim 1, characterized in that, The geometric parameters and corresponding initial confidence scores for generating candidate bounding boxes for small targets at each spatial location based on the high-resolution fused feature map include: Each spatial location of the high-resolution fused feature map is mapped to the image coordinate system of the sub-region to obtain the corresponding reference center point coordinates; Using a pre-defined detection network, a set of target parameters are predicted based on the features of each spatial location. These target parameters include the probability of target presence, center point offset, width adjustment, and height adjustment. Based on the reference center point coordinates and target parameters, generate the geometric parameters of the candidate bounding box for each small target at each spatial location. The geometric parameters include the center point coordinates, width, and height of the candidate bounding box in the sub-region. The predicted probability of the target's existence is used as the initial confidence level for the candidate box.

5. The method according to claim 1, characterized in that, The preliminary screening of each candidate box's geometric parameters and corresponding initial confidence level yields a candidate box set for the sub-region, including: Select geometric parameters from the candidate bounding boxes whose width and height both conform to the preset range of small target sizes; From the geometric parameters filtered by the small target size range, select the geometric parameters whose initial confidence is greater than or equal to the preset initial confidence threshold. The geometric parameters and their corresponding initial confidence scores that simultaneously meet both the small target size range and the initial confidence threshold are used to form the candidate box set for the sub-region.

6. The method according to claim 1, characterized in that, The step involves mapping candidate boxes from the candidate box sets of all sub-regions to the global coordinate system of the image to be detected based on the location information of each sub-region, and merging duplicate candidate boxes pointing to the same small target after mapping to form a global candidate box set, including: For each candidate box set in a sub-region, obtain the coordinates of the top-left corner of the sub-region in the image to be detected, and based on the coordinates, convert the center point coordinates of each candidate box in the candidate box set in the sub-region into the global center point coordinates in the global coordinate system of the image to be detected; For all candidate boxes transformed to the global coordinate system, if the intersection-union ratio between any two candidate boxes is greater than or equal to the first preset threshold, and / or the normalized center distance between any two candidate boxes is less than or equal to the dynamic distance threshold, the two candidate boxes are determined to point to the same small target. For candidate boxes identified as pointing to the same small target, the global center point coordinates, width, and height of each box are weighted and averaged according to their initial confidence scores to calculate the merged geometric parameters. The highest initial confidence score among the merged geometric parameters is then used as the initial confidence score of the merged candidate box. All candidate boxes after merging and their corresponding initial confidence scores are combined to form a global candidate box set.

7. The method according to claim 6, characterized in that, The process of refining the discrimination of each candidate box in the global candidate box set by cropping the corresponding local image region in the image to be detected to obtain the refined confidence score includes: Using the global center point coordinates of each candidate box in the global candidate box set as the center, a local image region including context information is cropped from the image to be detected according to a preset expansion coefficient. The cropped local image region is resampled to a preset uniform size; A local image region of uniform size is input into a preset lightweight discriminant network, which outputs a refined confidence score representing the probability that the local image region contains a real small target.

8. The method according to claim 1, characterized in that, The initial and refined confidence scores of each candidate box are fused to obtain a comprehensive confidence score. Based on the comparison between the comprehensive confidence score and an adaptive threshold related to the target size, the final small target detection result is output, including: For each candidate box, a dynamic weighted fusion formula is used. Calculate the overall confidence level, where, To assess the overall confidence level, As the initial confidence level, To refine the confidence level, and These are the first and second weighting coefficients; Calculate the adaptive decision threshold for each candidate box, where the adaptive decision threshold is calculated using the following formula: ,in, For adaptive decision threshold, and For the final width and final height of the candidate box, The downsampling step size of the high-resolution fused feature map relative to the sub-region. Indicates rounding down; Determine whether the overall confidence score is greater than or equal to the adaptive decision threshold; if the overall confidence score is greater than or equal to the adaptive decision threshold, output the candidate box and its overall confidence score as the final small object detection result; if the overall confidence score is less than the adaptive decision threshold, filter out the candidate box.

9. A small target detection apparatus based on a high-resolution fused feature map for performing the method according to any one of claims 1-8, characterized in that, include: The region segmentation module is used to segment the image to be detected into multiple overlapping sub-regions and record the position information of each sub-region in the image to be detected. The sub-region processing module, connected to the region partitioning module, performs the following processing on each sub-region: feature extraction and multi-scale fusion of the sub-region to generate a high-resolution fused feature map; based on each spatial location of the high-resolution fused feature map, generating the geometric parameters and corresponding initial confidence scores of candidate boxes for small targets; and performing preliminary screening on the geometric parameters and corresponding initial confidence scores of each candidate box to obtain a set of candidate boxes for that sub-region. The global fusion module, connected to the sub-region processing module, is used to map the candidate boxes in the candidate box set of all sub-regions to the global coordinate system of the image to be detected based on the position information of each sub-region, and merge the duplicate candidate boxes that point to the same small target after mapping to form a global candidate box set. The fine discrimination module, connected to the global fusion module, is used to perform fine discrimination on each candidate box in the global candidate box set, cropping the corresponding local image region in the image to be detected, and obtaining the fine-tuned confidence score. The results output module, connected to the fine discrimination module, is used to fuse the initial confidence and refined confidence of each candidate box to obtain the comprehensive confidence. Based on the comparison between the comprehensive confidence and the adaptive threshold related to the target size, the final small target detection result is output.

Images