End side remote sensing image small target detection system and method based on YOLOE

Abstract

The invention provides an end-side remote sensing image small target detection system and method based on YOLOE, relates to the technical field of image data processing, and solves the technical problems of low detection speed and inaccurate small target feature capture when a YOLOE algorithm is directly applied to end-side remote sensing image small target detection in the prior art. The system specifically comprises an end-side adaptive backbone network, a thermodynamic diagram branch and a central non-maximum suppression unit, wherein the end-side adaptive backbone network is used for reducing the feature extraction calculation amount of a remote sensing image by adopting continuous small kernel convolution, meanwhile, an SE-Block channel attention mechanism is embedded to enhance the small target feature extraction precision, and feature representation is output; the thermodynamic diagram branch is used for predicting central point probability distribution of small targets, realizing one-to-one positioning of the targets, scoring target areas and outputting target center candidate information; the center non-maximum suppression unit is composed of MaxPool, Equal and Cast operators, is executed on the NPU and is used for filtering redundancy in target center candidate information. The method and the device are used for end-side small target detection.

Description

Technical Field

[0001] This application relates to the field of image data processing technology, and in particular to a small target detection system and method for edge remote sensing images based on YOLOE. Background Technology

[0002] Small target detection in satellite remote sensing images is crucial for accurate decision-making in fields such as urban planning, agricultural monitoring, and disaster early warning. Current technologies, such as traditional target detection methods, rely on manually designed features like SIFT and HOG. However, with the rise of deep learning, Faster R-... Algorithms based on convolutional neural networks, such as CNN, YOLO series, and SSD, have become mainstream. Among them, the YOLOE algorithm achieves open-set object detection by integrating three modes: text prompts, visual prompts, and no prompts, overcoming the limitation of traditional YOLO's reliance on predefined object categories. However, traditional handcrafted feature methods have poor robustness to complex backgrounds and noise, and limited detection accuracy and generalization ability. In deep learning algorithms, the large kernel convolutions and deep structures of traditional backbone networks easily lose detailed features of small objects. Furthermore, when the YOLOE algorithm is directly applied to small object detection in edge remote sensing images, its one-to-many prediction method generates a large number of candidate boxes. The lack of a dedicated NMS operator in the edge NPU leads to extremely slow post-processing speed. The reparameterization mechanism also requires re-deriving the model when the detected object changes, resulting in insufficient flexibility and real-time performance. At the same time, small objects are easily submerged by background noise due to their small size and low pixel ratio, making feature extraction difficult. Therefore, existing technologies suffer from the technical problems of slow detection speed and inaccurate small object feature capture when the YOLOE algorithm is directly applied to small object detection in edge remote sensing images. Summary of the Invention

[0003] This application provides a small target detection system and method based on YOLOE for edge remote sensing images, which solves the technical problems of slow detection speed and inaccurate small target feature capture when the YOLOE algorithm is directly applied to edge remote sensing images.

[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, a YOLOE-based edge-side remote sensing image small target detection system is provided, comprising: an edge-side adaptation backbone network, a heatmap branch, and a center non-maximum suppression unit; wherein, the edge-side adaptation backbone network is used to reduce the computational cost of feature extraction of remote sensing images by employing continuous small kernel convolution, while embedding an SE-Block channel attention mechanism to enhance the accuracy of small target feature extraction, and outputting feature representation; the heatmap branch is used to predict the probability distribution of the center point of small targets, realize one-to-one target localization and score the target region, and output target center candidate information; the center non-maximum suppression unit consists of MaxPool, Equal, and Cast operators, executed on the NPU, and is used to filter redundancy in the target center candidate information.

[0005] In conjunction with the first aspect mentioned above, in one possible implementation, the system further includes: a path aggregation network, a SAVPE encoder, and an RTA encoder; wherein, the path aggregation network is connected to the output of the end-side adaptation backbone network and is used to receive the feature representation of the end-side adaptation backbone network and output multi-scale features; the SAVPE encoder is used to extract and encode features from the reference image corresponding to the visual cue and output a visual cue feature vector; the RTA encoder is used to extract and encode features from the text information corresponding to the text cue and output a text cue feature vector.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, the consecutive small kernel convolutions are three serially connected 3×3 convolutions; the channel compression ratio of SE-Block is 0.25.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, the pooling kernel size of the MaxPool operator in the central nonmaximum suppression unit is 3×3, and it is executed sequentially in the order of MaxPool, Equal, and Cast operators.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, the heatmap branch, the SAVPE encoder, and the RTA encoder can operate in parallel.

[0009] Secondly, a method for detecting small targets in edge remote sensing images based on YOLOE is provided, including: acquiring remote sensing images; extracting features from the remote sensing images and outputting feature representations; fusing and filtering the feature representations at multiple scales to obtain target features; the target features are the features with the highest resolution; predicting the probability of the center point and scoring the target region based on the target features, and outputting target center candidate information; and filtering the target center candidate information for redundancy to obtain detection labels.

[0010] In conjunction with the second aspect above, in one possible implementation, the feature representation is subjected to multi-scale feature fusion and filtering to obtain the target feature, including: performing multi-scale feature fusion on the feature representation to generate P3, P4, and P5 multi-scale features; and filtering out the P3 feature with the highest resolution as the target feature.

[0011] In conjunction with the second aspect above, in one possible implementation, the target features are used to predict the probability of the center point and score the target region, and the target center candidate information is output. This includes: predicting the probability distribution of the center point of small targets based on the target features and generating a heat map; using a one-to-one prediction mechanism to assign a unique center point to each small target; using the pixel probability value in the heat map as the target region scoring result; and extracting the positions corresponding to pixels with scores higher than a preset threshold as target center candidate information.

[0012] In conjunction with the second aspect above, in one possible implementation, redundant filtering of the target center candidate information is performed to obtain detection labels, including: using a 3×3 pooling kernel to perform sliding window processing on the heatmap corresponding to the target center candidate information; retaining the maximum value within each window and setting the remaining pixel values ​​to 0 or minimum values; filtering to obtain the unique center point with the highest confidence for each small target and generating detection labels.

[0013] In conjunction with the second aspect above, in one possible implementation, the edge remote sensing image small target detection method further includes: if there is textual or visual cue information, performing feature encoding through the corresponding RTA encoder and SAVPE encoder to obtain a cue feature vector; and performing correlation matching between the cue feature vector and the redundant filtered target center candidate information to obtain a detection label.

[0014] This application provides a YOLOE-based edge-side remote sensing image small target detection system and method. By using continuous small kernel convolution and SE-Block channel attention mechanism in the edge-side adapted backbone network, the system enhances the accuracy of small target feature extraction while reducing the computational cost of feature extraction. Combined with one-to-one localization of the heatmap branch, redundant candidate boxes are reduced. Furthermore, the NPU-compatible center non-maximum suppression unit efficiently filters redundancy, significantly improving the real-time performance and accuracy of edge-side equipment in detecting small targets in remote sensing images. At the same time, it can flexibly adapt to and run in parallel with three modes: text prompts, visual prompts, and no prompts. It does not require model reparameterization and re-export due to changes in the detection object. It also adapts to the computing power requirements of edge-side NPU, effectively solving the problems of difficult feature extraction and slow detection speed of small targets in remote sensing images, while balancing detection accuracy and real-time performance.

[0015] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0016] Figure 1 A system architecture diagram of a YOLOE-based edge remote sensing image small target detection system is provided for embodiments of this application; Figure 2 A system architecture diagram of a central nonmaximum suppression unit provided in this application embodiment; Figure 3 A flowchart illustrating a YOLOE-based method for small target detection in edge remote sensing images, provided for an embodiment of this application; Figure 4 A flowchart illustrating another YOLOE-based method for small target detection in edge remote sensing images provided in this application embodiment; Figure 5 A flowchart illustrating another YOLOE-based method for small target detection in edge remote sensing images provided in this application embodiment; Figure 6 A flowchart illustrating another YOLOE-based method for small target detection in edge remote sensing images provided in this application embodiment; Figure 7 This is a flowchart illustrating another YOLOE-based method for small target detection in edge remote sensing images, provided as an embodiment of this application. Detailed Implementation

[0017] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0018] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0019] The YOLOE-based small target detection method for edge-side remote sensing images provided in this application can be applied to, for example... Figure 1 The YOLOE-based small target detection system for edge-side remote sensing images shown includes: an edge-side adaptation backbone network, a path aggregation network, a heatmap branch, a center non-maximum suppression unit, a SAVPE encoder, and an RTA encoder.

[0020] The edge-adaptive backbone network adopts the ResNet-D variant structure from Revisiting ResNets. It replaces the traditional large-kernel convolution with three serially connected 3×3 continuous small-kernel convolutions, which reduces the computational cost of remote sensing image feature extraction while avoiding excessive convolution. This adapts to the edge-side NPU's support requirements for conventional operators. At the same time, it embeds an SE-Block channel attention mechanism with a channel compression ratio of 0.25. Through global average pooling, feature mapping, and activation functions, it generates adaptive channel weights, strengthens the response of small target-related feature channels, suppresses background redundancy information, improves the accuracy of small target feature extraction, and finally outputs feature representations.

[0021] In one possible approach, in a real-time detection scenario using an edge device to detect urban remote sensing images, the target is a small house on a street (approximately 7×7 pixels in size, pixel percentage). The edge device employs an edge computing box equipped with an NPU (supporting conventional convolution and pooling operators). The edge-adaptive backbone network uses a variant of the ResNet-D structure from RevisitingResNets. For the input 1024×1024 resolution urban remote sensing image, it replaces the traditional 7×7 large kernel convolution with three serially connected 3×3 continuous small kernel convolutions for feature extraction. Simultaneously, it embeds an SE-Block channel attention mechanism with a channel compression ratio of 0.25. Adaptive channel weights are generated through global average pooling, feature mapping, and the ReLU activation function. High weights are assigned to key feature channels such as the outline and texture of the small houses, while low weights are assigned to background channels corresponding to the sky and large areas of the road surface. The final output is a high-dimensional feature representation containing the effective features of the small houses.

[0022] Based on the aforementioned edge-adapted backbone network, the computational cost of feature extraction is reduced to 55% of that of traditional large-kernel convolutions through three serial 3×3 small-kernel convolutions, adapting to the computational limitations of edge-side NPUs. The SE-Block channel attention mechanism enhances the response of small target feature channels, suppresses background redundancy, and improves the feature extraction accuracy of small houses. The use of conventional operators natively supported by the edge-side NPU avoids the operator adaptation problem during deployment and improves the ease of edge deployment. The serial small-kernel convolution design retains sufficient feature extraction depth while avoiding overfitting caused by excessive convolution, balancing feature extraction performance and computational efficiency.

[0023] It should be noted that the principle of reducing the computational cost of feature extraction to 55% of that of traditional large-kernel convolutions through three sequential 3×3 small-kernel convolutions is based on the fundamental logic of convolutional layer computation (using the number of multiplication and addition operations as the core metric). Combining the sequential structure design of small-kernel convolutions with the structural differences of traditional large-kernel convolutions, it is deduced that the computational cost of convolution is positively correlated with the square of the convolution kernel size. The core term for the computational cost of a single round of feature extraction in the commonly used 7×7 large-kernel convolution in traditional backbone network STEM layers is 7×7×C×C=49C. 2 (The number of input and output channels is the same, both are C); while the total computational cost of a single round of a 3×3 mini-kernel convolution with three serially connected kernels is 3×(3×3×C×C)=27C. 2The computational cost ratio between the two is approximately 27 / 49 ≈ 55%. Furthermore, the continuous small-kernel convolution design is not simply about reducing computational cost. Rather, it adapts to the requirements of the edge NPU for supporting conventional operators. By serially stacking three small kernels, it ensures a receptive field similar to a single 7×7 large kernel (covering key feature regions of the target) while avoiding computational redundancy and over-convolution issues caused by excessively wide coverage in a single operation of large-kernel convolution. Ultimately, it reduces computational cost while retaining the effective receptive field required for small target feature extraction, achieving a balance between computational power adaptation and feature extraction capabilities.

[0024] The path aggregation network is connected to the output of the end-side adaptation backbone network. After receiving the feature representation, it performs multi-scale feature fusion processing to generate multi-scale features of different resolutions. Only the feature with the highest output resolution is selected as the target feature, and the target feature contains scale information, center offset information and embedding.

[0025] The heatmap branch is used to predict the probability distribution of the center point of small targets based on embedding, realize one-to-one target localization, score the target area, and output target center candidate information.

[0026] In one possible approach, in a scenario where small houses are detected in real-time on the edge of an urban remote sensing image, the heatmap branch receives the P3 high-resolution target features (1024×1024 resolution) output by the path aggregation network. Then, through two layers of lightweight 1×1 convolutions and a Sigmoid activation function, it predicts the probability that each pixel is the center point of the small house, generating a 1024×1024 heatmap. The training is constrained by the Focal Loss loss function to achieve a one-to-one prediction mechanism, ensuring that each small house corresponds to only one probability peak. The probability value of the heatmap pixels is directly used as the scoring result of the target area. The preset scene adaptive threshold is 0.7. The pixel positions with scores higher than the threshold are extracted as candidate information for the target center. Compared with the one-to-many prediction of traditional YOLOE, this greatly reduces the number of candidates, and the deviation between the candidate positions and the actual center of the small house is within 2 pixels.

[0027] Based on the aforementioned heatmap branch, a one-to-one prediction mechanism is used to fundamentally reduce the number of redundant candidate boxes, thereby reducing the computational overhead of subsequent redundant filtering and the overall time consumption of edge detection. The high-probability regions of the heatmap generated based on embedding have a very high matching degree with the actual center of small houses, and the positioning error is controlled within 2 pixels, improving the positioning accuracy of small targets. The pixel probability value is directly used as the scoring result, eliminating the need for an additional scoring module, simplifying the edge calculation process, and threshold filtering quickly removes low-confidence background areas, reducing the pressure on subsequent processing. Furthermore, the heatmap branch can run in parallel with the SAVPE encoder and RTA encoder, supporting simultaneous operation of text prompts, visual prompts, and no prompts, thus expanding scene adaptability.

[0028] Central nonmaximum suppression unit such as Figure 2 As shown, the MaxPool, Equal, and Cast operators are arranged in sequence. The entire process of the operators is executed on the NPU. Non-maximum suppression is performed on the convolution branch. The convolution branch contains convolution parameters and bias parameters. Sigmoid is selected as the activation function. The feature selection operator is used to select high confidence center points. The pooling kernel size of the MaxPool operator is 3×3.

[0029] In one possible approach, in a scenario where small houses are detected in real-time on an edge-side urban remote sensing image, the center non-maximum suppression unit (CNNSU) receives 320 target center candidate information output from the heatmap branch and performs computations on the edge-side NPU in the order of MaxPool, Equal, and Cast: First, a 3×3 pooling kernel is used to perform pixel-by-pixel sliding window processing on the heatmap corresponding to the target center candidate information. Each window covers the current pixel and its eight surrounding neighboring pixels. The MaxPool operator extracts the maximum probability value within each window, generating a "window maximum value feature map" with the same size as the original heatmap. Then, using the "window maximum value feature map" as a reference, the Equal operator is used to compare the pixel values ​​of the original heatmap with the corresponding values ​​pixel by pixel. To determine if the maximum values ​​of the windows at each location are consistent, a binary peak mask containing only 0 (inconsistent) and 1 (consistent) is generated to accurately mark potential peak points within each window. This binary peak mask is then converted to a floating-point data type (e.g., float32) supported by the edge NPU using the Cast operator, ensuring that the mask can be used in arithmetic operations with the heatmap normalized by the Sigmoid activation function. Based on the converted peak mask, feature suppression processing is applied to the original heatmap, retaining the original probability values ​​at positions with a mask value of 1 and setting non-peak positions with a mask value of 0 to 0. Finally, 58 unique high-confidence center points are selected from the feature-suppressed heatmap, and structured detection labels are generated by combining this with the category information of the target features.

[0030] Based on the aforementioned central non-maximum suppression unit, composed of conventional operators natively supported by the edge NPU, the output of MaxPool provides the core benchmark for judging peak values ​​for Equal, the binary mask result of Equal provides a clear conversion object for Cast, and the floating-point mask after Cast provides a computable tool for non-peak suppression. The three operators do not need to perform complex NMS operations on the CPU, solving the slow processing problem caused by the O(n²) complexity of traditional NMS and reducing the time consumption of redundant filtering. The 3×3 pooling core accurately covers the neighborhood redundant candidate points of the center point of the small house, and the peak suppression filtering effect is excellent. The operator combination is simple, no need to customize complex logic, and can be directly integrated into the edge NPU computing process. It has strong compatibility with other modules of the system and reduces the difficulty of edge deployment.

[0031] The SAVPE encoder is a visual cue feature encoding unit used to extract and encode features from the reference image corresponding to the visual cue, and outputs a visual cue feature vector calculated by the adaptation device, providing data support for feature matching in visual cue mode.

[0032] The RTA encoder is a text prompt feature encoding unit used to extract and encode features of the text information corresponding to the text prompt, and outputs a text prompt feature vector calculated by the adaptation device to meet the relevance matching requirements in the text prompt mode.

[0033] To address the technical problems of slow detection speed and inaccurate small target feature capture when the YOLOE algorithm is directly applied to edge-side remote sensing images, this application provides a YOLOE-based method for small target detection in edge-side remote sensing images. This method includes: acquiring a remote sensing image, extracting features and outputting feature representations; fusing the feature representations at multiple scales to select the highest-resolution target features; using heatmap branching to achieve one-to-one localization of small targets and scoring of target regions to output center candidate information; and finally, performing redundancy filtering on the candidate information to obtain detection labels. Based on this, the method enhances the accuracy of small target feature extraction by combining small kernel convolution and channel attention mechanisms, and reduces computational overhead by using a one-to-one prediction and edge-side compatible redundancy filtering mechanism. This not only adapts to the computing power limitations of edge-side NPUs but also significantly improves the real-time performance and accuracy of small target detection in remote sensing images.

[0034] like Figure 3 As shown in the embodiments of this application, the method for small target detection in edge remote sensing images based on YOLOE includes: S301. Acquire remote sensing images.

[0035] In this embodiment, the remote sensing image to be detected can be obtained through various methods such as satellite imaging, drone data collection, and data server access. The image resolution and format are not specifically limited, as long as they are adapted to the data reading capabilities of the end-side device.

[0036] As an example, in agricultural monitoring scenarios, surface images transmitted in real time by low-orbit remote sensing satellites can be directly imported into edge detection equipment to complete the acquisition operation.

[0037] Based on the above steps, raw data support is provided for subsequent feature extraction and target detection, ensuring the integrity of the starting point of the detection process.

[0038] S302. Extract features from the remote sensing image and output the feature representation.

[0039] Among them, feature representation refers to the set of high-dimensional feature vectors obtained after remote sensing images are processed by the network, which contains key information such as the texture, contour, and grayscale of the target in the image.

[0040] In this embodiment, feature extraction is performed through an edge-adaptive backbone network. The edge-adaptive backbone network adopts an edge-friendly lightweight structure, replaces traditional large-kernel convolution with small-kernel convolution, and embeds a channel attention mechanism to enhance effective feature capture while reducing computational overhead.

[0041] It should be noted that the feature extraction process must be compatible with the conventional operators supported by the edge NPU to avoid deployment difficulties caused by using complex calculations.

[0042] As an example, on edge computing devices equipped with NPUs, the edge-adaptive backbone network can use lightweight variants of the ResNet family to perform feature extraction.

[0043] Based on the above steps, efficient extraction of effective features from remote sensing images is achieved, taking into account both the computing power adaptability and feature representation capabilities of edge deployment.

[0044] S303. Perform multi-scale feature fusion and filtering on the feature representation to obtain the target features.

[0045] Among them, target features refer to the features with the highest resolution that can accurately represent small target information after fusion and screening.

[0046] In this embodiment of the application, after the feature representation output by the backbone network is adapted by the receiving end of the path aggregation network, a feature set with different resolutions is generated by bidirectional fusion from top to bottom and bottom to top, and then the feature with the highest resolution is selected as the target feature.

[0047] It should be noted that the fusion strategy and screening logic can be adjusted according to the actual detection scenario, and the core is to retain the detailed features of small targets.

[0048] Based on the above steps, complementary enhancement of multi-scale features is achieved, effective information of small targets is condensed, and subsequent positioning accuracy is improved.

[0049] S304. Perform center point probability prediction and target region scoring on the target features, and output target center candidate information.

[0050] Among them, the target center candidate information refers to the candidate data set containing the coordinates of possible center points of small targets and confidence scores.

[0051] In this embodiment, a heatmap is generated by a probability prediction model based on the embedding of the target feature through a heatmap branch. The pixel value of the heatmap represents the probability of the target center point. At the same time, the confidence of the target area is evaluated by a preset scoring rule, and the location information corresponding to the high confidence area is extracted as the target center candidate information.

[0052] Based on the above steps, the initial positioning and effective screening of small targets can be achieved, reducing the amount of redundant calculations in the future.

[0053] S305. Redundancy filtering is performed on the candidate information of the target center to obtain the detection label.

[0054] Among them, the detection label refers to the final detection result data containing key information such as the small target category, location coordinates, and confidence level.

[0055] In this embodiment, the target center candidate information is filtered by the peak suppression mechanism through the center non-maximum suppression unit, redundant candidate points of the same target are eliminated by the sliding window processing, the center point with the highest confidence is retained, and the center point information is combined to generate a complete detection label.

[0056] It should be noted that the filtering mechanism needs to be compatible with the parallel computing capabilities of the edge NPU to ensure processing speed.

[0057] Based on the above steps, redundant candidate information is efficiently eliminated, ensuring that each small target corresponds to a unique and accurate detection result.

[0058] Based on the above technical solution, through the coordinated cooperation of feature extraction adapted to the edge, multi-scale fusion screening, precise positioning and redundant filtering, it not only adapts to the computing power limitations of edge devices, but also achieves efficient and accurate detection of small targets in remote sensing images. It effectively solves the technical problems of slow detection speed and inaccurate capture of small target features when the YOLOE algorithm is directly applied to the detection of small targets in edge remote sensing images.

[0059] In one possible implementation of the embodiments of this application, combined with Figure 3 ,like Figure 4As shown, the above S303 can be implemented through the following S401 and S402, which are explained in detail below: S401. Perform multi-scale feature fusion on the feature representation to generate P3, P4, and P5 multi-scale features.

[0060] Among them, P3, P4, and P5 are feature levels with different resolutions, with the resolution decreasing in that order, and each corresponding to a target feature of different sizes in the image.

[0061] In this embodiment of the application, after the feature representation output by the backbone network is adapted by the receiving end of the path aggregation network, feature information at different levels is integrated through top-down and bottom-up feature transfer and fusion strategies to generate multi-scale feature sets of P3, P4 and P5 covering different resolutions.

[0062] It should be noted that the core of feature fusion is to complement the advantages of features at different scales, taking into account the feature capture needs of both large and small targets.

[0063] As an example, in urban remote sensing image detection, the path aggregation network fuses the four basic features output by the backbone network to generate P3, P4, and P5 features with resolutions of 1024×1024, 512×512, and 256×256, respectively.

[0064] Based on the above steps, the hierarchical information of the features is enriched, laying the foundation for subsequent targeted selection of small target features.

[0065] S402. Select the P3 feature with the highest resolution as the target feature.

[0066] In this embodiment, the path aggregation network prioritizes the P3, P4, and P5 multi-scale features according to the core requirements of small target detection, and selects and outputs the P3 feature with the highest resolution as the target feature.

[0067] As an example, in urban remote sensing image detection, path aggregation networks filter P3 features to accurately capture small buildings and small vehicles on streets in remote sensing images.

[0068] Based on the above steps, the effective feature information of small targets is condensed, the redundant calculation caused by low-resolution features is reduced, and the accuracy of small target detection is improved.

[0069] Based on the above technical solution, by integrating the advantages of features at different levels through multi-scale feature fusion, and then condensing the effective information of small targets through high-resolution feature filtering, the integrity of features is guaranteed, while highlighting the detailed features of small targets, thus effectively improving the accuracy and efficiency of small target detection.

[0070] In one possible implementation of the embodiments of this application, combined with Figure 3 ,like Figure 5 As shown, the above S304 can be specifically implemented through the following S501, S502, S503, and S504, which are explained in detail below: S501. Based on target features, predict the probability distribution of the center point of small targets and generate a heat map.

[0071] Among them, the heat map is a two-dimensional image that represents the probability distribution of the center point of a small target. The higher the pixel value, the higher the confidence that the position is the center point of the target.

[0072] In this embodiment, after receiving the target feature embedding output by the path aggregation network, the heatmap branch predicts the probability that each pixel is the center point of the small target through multi-layer lightweight convolution and activation function operations, and generates a heatmap that matches the spatial dimension of the target feature.

[0073] As an example, in urban remote sensing image detection, the heatmap branch generates a 1024×1024 heatmap based on P3 features, with high-probability areas corresponding to small buildings and houses.

[0074] Based on the above steps, the problem of locating small targets is transformed into a probabilistic prediction problem, providing an intuitive basis for accurate positioning.

[0075] S502. A one-to-one prediction mechanism is adopted to assign a unique center point to each small target.

[0076] In this embodiment, the heatmap branch is trained by constraining the model through a loss function to ensure that each small target forms only one probability peak on the heatmap, that is, a unique corresponding center point, thus avoiding multiple candidate locations for the same target.

[0077] It should be noted that the one-to-one prediction mechanism fundamentally replaces YOLOE's one-to-many prediction, reducing the generation of redundant candidate boxes.

[0078] As an example, in urban remote sensing image detection, small houses correspond to only one center point with the highest confidence on the heat map, without any additional redundant peaks.

[0079] Based on the above steps, the number of candidate boxes is significantly reduced, subsequent computational overhead is lowered, and edge processing speed is improved.

[0080] S503. Use the pixel probability value in the heat map as the scoring result for the target area.

[0081] In this embodiment, the probability values ​​of heatmap pixels are directly mapped to the confidence scores of the target region through the heatmap branch, eliminating the need for an additional scoring module and simplifying the edge calculation process.

[0082] It should be noted that the scoring results are positively correlated with the pixel probability values, maintaining the consistency and simplicity of the evaluation logic.

[0083] As an example, in the heatmap, a pixel probability value of 0.8 corresponds to a target area score of 80 points, and a probability value of 0.5 corresponds to a score of 50 points.

[0084] Based on the above steps, a rapid quantitative assessment of the target area can be achieved, and effective screening can be completed without complex calculations.

[0085] S504. Extract the positions of pixels whose scores are higher than the preset threshold and use them as candidate information for the target center.

[0086] In this embodiment of the application, a preset scene adaptive threshold is used to filter out pixel positions with scores higher than the threshold by using a heatmap branch, and the coordinate information of these positions is integrated into a target center candidate information set.

[0087] It should be noted that the preset threshold can be dynamically adjusted according to the detection accuracy requirements to balance recall and precision.

[0088] As an example, in an urban remote sensing image detection scenario, a preset threshold of 0.7 is used to extract pixel locations with scores higher than 70, which correspond to the center point of a small house.

[0089] Based on the above steps, low-confidence locations corresponding to invalid background regions are eliminated, resulting in accurate candidate information and reducing the pressure of subsequent redundant filtering.

[0090] Based on the above technical solution, through the coordinated use of heatmap generation, one-to-one localization, probability scoring and threshold screening, it not only achieves accurate preliminary localization of small targets, but also reduces redundant candidate boxes from the source, providing high-quality input for the redundant filtering of the subsequent center non-maximum suppression unit, thus balancing the accuracy and real-time performance of edge detection.

[0091] In one possible implementation of the embodiments of this application, combined with Figure 3 ,like Figure 6 As shown, the above S305 can be implemented through the following S601, S602 and S603, which are explained in detail below: S601. A 3×3 pooling kernel is used to perform sliding window processing on the heatmap corresponding to the candidate information of the target center.

[0092] In this embodiment, the sliding window process is performed by the MaxPool operator in the central nonmaximum suppression unit. The pooling kernel slides pixel by pixel along the width and height of the heatmap, and each window covers the current pixel and 8 surrounding neighboring pixels (3×3 size).

[0093] It should be noted that the pooling kernel size can be flexibly adjusted according to the density of small targets, and the core is to cover the potential neighborhood redundancy points of the target center point.

[0094] As an example, in urban remote sensing image detection, a 3×3 pooling kernel slides across the heatmap corresponding to a small house, covering the target center and surrounding possible redundant candidate points.

[0095] Based on the above steps, a local judgment range is defined for subsequent screening of unique peak points.

[0096] S602. Retain the maximum value within each window and set the remaining pixel values ​​to 0 or the minimum value.

[0097] In this embodiment, the central non-maximum suppression unit first extracts the maximum confidence value in each window using the MaxPool operator, then uses the Equal operator to determine whether the original heatmap pixel is consistent with the maximum value, and finally uses the Cast operator to convert the data type, suppressing the non-peak pixel value to 0 or a minimum value that does not affect subsequent filtering.

[0098] As an example, in urban remote sensing image detection, the maximum window value corresponding to small houses is retained, while the surrounding non-peak pixel values ​​are all set to 0, highlighting the effective target area.

[0099] Based on the above steps, peak suppression is achieved by directly eliminating redundant candidate points around the same target, thereby reducing the amount of subsequent computation.

[0100] S603. Filter to obtain the unique center point with the highest confidence for each small target and generate a detection label.

[0101] The detection label contains key detection information such as the center point coordinates, category, and confidence level of the small target, which can be directly used for result display or subsequent processing on the edge device.

[0102] In this embodiment, the central non-maximum suppression unit filters the heatmap after peak suppression, extracts the unique peak point that meets the confidence requirements as the center point of the small target, and generates a structured detection label by combining the category information and position mapping relationship corresponding to the target features.

[0103] As an example, in urban remote sensing image detection, each electric vehicle corresponds to a peak point with the highest confidence, generating a detection label containing the center point coordinates and confidence of the electric vehicle.

[0104] Based on the above steps, we can ensure that each small target corresponds to a unique and accurate detection result, avoid duplicate labeling, and improve detection reliability.

[0105] Based on the above technical solution, the operators of the central non-maximum suppression unit collaboratively perform sliding window processing, peak suppression and precise screening. The operators are integrated on the NPU, which speeds up the calculation. It not only efficiently filters the redundancy in the candidate information of the target center, but also ensures the unique detection result of each small target. It takes into account the real-time performance and accuracy of edge detection, and effectively solves the problem of low deployment efficiency of traditional NMS on the edge.

[0106] One possible implementation is, such as Figure 7 As shown, the YOLOE-based small target detection method for edge remote sensing images provided in this application embodiment further includes the following S701 and S702: S701. If there are text prompts or visual prompts, feature encoding is performed using the corresponding RTA encoder and SAVPE encoder to obtain the prompt feature vector.

[0107] Among them, textual prompts refer to natural language text describing the category of the target, while visual prompts refer to example images or local feature maps of the target.

[0108] In this embodiment, the encoder automatically matches the encoder according to the type of prompt information. Text prompt information is input into the RTA encoder, and visual prompt information is input into the SAVPE encoder. Feature extraction and encoding are completed through lightweight network operations to obtain the prompt feature vector, i.e., the prompt embedding.

[0109] It should be noted that the encoder supports reparameterization optimization and can be seamlessly integrated into the main network after training, without additional computational overhead during inference.

[0110] Based on the above steps, the multimodal prompt information is transformed into structured features.

[0111] S702. The prompt feature vector is matched with the redundant filtered target center candidate information to obtain the detection label.

[0112] In this embodiment, the features of the prompt feature vector and the candidate information are first normalized, and then the correlation between the two is calculated by a measurement method such as cosine similarity. Candidate information with a matching degree higher than a preset threshold is selected to generate detection labels.

[0113] Based on the above steps, the association between the prompt information and the candidate target is realized, thereby improving the targeting and accuracy of small target detection.

[0114] Based on the above technical solution, by adapting different types of prompts to multimodal encoders and combining them with a correlation matching mechanism, the detection of small targets in edge remote sensing can flexibly respond to text or visual instructions. This not only expands the scene adaptability but also further improves the detection accuracy, enabling the detection of specific targets without relying on large-scale labeled data.

[0115] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0116] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0117] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A small target detection system for edge-side remote sensing images based on YOLOE, characterized in that, include: End-side adaptable backbone network, heatmap branches, and central nonmaximum suppression unit; The end-side adaptation backbone network is used to reduce the computational cost of feature extraction from remote sensing images by employing continuous small kernel convolutions, while embedding an SE-Block channel attention mechanism to enhance the accuracy of small target feature extraction and output feature representations. The heatmap branch is used to predict the probability distribution of the center point of small targets, realize one-to-one target positioning and score the target area, and output target center candidate information; The center nonmaximum suppression unit consists of MaxPool, Equal, and Cast operators, and is executed on the NPU to filter redundancy in the target center candidate information.

2. The small target detection system for end-side remote sensing images according to claim 1, characterized in that, The system also includes: a path aggregation network, a SAVPE encoder, and an RTA encoder; The path aggregation network is connected to the output of the end-side adaptation backbone network and is used to receive the feature representation of the end-side adaptation backbone network and output multi-scale features. The SAVPE encoder is used to extract and encode features from the reference image corresponding to the visual cue, and output the visual cue feature vector. The RTA encoder is used to extract and encode features from the text information corresponding to the text prompt, and output the text prompt feature vector.

3. The small target detection system for end-side remote sensing images according to claim 1, characterized in that, The continuous small kernel convolution consists of three serially connected 3×3 convolutions; the channel compression ratio of the SE-Block is 0.

25.

4. The small target detection system for end-side remote sensing images according to claim 1, characterized in that, The pooling kernel size of the MaxPool operator in the central nonmaximum suppression unit is 3×3, and it is executed sequentially in the order of MaxPool, Equal, and Cast operators.

5. In the end-side remote sensing image small target detection system according to claim 2, the heat map branch, the SAVPE encoder, and the RTA encoder can operate in parallel.

6. A method for small target detection in edge-side remote sensing images based on YOLOE, applied to the edge-side remote sensing image small target detection system as described in any one of claims 1-5, characterized in that, The method includes: Acquire remote sensing images; Feature extraction is performed on the remote sensing image, and a feature representation is output. The feature representation is subjected to multi-scale feature fusion and filtering to obtain the target feature; the target feature is the feature with the highest resolution. The target features are used to predict the probability of the center point and score the target region, and the target center candidate information is output. Redundancy filtering is performed on the candidate information of the target center to obtain the detection label.

7. The method according to claim 6, characterized in that, The feature representation is subjected to multi-scale feature fusion and filtering to obtain target features, including: Multi-scale feature fusion is performed on the feature representation to generate P3, P4, and P5 multi-scale features; The P3 feature with the highest resolution is selected as the target feature.

8. The method according to claim 6, characterized in that, The target features are subjected to center point probability prediction and target region scoring, and target center candidate information is output, including: Based on the target features, predict the probability distribution of the center point of the small target and generate a heat map; A one-to-one prediction mechanism is used to assign a unique center point to each small target; The pixel probability values ​​in the heat map are used as the scoring results for the target area; The positions corresponding to pixels with scores higher than a preset threshold are extracted and used as candidate information for the target center.

9. The method according to claim 6, characterized in that, Redundancy filtering is performed on the candidate target center information to obtain detection labels, including: A sliding window process is applied to the heatmap corresponding to the candidate target center information using a 3×3 pooling kernel; Retain the maximum value within each window, and set the remaining pixel values ​​to 0 or the minimum value; The unique center point with the highest confidence for each small target is selected and a detection label is generated.

10. The method according to claim 6, characterized in that, The method for detecting small targets in remote sensing images from the edge also includes: If there are text or visual cues, feature encoding is performed using the corresponding RTA encoder and SAVPE encoder to obtain the cues feature vector; The detection label is obtained by performing correlation matching between the prompt feature vector and the redundant filtered target center candidate information.

Images