A small target detection and training method based on image division
By combining the YOLOv5 network with image segmentation and data augmentation methods, the problems of overlapping small targets and class imbalance in UAV images were solved, improving detection accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-06
- Publication Date
- 2026-03-24
AI Technical Summary
Existing technologies suffer from poor small target detection performance due to overlapping small targets in drone images, difficulty in distinguishing similar categories, and an imbalance in the number of categories caused by the limitations of drone-acquired images.
Using the YOLOv5 network as the basic detector, image segmentation and data augmentation methods, including image stitching and copy-paste, are employed to simulate dense target scenes, thereby improving detection accuracy and efficiency.
It significantly improves the accuracy and efficiency of small target detection in UAV application scenarios, solves the problems of scarce target quantity and unbalanced target categories, and improves the performance of the detector.
Smart Images
Figure CN116403130B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for small object detection and training based on image segmentation, belonging to the field of computer vision technology. Background Technology
[0002] Object detection is a crucial research area in computer vision, with wide applications in facial recognition, identification of irregular driving behavior, and medical diagnosis. However, drone object detection technology still faces several challenges, such as the presence of numerous overlapping target clusters in drone images, varying target scales, imbalanced target categories, and difficulty in distinguishing similar categories. Because the appearance features of small targets are less distinct than those of large targets, and due to the influence of the drone's shooting angle and altitude, targets often cluster together, such as pedestrians waiting at traffic lights or closely spaced shared bicycles, exhibiting significant occlusion and overlap. Furthermore, according to visual theory, objects appear larger when closer and smaller when farther away, leading to excessively large scale differences among targets. In addition, since drone images are mostly captured in specific urban scenes, an imbalance in target categories also exists, such as differences in the number of people and cars on public roads.
[0003] Current object detection methods can be categorized into traditional methods and deep learning-based methods. Traditional methods consist of three steps: region selection, feature extraction, and classifier classification. However, with in-depth research into object detection algorithms, researchers have discovered three major problems with these methods: firstly, the region selection strategy lacks specificity; secondly, the time complexity is high; and thirdly, hand-designed features have poor robustness, resulting in unsatisfactory detection performance.
[0004] With the rapid development of Convolutional Neural Networks (CNNs), deep learning-based object detection methods have gradually emerged. From the initial two-step R-CNN and Fast R-CNN to Faster R-CNN object detection algorithms, these algorithms have improved the accuracy of object detection bounding boxes. Then came one-step SSD and YOLO series object detection algorithms. SSD is an object detection algorithm that directly predicts the coordinates and category of the bounding box, further improving detection speed compared to the Fast R-CNN series. The YOLO series, on the other hand, defines object detection as a regression problem of bounding boxes and classification confidence. Different versions employ different improvement methods, further enhancing object detection accuracy. YOLOv5 includes networks of different depths and widths, allowing users to choose between larger networks with better performance or smaller networks with better speed for different application scenarios.
[0005] Furthermore, for the detection of small targets, Tsung-Yi Lin et al. proposed Feature Pyramid Networks (FPN) and a Top-Down structure. This approach utilizes multi-scale feature fusion to map high-level features containing rich semantic information to high-resolution, detail-rich low-level features, thus improving the detection results for small targets. To avoid excessive dimensionality caused by direct feature stacking, Abhinav Shrivastava et al. proposed a Top-Down modulation structure, which fuses high-level and low-level features through convolution, rather than directly stacking them as in FPN. To further improve the detection performance of small targets, D. Dwibedi et al. proposed a copy-and-paste data augmentation method. This method is very simple: select a target from other images, copy it multiple times, and arbitrarily place it in the image to improve detection performance. Alexey et al. proposed an augmentation method in YOLOv4 that involves cropping the target bounding box and packaging it into a mosaic image. The principle is to use four 2×2 stitched images as a new image, called a mosaic image.
[0006] However, the above methods are difficult to achieve good detection performance when faced with overlapping small targets, difficult-to-distinguish similar categories, and uneven number of categories due to the limitations of drone-collected images. Summary of the Invention
[0007] To address the limitations of existing technologies in achieving good detection performance when faced with overlapping small targets, difficult-to-distinguish similar categories, and uneven distribution of target types due to the limitations of UAV-acquired images, the main objective of this invention is to propose a small target detection and training method based on image segmentation. This method utilizes two data augmentation methods—enhanced image stitching and copy-paste—to improve the accuracy and efficiency of small target detection in UAV-acquired images, thereby enhancing the detection accuracy and efficiency of small targets in UAV application scenarios.
[0008] The objective of this invention is achieved through the following technical solution:
[0009] This invention discloses a small target detection and training method based on image segmentation. It employs a YOLOv5 network as the base detector, segmenting the input image through a translational window, and then performing target detection on both the segmented and original images. Improved image stitching and copy-paste methods are introduced in the data augmentation part to address the issues of reduced target quantity and sparse quantity of certain categories in the segmented image. Simultaneously, this method simulates dense target clusters in real-world scenarios, thereby improving the accuracy and efficiency of small target detection in UAV applications.
[0010] This invention discloses a small target detection and training method based on image segmentation, comprising the following steps:
[0011] Step 1: Image segmentation and detection framework. Divide the input image into small blocks while maintaining the aspect ratio of the segmentation boxes. Figure 1 To.
[0012] The original image has dimensions A and B, and a scaling factor ψ, where ψ∈(0,1]. The dimensions of each sub-block are a=ψA and b=ψB, respectively. Under the same input size, the relative scale of the target on each sub-block increases by (1 / ψ-1), meaning the input image is magnified by a factor of 1 / ψ. N is the number of samples along the axis (N>1), and the total number of blocks is N. 2 The number of input images is equal to (N) 2 +1). Therefore, the overlap between two consecutive sub-blocks is That is, for any target with a maximum relative scale less than L, regardless of its location on the image, there must exist at least one sub-block in which the target is completely contained without any truncation. Therefore, the interval between sub-blocks is...
[0013] Step 2: Set relative scale thresholds δ1 and δ2, where δ1,δ2∈(0,1), to filter the size of the target to distinguish whether the target is a small target, a medium target or a large target.
[0014] The maximum scale threshold δ1 depends on the maximum scale of the targets in the entire dataset, while the minimum scale threshold δ2 is lower than the overlap L of contiguous sub-blocks. Targets with a maximum relative scale smaller than the minimum scale threshold δ2 are considered small targets and are assigned to sub-blocks for local detection during training and inference. Similarly, targets with a minimum relative scale greater than the maximum scale threshold δ1 are considered large targets and are only subject to global detection. Targets of other scales participate in both global and local detection tasks.
[0015] Step 3: During training, the length and width of image patches are controlled to flexibly stitch them into new images. The stitching method aims to completely destroy semantic information while maintaining the ratio between the stitched image and the original image, thus performing image stitching enhancement.
[0016] Step 3.1: Define the basic combination unit as two image blocks stitched together in two directions, either horizontally or vertically, i.e., two blocks with widths of a×b / 2 or a / 2×b, generating a standard a×b patch. Then, further combine three patches: first, stitch two a / 2×b / 2 patches vertically, then stitch an a / 2×b patch horizontally. Similarly, stitching four patches with a width of a / 2×b / 2 three times yields an a×b image.
[0017] Step 3.2: In addition to the ordinary image stitching in Step 3.1, two enhanced stitching methods are defined: patch embedding and triangular stitching.
[0018] The specific operation method of patch embedding is as follows: First, a standard patch of a×b and a smaller patch of length and width of ρ1a×ρ2b are cut, where ρ is the sampling factor of uniform distribution U(0.3,0.6). Then, the smaller patch is randomly placed on the standard patch.
[0019] The specific operation method of triangular splicing is as follows: First, cut two standard patches and fill them into a square matrix of the same size. Then, generate an upper triangular matrix and a lower triangular matrix along the main diagonal or anti-diagonal as a cover for the patches. After matrix calculation, a combination of two triangular patches is obtained.
[0020] Step 4: Perform copy-paste enhancement, where the pasted object has the same category and similar background as the original image to simulate overlapping dense targets in a city;
[0021] First, a target object is randomly selected from either the stitched image patch or the original image as the prototype for copying. Then, similar target objects from other images or the original image are randomly placed around this object, allowing for slight overlap between the pasted objects. Additionally, target objects from other categories are also selected as prototypes for pasting to enhance the realism of the simulation.
[0022] Building upon the conventional methods described above, two enhanced copy-paste methods are defined. The first method selects several target objects as prototypes for copying, and pastes them around these prototypes to create several relatively independent clusters. The second method first selects an object as a prototype, performs random pasting operations, then selects a new prototype from the generated clusters and performs random pasting again, repeating this process until a large cluster is obtained. Furthermore, when selecting objects to paste, the probability of rare categories is appropriately increased to address the problem of excessively large differences in the proportion of target categories.
[0023] Step 5: Train the target detection model based on local-global hybrid data with relative target cropping;
[0024] Step 5.1: Random Cropping of Relative Fixed Targets. During training, valid training sub-blocks are cropped online as an enhancement measure, where valid means that the image contains at least one complete target object. First, a target is randomly selected as the relative fixed target. Then, sub-blocks are obtained by randomly selecting positions around the target, while ensuring that the relative fixed target is within the sub-block and is not truncated. Finally, the truncation of targets in all sub-blocks is checked. If the area of a target in a sub-block is smaller than a set proportion of its complete area, it is determined to be invalid and removed. Otherwise, the valid target box is converted into a relative target on the sub-block, and its side length is scaled according to the sub-block scaling ratio 1 / ψ.
[0025] Step 5.2: Pre-divide the training image according to the partitioning method in Step 1, then use the original image and the sub-blocks together as training samples. Simultaneously, enhance the image using the method in Step 5.1, and employ a local-global hybrid training strategy. Global detection is performed on the original image to ensure the detection performance of larger targets, while local detection is performed on the partitioned sub-blocks to avoid truncation of small targets. This yields the target detection model.
[0026] Step 6: After resizing and padding the sub-blocks, combine them with the original image to form a mini-batch tensor for parallel inference. Then, scale the tensor and add a positional bias ε. i The predicted bounding boxes on the sub-blocks are converted into absolute coordinates, enabling simultaneous prediction and non-maximum suppression operations. Since the sub-blocks have a clear relative positional relationship with the original image, these operations can be performed efficiently in parallel.
[0027] It also includes step 7: based on the trained target detection model obtained in step 5, the optimal weights are input into the inference process of step 6 to obtain the detection results on the validation set. The validated best model is then deployed on a neuromorphic chip and applied to drone live-action sequences, which can display the target detection results in real time and improve the target detection accuracy.
[0028] Beneficial effects
[0029] 1. This invention discloses a small target detection and training method based on image segmentation. By using a segmentation detection framework and a hybrid data training strategy, the performance gap between the detector and targets of different sizes is reduced. Simultaneously, two data augmentation methods are introduced. On the one hand, enhanced image stitching completely destroys the semantic features of the image, thereby increasing the number of effective targets and alleviating the problem of scarce targets in certain categories. It also increases the complexity of the background, significantly improving the detector's detection accuracy. On the other hand, an improved copy-paste data augmentation method simulates challenging real-world scenarios such as dense clusters and overlapping occlusions, further improving the detector's accuracy.
[0030] 2. The present invention discloses a small target detection and training method based on image segmentation. It uses a segmentation detection framework with a hybrid data strategy in training and inference to avoid target truncation while handling targets of all scales. It constructs effective training sub-blocks by random online relative target cropping and then allocates targets through scale filtering. It selects global detection or local detection according to different target characteristics, which can obtain the best prediction results while keeping the scale unchanged. Most of the additional operations are processed in parallel mode, which improves detection efficiency. Attached Figure Description
[0031] Figure 1 This is a flowchart of a small target detection and training method based on image segmentation disclosed in this invention;
[0032] Figure 2 This is a schematic diagram illustrating the principle of a small target detection and training method based on image segmentation disclosed in this invention;
[0033] Figure (a) is a schematic diagram of the object detection inference framework of the method of the present invention, (b) is a schematic diagram of the principle of size filtering in the method of the present invention, and (c) is a schematic diagram of the object detection training framework of the method of the present invention.
[0034] Figure 3 This is a schematic diagram illustrating the principle of enhanced image stitching data augmentation in the method of the present invention;
[0035] Figure (a) shows the original sub-block, Figure (b) shows the basic unit, Figure (c) shows the embedded splicing, Figure (d) shows the triangular splicing, Figure (e) shows the three-image splicing, and Figure (f) shows the four-image splicing.
[0036] Figure 4 This is a schematic diagram illustrating the principle of enhanced copy-paste data augmentation in the method of the present invention;
[0037] Figure 5 This is a comparison of the detection results of the method of the present invention and conventional object detection methods on the VisDrone dataset;
[0038] Figure 6 This is a comparison of the detection results of the method of this invention and conventional object detection methods on the UAVDT dataset;
[0039] Figure 7 The qualitative detection results of the method of this invention on the VisDrone dataset;
[0040] Figure 8 This is the qualitative detection result of the method of the present invention on the UAVDT dataset. Detailed Implementation
[0041] The method of the present invention will be specifically described below with reference to the accompanying drawings and embodiments. The technical problems solved by the technical solution of the present invention and its beneficial effects are also described. It should be noted that the described embodiments are only intended to facilitate understanding of the present invention and do not constitute any limitation thereof.
[0042] This embodiment discloses a small target detection and training method based on image segmentation. It utilizes two RTX 2080TI processors, employs Python and PyTorch deep learning APIs, and uses the YOLOv5 detector as a foundation. Simulation experiments were conducted on the VisDrone and UAVDT datasets, as well as real-world drone image detection frameworks. Figure 1 As shown, it includes the following steps:
[0043] Step 1: Image segmentation and detection framework. Divide the input image into small blocks while maintaining the aspect ratio of the segmentation boxes. Figure 1 To.
[0044] The original input image has dimensions A = 960 and B = 540, with a scaling factor ψ = 0.4. Therefore, the dimensions of each sub-block are a = ψA = 384 and b = ψB = 216. At the same input size, the relative scale of the target on each sub-block increases by (1 / ψ - 1) = 1.5, equivalent to magnifying the input image by a factor of 2.5. For example... Figure 2 As shown in (a), the number of samples along the axis is N = 3, and the total number of blocks is N. 2 =9, the number of input images is equal to the number of sub-blocks plus the number of original images, that is (N 2 +1) = 10. Therefore, the overlap between two consecutive sub-blocks is In other words, for any target with a maximum relative scale less than 0.1, regardless of its location on the image, there must exist at least one sub-block such that the target can be completely contained within that sub-block without any truncation. Therefore, the interval between sub-blocks is...
[0045] Step 2: Size Filtering. Set relative scale thresholds δ1 and δ2, where δ1,δ2∈(0,1), to determine the size of the target and distinguish whether the target is a small, medium or large target.
[0046] The maximum scale threshold δ1 depends on the maximum scale of the target in the entire dataset, while the minimum scale threshold δ2 should be lower than the overlap L of consecutive sub-blocks. For example... Figure 2As shown in (b), sub-blocks a and b contain targets of three different scales: target 5 with height h1, target 4 with height h2, and target 7 with height h3. Since the relative scale of target 5 to the sub-block exceeds the maximum scale threshold δ1, it will be considered a large target that cannot be matched with a suitable anchor box. During inference, the prediction confidence score for excessively large targets will be set to 0 accordingly. For target 7, h3 is much smaller than L, and at least one sub-block can completely contain it; therefore, it is considered a small target.
[0047] In summary, targets with a maximum relative scale smaller than the minimum scale threshold δ2 in the original image are considered small targets and are assigned only to sub-blocks during training and inference. Similarly, targets with a minimum relative scale still larger than the maximum scale threshold δ1 are considered large targets and are only used for global detection tasks. Targets of other scales participate in both global and local tasks.
[0048] Step 3: During training, the length and width of image patches are controlled to flexibly stitch them into new images. The stitching method aims to completely destroy semantic information while maintaining the ratio between the stitched image and the original image, thus enhancing the image stitching.
[0049] Step 3.1: First, define the basic combining unit as two image blocks stitched together in two directions, either horizontally or vertically, i.e., two image blocks with dimensions of 2×1 or 1×2, generating a standard 2×2 patch. Then, further obtain the combination of three patches: as follows... Figure 3 As shown, first, two 1×1 patches are stitched together vertically, and then a 1×2 patch is stitched together horizontally. Similarly, four patches with a width of a / 2×b / 2 are stitched together three times to obtain an a×b image.
[0050] Step 3.2: In addition to the ordinary image stitching in Step 3.1, two additional stitching methods are defined: patch embedding and triangle stitching, such as... Figure 3 As shown.
[0051] The specific operation method for patch embedding is as follows: First, a standard 2×2 patch and a smaller patch with dimensions ρ1a×ρ2b=1×3 are cut, where ρ is the sampling factor of the uniform distribution U(0.3,0.6). Then, the smaller patch is randomly placed on the standard patch, such as... Figure 3 Patches 2 and 4 are embedded into standard patches 1 and 3, respectively.
[0052] The specific operation of triangular patching is as follows: First, cut two 2×2 standard patches and fill them into square matrices of the same size. Then, generate upper and lower triangular matrices along the main diagonal or anti-diagonal as the data basis for the patches. After certain matrix calculations, the combination of two triangularly patched structures is obtained.
[0053] Step 4: Copy and paste to enhance.
[0054] Copy and paste is a common data augmentation technique. For images acquired by drones, this invention proposes an improved copy and paste operation, in which the pasted object has the same category and similar background as the original image, to simulate overlapping dense targets in a city. The specific method is as follows:
[0055] First, a target object is randomly selected from either the stitched image patch or the original image as the prototype for copying. Then, similar target objects from other images or the original image are randomly placed around this object, allowing for slight overlap between the pasted objects. Additionally, target objects from other categories are also selected as prototypes for pasting to enhance the realism of the simulation.
[0056] Based on the conventional methods described above, this invention defines two enhanced copy-paste methods, such as... Figure 4 As shown in the left image, the first method involves selecting two target objects as prototypes for copying, and then pasting several objects (with dotted dashed borders and short dashed borders) around them, resulting in two relatively independent clusters; as shown... Figure 4 As shown in the right figure, the second method involves first selecting one object as a prototype (the dashed box in the middle). After a random pasting operation, a new prototype is selected from the generated cluster, and the random pasting operation is performed again. This process is repeated until a large cluster is obtained. Furthermore, when selecting objects to paste, the probability of rare categories is appropriately increased to address the problem of excessively large differences in the proportion of target categories. Additionally, the pasted objects should have different aspect ratios.
[0057] Step 5: Training based on local-global hybrid data with relative target cropping.
[0058] Step 5.1: Random Cropping of Relative Fixed Targets. During training, valid training sub-blocks need to be cropped online, where "valid" means that the image should contain at least one complete target object. First, a target is randomly selected as the relative fixed target. Then, different sub-blocks containing the relative fixed target are obtained by randomly selecting the size and position around the target, while ensuring that the relative fixed target is within the sub-block and is not truncated. Finally, the truncation of targets in all sub-blocks is checked. If the area of a target in a sub-block is less than 60% of its complete area, it is considered invalid and removed. Otherwise, the valid target box is converted into a relative target (relative coordinates) on the sub-block, and its side length is scaled according to the sub-block scaling ratio 1 / ψ = 2.5.
[0059] like Figure 2As shown in (c), two clipping sub-blocks A and B (dotted dashed box and short dashed box) of different sizes are randomly generated around the selected relative target (target number 4). Sub-block A contains targets numbered 2, 3, and 4. The upper part of target number 2 is clipped, and the remaining area is less than 60% of its full area. Therefore, it will be removed after the truncation check. Sub-block B contains targets numbered 4, 5, and 6. The lower part of target number 6 is significantly clipped, so it is also removed after the truncation check.
[0060] Step 5.2: Pre-divide the training image according to the division method in Step 3, and then use the original image and the sub-blocks together as training samples. At the same time, enhance the image using the method proposed in Step 5.1, and adopt a local-global hybrid training strategy. In this strategy, global detection is performed on the original image to ensure the detection performance of larger targets, and local detection is performed on the sub-blocks to avoid truncation of small targets.
[0061] Step 6: Based on Step 3, combine the resized and padded sub-blocks with the original image to form a mini-batch tensor for parallel inference. Then, scale the tensor and add a positional bias ε. i The predicted bounding boxes on the sub-blocks are converted into absolute coordinates, enabling simultaneous prediction and NMS operations. Because the sub-blocks have a clear relative positional relationship with the original image, these operations can be performed efficiently in parallel.
[0062] The results are as follows:
[0063] Figure 5 This paper compares the detection results of the method of this invention with conventional object detection methods on the VisDrone dataset. The comparison includes algorithms with different backbones, such as Faster RCNN using ResNet50 as the backbone and HRDNet using ResNet101 as the backbone. As can be seen from the figure, the detection performance of the present invention is superior in the UAV object detection task on this dataset.
[0064] Figure 6 This paper compares the detection results of the method of this invention with those of conventional object detection methods on the UAVDT dataset. Multiple object detection algorithms are also compared, demonstrating the superior detection performance (AP) of the method of this invention. 50 It outperforms the most advanced UFPMP-DET algorithm by 7.8%;
[0065] Figure 7The figure shows the qualitative detection results of the method of the present invention on the VisDrone dataset. As can be seen from the figure, the target detection results obtained by the method of the present invention are quite ideal, and it can detect most of the targets in the figure, including cars, trucks, vans, and objects that are overlapping, occluded, or small in scale.
[0066] Figure 8 The figure shows the qualitative detection results of the method of the present invention on the UAVDT dataset. As can be seen from the figure, the method of the present invention has good detection effect on UAV ground targets in different scenarios.
[0067] As demonstrated above, the small target detection and training method based on image segmentation disclosed in this embodiment can improve the accuracy and efficiency of small target detection in UAV application scenarios.
[0068] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for small target detection and training based on image segmentation, characterized in that: Includes the following steps, Step 1: Image segmentation detection framework, which divides the input image into small blocks while maintaining the aspect ratio of the segmentation boxes consistent with the original image; The implementation method for step 1 is as follows: The original image has dimensions A and B, and a scaling factor ψ, where ψ∈(0,1]. The dimensions of each sub-block are a=ψA and b=ψB, respectively. Under the same input size, the relative scale of the target on each sub-block increases by (1 / ψ-1), meaning the input image is magnified by a factor of 1 / ψ. N is the number of samples along the axis (N>1), and the total number of blocks is N. 2 The number of input images is equal to (N) 2 +1); therefore, the overlap between two consecutive sub-blocks is That is, for any target with a maximum relative scale less than L, regardless of its location on the image, there must exist at least one sub-block in which the target is completely contained without any truncation; therefore, the interval between sub-blocks is... Step 2: Set relative scale thresholds δ1 and δ2, where δ1,δ2∈(0,1), to filter targets by size to distinguish whether the target is a small target, a medium target, or a large target; Step 3: During training, the length and width of image patches are controlled to flexibly stitch them into new images. The stitching method aims to completely destroy semantic information while maintaining the ratio between the stitched image and the original image, thus enhancing the image stitching. The implementation method for step 3 is as follows: Step 3.1: Define the basic combination unit as two image blocks stitched together in two directions, either horizontally or vertically, i.e., two blocks with widths of a×b / 2 or a / 2×b, to generate a standard patch of a×b; then further obtain a combination of three patches: first stitch two patches of a / 2×b / 2 in the vertical direction, and then stitch a patch of a / 2×b in the horizontal direction; similarly, stitching four patches of width a / 2×b / 2 three times can obtain an image of a×b. Step 3.2: In addition to the ordinary image stitching in Step 3.1, two enhanced stitching methods are defined: patch embedding and triangular stitching. The specific operation method of patch embedding is as follows: First, a standard patch of a×b and a smaller patch of length and width of ρ1a×ρ2b are cut, where ρ is a sampling factor of uniform distribution U(0.3,0.6). Then, the smaller patch is randomly placed on the standard patch. The specific operation method of triangular splicing is as follows: First, cut two standard patches and fill them into a square matrix of the same size. Then, generate an upper triangular matrix and a lower triangular matrix along the main diagonal or anti-diagonal as a cover for the patches. After matrix calculation, a combination of two triangular patches is obtained. Step 4: Perform copy-paste enhancement, where the pasted object has the same category and similar background as the original image to simulate overlapping dense targets in a city; The implementation method for step 4 is as follows: First, a target object is randomly selected from the stitched image patch or the original image as the prototype for copying. Then, target objects of the same category from other images or the original image are randomly placed around this object, where slight overlap between the pasted objects is allowed. In addition, target objects of other categories are also selected as prototypes for pasting to enhance the realism of the simulation. Based on the conventional method described above, two enhanced copy-paste methods are defined. The first method selects several target objects as prototypes for copying and pastes them around them to obtain several relatively independent clusters. The second method selects an object as a prototype, performs a random paste operation, selects a new prototype from the generated cluster, and performs another random paste operation, repeating the process until a large cluster is obtained. In addition, when selecting objects to paste, the probability of rare categories is appropriately increased to solve the problem of excessively large differences in the proportion of target categories. Step 5: Train the target detection model based on local-global hybrid data with relative target cropping; Step 6: After resizing and padding the sub-blocks, combine them with the original image to form a mini-batch tensor for parallel inference; then scale and add a positional bias ε. i This converts the prediction boxes on the sub-blocks into absolute coordinates, enabling both prediction and non-maximum suppression operations to be performed simultaneously.
2. The small target detection and training method based on image segmentation as described in claim 1, characterized in that: It also includes step 7: based on the trained target detection model obtained in step 5, the optimal weights are input into the inference process of step 6 to obtain the detection results on the validation set. The validated best model is then deployed on a neuromorphic chip and applied to drone live-action sequences, which can display the target detection results in real time and improve the target detection accuracy.
3. The small target detection and training method based on image segmentation as described in claim 1, characterized in that: The implementation method for step 2 is as follows: The value of the maximum scale threshold δ1 depends on the maximum scale of the target in the entire dataset, while the value of the minimum scale threshold δ2 is lower than the overlap L of the continuous sub-blocks. For targets in the image whose maximum relative scale is less than the minimum scale threshold δ2, they are regarded as small targets and are only assigned to sub-blocks for local detection tasks during training and inference. Similarly, targets whose minimum relative scale is still greater than the maximum scale threshold δ1 are regarded as large targets and are only used for global detection tasks. Targets of other scales participate in both global and local detection tasks.
4. The small target detection and training method based on image segmentation as described in claim 1, characterized in that: The implementation method for step 5 is as follows: Step 5.1: Random cropping of relative fixed targets; During training, effective training sub-blocks are cropped online as an enhancement measure, where effective means that the image contains at least one complete target object; First, a target is randomly selected as the relative fixed target, and then sub-blocks are obtained by randomly selecting positions around the target, while ensuring that the relative fixed target is in the sub-block and there is no truncation; Finally, the truncation of targets in all sub-blocks is checked. If the area of a target on the sub-block is smaller than a set ratio of its complete area, it is determined to be invalid and removed; otherwise, the effective target box is converted into a relative target on the sub-block, and its side length is scaled according to the scaling ratio 1 / ψ of the sub-block; Step 5.2: Pre-divide the training image according to the division method in Step 1, and then use the original image and the sub-blocks together as training samples. At the same time, enhance the image using the method in Step 5.1, and adopt a local-global hybrid training strategy. Global detection is performed on the original image to ensure the detection performance of larger targets, and local detection is performed on the divided sub-blocks to avoid truncation of small targets.