Small target detection method, equipment and device and storage medium
By adding a small object detection layer and an SPD-Conv layer to the YOLOv8 network, and combining it with the OKM module and EMA-SlideLoss function for training, the problems of increased computational cost and training difficulty in small object detection are solved, thereby improving detection accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DATANG MOBILE COMM EQUIP CO LTD
- Filing Date
- 2024-11-21
- Publication Date
- 2026-05-22
AI Technical Summary
In existing technologies, adding a detection layer in small object detection significantly increases computational load and puts pressure on hardware resources. Replacing the model network structure leads to increased training difficulty and reduced detection efficiency.
A small object detection layer is added to the YOLOv8 network structure, and the original convolutional layers are replaced with spatial-to-depth convolutional SPD-Conv layers. At the same time, an all-kernel OKM module is introduced into the neck network, and a sliding window loss function with exponential moving average is used for training to construct a small object detection model.
It improves the performance of small target detection, reduces the increase in computation and parameters, and enhances the model's detection accuracy and efficiency for small targets, adapting to complex scenarios and multi-scale targets.
Smart Images
Figure CN122073004A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, and in particular to a method, device, apparatus and storage medium for small target detection. Background Technology
[0002] Small object detection remains a challenging task in the field of computer vision. Especially in complex scenes and multi-scale object detection, small objects, due to their small size and inconspicuous features, often struggle with the traditional P3, P4, and P5 feature levels of YOLO (You Only Look Once) v8. This not only affects detection accuracy but also limits the improvement of detection efficiency.
[0003] To address this challenge, a common industry practice is to add additional detection layers, such as the P2 detection layer, to the detection framework to focus on capturing features of small targets. However, while this approach improves the detection performance of small targets to some extent, it also introduces new problems. On the one hand, adding extra detection layers significantly increases computational load, putting greater pressure on hardware resources; on the other hand, post-processing becomes more time-consuming, impacting overall detection efficiency. Besides adding detection layers, existing improvement schemes also design more targeted network structures for small targets, replacing the backbone and neck network of YOLOv8. However, this approach may increase computational complexity. More importantly, the new network structure may differ significantly from the original, leading to increased training difficulty. This requires not only more training data and time to achieve better performance but also more refined parameter tuning and optimization strategies. Summary of the Invention
[0004] This application provides a method, device, apparatus, and storage medium for small target detection, which addresses the technical problems in related technologies where adding a detection layer significantly increases computational load, puts greater pressure on hardware resources, and replacing the model network structure leads to increased training difficulty when performing small target detection.
[0005] In a first aspect, embodiments of this application provide a small target detection method, including: The image to be detected is input into the trained small object detection model to obtain the small object detection result of the image to be detected; The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
[0006] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. The small target detection model is constructed based on the YOLOv8 network structure after adding a small target detection layer and replacing the SPD-Conv layer.
[0007] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
[0008] In some embodiments, the OKM module consists of global branches, large-scale branches, and local branches.
[0009] In some embodiments, the small object detection model is trained based on a labeled sample image dataset to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, the small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a trained small object detection model.
[0010] Secondly, embodiments of this application provide an electronic device, including a memory, a transceiver, and a processor; A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations: The image to be detected is input into the trained small object detection model to obtain the small object detection result of the image to be detected; The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
[0011] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. The small target detection model is constructed based on the YOLOv8 network structure after adding a small target detection layer and replacing the SPD-Conv layer.
[0012] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
[0013] In some embodiments, the OKM module consists of global branches, large-scale branches, and local branches.
[0014] In some embodiments, the small object detection model is trained based on a labeled sample image dataset to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, the small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a trained small object detection model.
[0015] Thirdly, embodiments of this application provide a small target detection device, comprising: The acquisition module is used to input the image to be detected into the trained small object detection model to obtain the small object detection result of the image to be detected. The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
[0016] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. The small target detection model is constructed based on the YOLOv8 network structure after adding a small target detection layer and replacing the SPD-Conv layer.
[0017] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
[0018] In some embodiments, the OKM module consists of global branches, large-scale branches, and local branches.
[0019] In some embodiments, the small object detection model is trained based on a labeled sample image dataset to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, the small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a trained small object detection model.
[0020] Fourthly, embodiments of this application also provide a non-transiently readable storage medium storing a computer program for causing a processor to execute the small target detection method described in the first aspect above.
[0021] Fifthly, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to execute the small target detection method described in the first aspect above.
[0022] In a sixth aspect, embodiments of this application also provide a computer-readable storage medium storing a computer program for causing a computer to perform the small target detection method described in the first aspect above.
[0023] In a seventh aspect, embodiments of this application also provide a communication device, wherein the communication device stores a computer program, the computer program being used to cause the communication device to perform the small target detection method as described in the first aspect above.
[0024] Eighthly, embodiments of this application also provide a chip product, wherein the chip product stores a computer program for causing the chip product to perform the small target detection method as described in the first aspect above.
[0025] The small target detection method, device, apparatus, and storage medium provided in this application add a small target detection layer and an SPD-Conv layer to the YOLOv8 network structure to construct a small target detection model. The small target detection layer can learn the feature information of small targets, and the SPD-Conv layer can compensate for the increased computation and parameters caused by adding the small target detection layer. This allows for the acquisition of more fine-grained information and improves the model's detection performance for small targets. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic diagram of the original YOLOv8 network structure provided by related technologies; Figure 2 This is a flowchart illustrating the small target detection method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the model network structure with an added small target detection layer provided in an embodiment of this application; Figure 4 This is a schematic diagram of the model network structure with an added small target detection layer and a replaced SPD-Conv layer provided in the embodiments of this application; Figure 5 This is a schematic diagram of the network structure of the OKM module provided in the embodiments of this application; Figure 6 This is a schematic diagram of the model network structure provided in this application embodiment, which adds a small target detection layer, replaces the SPD-Conv layer, and adds an OKM module; Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the small target detection device provided in the embodiments of this application. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0029] To better understand the solutions provided in the embodiments of this application, a brief introduction to the relevant technologies of the embodiments of this application will be given first.
[0030] Figure 1 This is a schematic diagram of the original YOLOv8 network structure provided by relevant technologies, such as... Figure 1 As shown, the YOLOv8 network structure consists of three parts: the backbone, the neck, and the head. The backbone is responsible for feature extraction, employing a series of convolutional and deconvolutional layers, along with residual connections and bottleneck structures to reduce network size and improve performance. The neck is responsible for feature fusion, using a Path Aggregation Network (PAN) structure on top of the Feature Pyramid Network (FPN), a highly efficient feature pyramid network. By fusing feature maps of different scales through bottom-up and top-down paths, it enhances the model's multi-scale detection capabilities. The head contains three detection heads responsible for outputting detection information. It employs a decoupled head structure similar to YOLOX, separating the regression and prediction branches, making the model more likely to converge during training and resulting in better detection performance.
[0031] YOLOv8 object detection relies entirely on features at three levels: P3, P4, and P5, with head sizes of 80×80, 40×40, and 20×20, respectively.
[0032] Existing improved small target detection schemes include adding a small target detection layer (P2), which adds a 160×160 detection feature map for detecting targets larger than 4×4. This new map contains richer low-level feature information about the target. The two P2 layer features obtained from top-down and bottom-up in the neck network are fused with the same-scale features in the backbone network. The output feature is the fusion result of the three input features. This allows the P2 layer detector to detect small targets quickly and effectively. The P2 layer detector, combined with the original three detectors, can effectively mitigate the negative impact of scale variance.
[0033] The improved method significantly enhances the detection accuracy for small targets. However, adding an extra detection layer to the detection framework increases the computational complexity and memory usage of the model as the network depth and the number of feature maps increase.
[0034] In addition to adding detection layers, many improvements to small target detection start with the YOLOv8 network structure, such as replacing the backbone network or neck network. By designing a network structure more suitable for small targets, the network's ability to process multi-scale information is enhanced, thereby achieving higher detection accuracy.
[0035] However, directly replacing the YOLOv8 network structure may lead to model instability during training. The original hyperparameters may need to be readjusted, which not only increases the workload but may also lead to a decrease in model performance. In addition, the training difficulty of the new network structure will increase in order to ensure the generalization ability of the model, but it is also difficult to guarantee good domain adaptability.
[0036] To address the aforementioned issues, this application provides a method, device, apparatus, and storage medium for small object detection. To avoid the irreparable problems such as gradient vanishing, gradient exploding, increased training difficulty, and limited generalization ability that may result from directly replacing the YOLOv8 network structure, this application retains the original YOLOv8 network structure. While preserving the advantages of the original model, it starts with the backbone network, adds a small object detection layer, and then improves it through convolution to compensate for the increased computational complexity caused by adding the small object detection layer.
[0037] Figure 2 This is a flowchart illustrating the small target detection method provided in the embodiments of this application, as shown below. Figure 1 As shown, this application provides a method for detecting small targets, the execution subject of which can be an electronic device with a certain computing power, such as a computer device. The method includes the following steps: Step 200: Input the image to be detected into the trained small object detection model to obtain the small object detection result of the image to be detected.
[0038] The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
[0039] Specifically, in this embodiment of the application, a small object detection model can be constructed by combining a small object detection layer and a space-to-depth-convolutional (SPD-Conv) layer on the basis of the YOLOv8 network structure.
[0040] As mentioned earlier, small object detection often encounters scenarios where large and small objects coexist. During feature learning, the detection of large objects is prioritized, resulting in accurate detection of large objects but failure to detect small objects. The original YOLOv8 has three detection heads: 20×20 (large objects), 40×40 (medium objects), and 80×80 (small objects), corresponding to the P5, P4, and P3 feature layers, respectively. Although the feature pyramid structure used in the YOLOv8 neck network can reference feature information from multi-scale feature maps and improve the problem of small object detection, after multiple downsampling pooling operations, the P3 layer still struggles to accurately detect small objects.
[0041] Therefore, this application embodiment adds a small target detection layer to YOLOv8. Figure 3 This is a schematic diagram of the model network structure with an added small target detection layer provided in an embodiment of this application, as shown below. Figure 3 As shown, a 160×160 detection head was added in the previous step of the 80×80, thereby enhancing the detection capability of small targets.
[0042] SPD-Conv improves upon traditional convolutional methods by employing a space-to-depth (SPD) transformation technique, addressing the issue of information loss and performance degradation caused by the introduction of stride convolutions and pooling layers in traditional convolutions. By avoiding large stride convolutions and pooling layers, SPD-Conv also reduces unnecessary computation to some extent, while maintaining or even improving the model's detection performance.
[0043] SPD-Conv mainly consists of two parts: SPD layers and non-stretch convolutional layers.
[0044] The role of the SPD layer is to rearrange the spatial blocks (pixel blocks) of the input feature map onto the depth (channel) dimension, thereby increasing the number of channels and reducing the spatial resolution without losing information. This transformation allows Convolutional Neural Networks (CNNs) to better capture and process features of small objects and low-resolution images while preserving fine-grained information.
[0045] Specifically, the SPD feature transformation will have a size of Feature map Over-slicing Individual feature map As shown in the following formula: Where scale is the scaling factor. As part of feature map X, by composition, and It is divisible by the scaling factor. When the scaling factor is 2, four sub-maps are obtained. , , , The shape of each sub-map is The feature map X is downsampled by a factor of two. This step increases the number of channels in the feature map by reducing its spatial allocation ratio. The sub-feature maps are then merged along the channel dimension to obtain a new feature map. .
[0046] Following the SPD layer, a non-stretch convolutional layer with a stride of 1 is used to further process the rearranged feature map. This non-stretch convolutional layer ensures that every position on the feature map is covered by the convolutional kernel, thereby preserving information to the greatest extent and generating rich feature representations.
[0047] Specifically, after SPD feature transformation, a feature is added with A convolutional layer with a filter and a stride of 1, wherein and further transformation In this process, feature information is preserved as much as possible, while the problem of increased channel dimension caused by SPD is improved.
[0048] This application's embodiments, based on adding a small object detection layer to the traditional YOLOv8 model, use SPD-Conv to replace the original convolution. This can improve the information loss caused by the introduction of stride convolution and pooling layers in traditional convolution, thus improving detection performance. By avoiding the use of large stride convolution and pooling layers, SPD-Conv reduces unnecessary computation to a certain extent, making up for the increased computation caused by adding a small object detection layer.
[0049] After constructing a small object detection model based on the YOLOv8 network structure, small object detection layer, and SPD-Conv layer, the small object detection model can be trained using a dataset of labeled sample images. The training method can refer to the training method of the original YOLOv8 model, and is not limited here.
[0050] Once a well-trained small object detection model is obtained, when small object detection is needed on an image to be detected, the image to be detected can be input into the well-trained small object detection model to obtain the small object detection result of the image to be detected.
[0051] The small target detection method provided in this application adds a small target detection layer and an SPD-Conv layer to the YOLOv8 network structure to construct a small target detection model. The small target detection layer can learn the feature information of small targets, and the SPD-Conv layer can make up for the increased computation and parameters caused by adding the small target detection layer. This allows for the acquisition of more fine-grained information and improves the model's detection performance for small targets.
[0052] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. A small target detection model is constructed based on the YOLOv8 network structure with the addition of a small target detection layer and the replacement of the SPD-Conv layer.
[0053] Specifically, in the process of constructing a small target detection model based on the YOLOv8 network structure, a small target detection layer, and an SPD-Conv layer, the embodiments of this application can add a small target detection layer to the P2 feature layer of the YOLOv8 network structure and replace the original convolutional layer in the P2 feature layer with an SPD-Conv layer to construct a small target detection model.
[0054] Figure 4 This is a schematic diagram of the model network structure with an added small target detection layer and a replaced SPD-Conv layer provided in the embodiments of this application, as shown below. Figure 4 As shown, in order to avoid the increase in parameters that may result from the introduction of the SPD-Conv layer, and to avoid the problem of network layer deepening that may result from using the SPD-Conv layer in all stages, thereby exacerbating the resistance to data flow, the embodiments of this application can add a small target detection layer only to the P2 feature layer of the YOLOv8 network structure, and replace Conv with a finer-grained SPD-Conv when the P2 layer is downsampled by two times.
[0055] After being processed by SPD-Conv, the feature map obtained from the P2 feature layer will be richer in small target information. These enhanced features will then be used to fuse with the P3 layer, enabling the P3 layer to better perceive the presence of small targets while detecting medium and large targets, thereby improving the visibility and detection performance of small targets in the P3 layer feature map.
[0056] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
[0057] Specifically, in the embodiments of this application, an Omni-Kernel Module (OKM) can also be introduced to construct a small object detection model. OKM is a network module for image restoration tasks, which aims to enhance the image restoration effect through multi-scale feature representation learning.
[0058] In some embodiments, the OKM module consists of a global branch, a large-scale branch, and a local branch. Figure 5 This is a schematic diagram of the network structure of the OKM module provided in the embodiments of this application, as shown below. Figure 5 As shown, this module consists of three branches: a global branch, a large-scale branch, and a local branch, which can efficiently capture multi-scale receptive fields in image restoration tasks. OKM effectively enhances the model's ability to represent image features by introducing large kernel convolution and a global processing mechanism.
[0059] The local branch can use 1x1 depthwise convolutions to supplement local information in the image. This extremely simple operation is efficient and effective, especially suitable for handling small-scale degradation. The large-scale branch can capture large-scale contextual information and provide multi-granular receptive fields by using depthwise separable convolutions with exceptionally large kernel sizes (such as 63×63) and strip convolutions of different shapes (such as 1×63 and 63×1). The global branch can realize the global receptive field through dual-domain channel attention (DCAM) and frequency gating mechanism. This branch uses Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (IFFT) to globally modulate features in the frequency domain and uses Spatial and Channel Attention (SCA) to further refine features.
[0060] As mentioned earlier, YOLOv8's neck network uses an FPN+PAN structure. During the construction of the FPN, as the network depth increases, the resolution of the feature maps gradually decreases. Feature information of small targets is easily lost or diluted in this process, potentially limiting the model's performance when detecting small targets in deep feature maps, especially when the target distribution in the image is uneven or the background is complex. Although the PAN structure can effectively fuse features of different scales, its receptive field expansion is relatively limited. This may restrict the model's ability to capture long-distance contextual information and fail to fully consider the differences between features of different scales. This results in inaccurate fused feature representations, thus affecting the model's detection performance, especially when dealing with complex scenes and diverse targets, where existing fusion strategies cannot effectively integrate feature information of different scales.
[0061] In YOLOv8, lower-resolution feature maps are upsampled and merged with higher-resolution feature maps. In this embodiment, the neck network of YOLOv8 can be improved by adding an OKM module before the upsampling layer in the YOLOv8 neck network. Then, based on the YOLOv8 network structure after adding the OKM module, the small object detection layer, and the spatial-to-depth convolutional SPD-Conv layer, a small object detection model is constructed.
[0062] Figure 6 This is a schematic diagram of the model network structure provided in this application embodiment, which includes adding a small target detection layer, replacing the SPD-Conv layer, and adding the OKM module. Figure 6As shown, applying the OKM module to the low-resolution feature maps before upsampling enhances the information in these feature maps, making the merged feature maps richer and more discriminative. Deploying the OKM module in the neck network of YOLOv8 not only limits computational overhead but also enables the model to handle multi-scale degradation.
[0063] In some embodiments, a small object detection model is trained based on a labeled sample image dataset to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, a small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a well-trained small object detection model.
[0064] Specifically, in the Head stage of the YOLOv8 model, the model predicts the category of the target object contained in each anchor. This prediction is normalized to a probability distribution using a softmax function and then compared with the true category label (usually one-hot encoded). The distance between them is calculated to evaluate the classification performance. The small object detection model provided in this application embodiment is based on the YOLOv8 network structure and is trained using a classification loss function on a dataset of labeled sample images.
[0065] In related technologies, YOLOv8 primarily uses the cross-entropy loss function for classification. However, cross-entropy loss requires high model stability. In multi-class classification tasks, the number of samples in different classes can be extremely imbalanced. The cross-entropy loss function treats all classes equally, failing to consider the impact of class imbalance on the loss contribution. This may cause the model to focus too much on the more numerous classes during training, while ignoring the fewer numerous classes, thus affecting the overall classification performance of the model.
[0066] Therefore, in this embodiment of the application, the classification loss function calculated in the head stage can be replaced by the sliding window loss function based on exponential moving average (EMA-SlideLoss) to effectively deal with the situation of sample imbalance.
[0067] The EMA-SlideLoss loss function combines the Exponential Moving Average (EMA) and the SlideLoss loss function.
[0068] EMA is a statistical calculation method used to smooth time series data. It is more sensitive to recent data changes by giving higher weight to the most recent data points.
[0069] The SlideLoss loss function uses a weighted strategy to differentiate between predicted and ground truth bounding boxes based on their Intersection over Union (IoU), thereby distinguishing between easy and hard samples and effectively addressing sample imbalance. The SlideLoss loss function is shown below: in, The average of all bounding box IoUs, less than As a negative sample, greater than The positive samples are those that are positive.
[0070] Because the classification is ambiguous, samples near the boundary often suffer significant losses. To enable the model to utilize these samples more fully, [the following is done]: The samples are divided into positive and negative samples, and then the samples at the boundary are enhanced by SlideLoss.
[0071] The EMA-SlideLoss loss function is shown in the following formula: in, It is the current iteration number. The observed values, It is the attenuation factor. This represents the current iteration number. This represents the total number of training iterations. It can be determined using the formula above: in the early stages of training, the model considers historical data more; in the later stages of training, it considers current observations more.
[0072] This application proposes using EMA-SlideLoss to replace the cross-entropy loss function. It introduces the concept of EMA into SlideLoss, applying EMA to dynamically adjust the loss function. This allows for adaptive adjustment of the loss, making the weight changes of the model more stable during training. It can dynamically adjust the coefficients of positive and negative samples, making the model pay more attention to difficult-to-classify and misclassified samples, thus improving robustness.
[0073] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 7 As shown, the electronic device includes a memory 703, a transceiver 701, and a processor 702, wherein: The memory 703 is used to store computer programs; the transceiver 701 is used to send and receive data under the control of the processor 702; the processor 702 is used to read the computer program in the memory 703 and perform the following operations: Input the image to be detected into the trained small object detection model to obtain the small object detection result of the image to be detected; The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
[0074] Among them, Figure 7 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 702) and memory (memory 703). The bus architecture can also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 701 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium, including wireless channels, wired channels, optical fibers, and other transmission media.
[0075] The processor 702 is responsible for managing the bus architecture and general processing, while the memory 703 can store the data used by the processor 702 when performing operations.
[0076] In some embodiments, the processor 702 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.
[0077] The processor executes any of the methods described in the embodiments of this application according to the obtained executable instructions by calling a computer program stored in memory. The processor and memory may also be physically separated.
[0078] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. A small target detection model is constructed based on the YOLOv8 network structure with the addition of a small target detection layer and the replacement of the SPD-Conv layer.
[0079] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
[0080] In some embodiments, the OKM module consists of global branches, large-scale branches, and local branches.
[0081] In some embodiments, a small object detection model is trained based on a labeled sample image dataset to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, a small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a well-trained small object detection model.
[0082] It should be noted that the electronic device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail.
[0083] Figure 8 This is a schematic diagram of the small target detection device provided in the embodiments of this application, as shown below. Figure 8 As shown, this application embodiment provides a small target detection device, including an acquisition module 800, wherein: The acquisition module 800 is used to input the image to be detected into the trained small object detection model and obtain the small object detection result of the image to be detected. The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
[0084] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. A small target detection model is constructed based on the YOLOv8 network structure with the addition of a small target detection layer and the replacement of the SPD-Conv layer.
[0085] In some embodiments, a small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
[0086] In some embodiments, the OKM module consists of global branches, large-scale branches, and local branches.
[0087] In some embodiments, a small object detection model is trained based on a labeled sample image dataset to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, a small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a well-trained small object detection model.
[0088] Specifically, the small target detection device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.
[0089] It should be noted that the division of units / modules in the above embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.
[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0091] In some embodiments, a non-transiently readable storage medium is also provided, the non-transiently readable storage medium storing a computer program for causing a processor to execute the small target detection method provided in the above method embodiments.
[0092] Specifically, the non-transiently readable storage medium provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.
[0093] It should be noted that the non-transiently readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).
[0094] In some embodiments, a processor-readable storage medium is also provided, the processor-readable storage medium storing a computer program for causing a processor to execute the small target detection method provided in the above method embodiments.
[0095] Specifically, the processor-readable storage medium provided in this application embodiment can implement all the method steps implemented in the above method embodiments and achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.
[0096] In some embodiments, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program for causing a computer to perform the small target detection method provided in the above method embodiments.
[0097] Specifically, the computer-readable storage medium provided in the embodiments of this application can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.
[0098] In some embodiments, a communication device is also provided, wherein the communication device stores a computer program for causing the communication device to execute the small target detection method provided in the above-described method embodiments.
[0099] Specifically, the communication device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.
[0100] In some embodiments, a chip product is also provided, wherein the chip product stores a computer program for causing the chip product to perform the small target detection method provided in the above-described method embodiments.
[0101] Specifically, the chip product provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.
[0102] In this application, "determining B based on A" means that factor A must be considered when determining B. It is not limited to "B can be determined based solely on A," but should also include: "determining B based on A and C," "determining B based on A, C, and E," "determining C based on A, and further determining B based on C," etc. It can also include using A as a condition for determining B, for example, "when A satisfies the first condition, B is determined using the first method"; or "when A satisfies the second condition, B is determined," or "when A satisfies the third condition, B is determined based on the first parameter," etc. Of course, it can also be a condition where A is a factor in determining B, for example, "when A satisfies the first condition, C is determined using the first method, and B is further determined based on C," etc.
[0103] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0104] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0105] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0106] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0107] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for detecting small targets, characterized in that, include: The image to be detected is input into the trained small object detection model to obtain the small object detection result of the image to be detected; The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
2. The small target detection method according to claim 1, characterized in that, A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. The small target detection model is constructed based on the YOLOv8 network structure after adding a small target detection layer and replacing the SPD-Conv layer.
3. The small target detection method according to claim 1 or 2, characterized in that, A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
4. The small target detection method according to claim 3, characterized in that, The OKM module consists of global branches, large-scale branches, and local branches.
5. The small target detection method according to claim 1, characterized in that, The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, the small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a trained small object detection model.
6. An electronic device, characterized in that, Includes memory, transceiver, and processor; A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations: The image to be detected is input into the trained small object detection model to obtain the small object detection result of the image to be detected; The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
7. The electronic device according to claim 6, characterized in that, A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: A small target detection layer is added to the P2 feature layer of the YOLOv8 network structure, and the original convolutional layer in the P2 feature layer is replaced with a spatial-to-depth convolutional SPD-Conv layer. The small target detection model is constructed based on the YOLOv8 network structure after adding a small target detection layer and replacing the SPD-Conv layer.
8. The electronic device according to claim 6 or 7, characterized in that, A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer, including: Add a full-core OKM module before the upsampling layer in the YOLOv8 neck network; A small object detection model is constructed based on the YOLOv8 network structure with the addition of the OKM module, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer.
9. The electronic device according to claim 8, characterized in that, The OKM module consists of global branches, large-scale branches, and local branches.
10. The electronic device according to claim 6, characterized in that, The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model, including: Based on a dataset of labeled sample images, the small object detection model is trained using a sliding window loss function based on exponential moving average, resulting in a trained small object detection model.
11. A small target detection device, characterized in that, include: The acquisition module is used to input the image to be detected into the trained small object detection model to obtain the small object detection result of the image to be detected. The trained small object detection model is obtained in the following way: A small object detection model is constructed based on the YOLOv8 network structure, a small object detection layer, and a spatial-to-depth convolutional SPD-Conv layer; The small object detection model is trained based on a dataset of labeled sample images to obtain a trained small object detection model.
12. A non-transiently readable storage medium, characterized in that, The non-transiently readable storage medium stores a computer program that causes a processor to execute the small target detection method according to any one of claims 1 to 5.