A small size and general target joint detection method and device based on YOLOv5

Abstract

The application discloses a small-size and general target joint detection method and device based on YOLOv5, and relates to the field of data processing.In the method, a target detection network is constructed based on a preset reference target detection network; a to-be-detected image is divided by a sliding window to generate a plurality of detection windows with overlapping regions; each detection window is input into the target detection network to obtain a preliminary detection result corresponding to each detection window; a non-maximum suppression process is performed on the preliminary detection results of each detection window to obtain a first detection result containing a micro-size target; a complete image of the to-be-detected image is input into the preset reference target detection network to obtain a second detection result containing a general target; and the first detection result and the second detection result are fused to generate a joint detection result, and the joint detection result is output.The technical scheme provided in the application improves the comprehensiveness of detection.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a method and apparatus for joint detection of small-sized and conventional targets based on YOLOv5. Background Technology

[0002] Object detection is one of the core tasks in computer vision, aiming to identify objects of a specific category in an image and locate their positions. Current mainstream object detection methods can be divided into two-stage detectors (such as the Faster R-CNN series) and single-stage detectors (such as YOLO and SSD series). Among them, single-stage detectors, represented by YOLOv5, are widely used due to their excellent balance between detection speed, accuracy, and ease of deployment.

[0003] However, YOLOv5's inherent design has limitations when processing high-resolution images (such as drone aerial photography and industrial quality inspection). When an entire high-resolution image containing a large number of tiny targets is forcibly scaled to the network's fixed input size (e.g., 640x640 pixels), the pixel information and texture features of the tiny targets are severely lost, making it difficult for the network to learn and recognize them, resulting in a large number of missed detections and incomplete detection. Summary of the Invention

[0004] This application provides a method and apparatus for joint detection of small and conventional targets based on YOLOv5, which solves the problem of missed detection of small targets in high-resolution images due to scaling, and improves the comprehensiveness of detection.

[0005] A first aspect of this application provides a method for joint detection of small and regular targets based on YOLOv5. The method includes: constructing a target detection network based on a preset benchmark target detection network; dividing the image to be detected into multiple detection windows with overlapping regions using a sliding window method; inputting each detection window into the target detection network to obtain preliminary detection results corresponding to each detection window; performing non-maximum suppression processing on the preliminary detection results of each detection window to obtain a first detection result containing small targets; inputting the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing regular targets; fusing the first detection result and the second detection result to generate a joint detection result, and outputting the joint detection result.

[0006] By adopting the above technical solution, this YOLOv5-based joint detection method for small and regular targets achieves simultaneous detection of small and regular targets in an image. This method involves constructing a target detection network, dividing the image into multiple overlapping detection windows using a sliding window approach, inputting each detection window into the target detection network to obtain preliminary detection results, performing non-maximum suppression on the preliminary detection results to obtain a first detection result containing small targets, inputting the complete image of the image to be detected into a preset benchmark target detection network to obtain a second detection result containing regular targets, and finally fusing the first and second detection results to generate and output a joint detection result. Specifically, by dividing the image into multi-scale sliding windows and using a target detection network optimized for small targets to perform local detection in each detection window, the detection capability for small targets in the image can be effectively improved. Simultaneously, using a preset benchmark target detection network to detect the entire image ensures the detection performance for regular targets. Finally, by fusing the two detection results, a joint detection result covering targets of different sizes can be obtained, improving the detection capability for small targets while also considering the detection effect for regular targets, thus achieving more comprehensive and effective target detection.

[0007] Optionally, the construction of the target detection network based on the preset benchmark target detection network specifically includes: adding a feature map with a downsampling factor of a first preset value as a shallow feature map in the feature pyramid of the preset benchmark target detection network, and removing feature maps with a downsampling factor of a second preset value, wherein the first preset value is less than the second preset value; performing a channel attention operation on the shallow feature map to generate channel attention weights; multiplying the channel attention weights by the shallow feature map to obtain a first feature map adjusted by channel attention; performing a spatial attention operation on the first feature map to generate spatial attention weights; adjusting the first feature map based on the spatial attention weights to obtain a second feature map, thereby obtaining the target detection network.

[0008] By adopting the above technical solution, feature maps with smaller downsampling factors are introduced as shallow feature maps into the feature pyramid of the preset benchmark target detection network, while feature maps with larger downsampling factors are removed. The shallow feature maps are then optimized using channel attention and spatial attention mechanisms, effectively enhancing the target detection network's ability to represent and extract small targets. Specifically, introducing shallow feature maps with higher resolution better preserves the detailed information of small targets; channel attention operations generate channel attention weights, which are then used to adjust different channels of the shallow feature maps, highlighting the salient features of small targets and suppressing background interference; spatial attention operations generate spatial attention weights, which are then used to adjust the first feature map, enhancing the response of small target regions in the spatial dimension and further improving detection accuracy. Ultimately, through a series of improvements optimized for small targets, the target detection network's ability to detect tiny targets is significantly strengthened, laying a solid foundation for subsequent multi-scale detection.

[0009] Optionally, the step of performing channel attention operation on the shallow feature map to generate channel attention weights specifically includes: performing a global average pooling operation on the shallow feature map to obtain an average pooling feature vector, and performing a global max pooling operation on the shallow feature map to obtain a max pooling feature vector; inputting the average pooling feature vector and the max pooling feature vector into a first fully connected layer for channel dimensionality reduction processing to generate corresponding first intermediate vector and second intermediate vector; inputting the first intermediate vector and the second intermediate vector into a second fully connected layer for channel dimensionality increase processing to restore the original number of channels of the shallow feature map, thereby generating a first processed vector and a second processed vector respectively; performing an element-wise addition operation on the first processed vector and the second processed vector to generate a merged feature vector; performing a Sigmoid activation function operation on the merged feature vector to map the values ​​of each element of the merged feature vector to the interval between 0 and 1, thereby generating the channel attention weights.

[0010] By employing the aforementioned technical solution, global average pooling and global max pooling operations are performed on shallow feature maps, aggregating global information from different perspectives. This highlights both the overall response intensity of each channel and focuses on local features in salient regions. Then, two fully connected layers are used to perform dimensionality reduction and expansion on the pooled feature vectors, extracting more abstract and higher-level semantic information while reducing computational cost. Next, the two processed feature vectors are fused through element-wise addition, taking into account the different features extracted by average pooling and max pooling. Finally, the sigmoid activation function maps the feature values ​​to the 0-1 range, resulting in channel attention weights consistent with the original number of channels. This channel attention mechanism adaptively adjusts the importance of different channels, highlighting salient features related to small targets and suppressing interference from irrelevant backgrounds, thereby effectively enhancing the object detection network's ability to represent and extract small targets and improving detection accuracy.

[0011] Optionally, the step of dividing the image to be detected into multiple detection windows with overlapping regions by sliding window division specifically includes: setting a minimum detection window size and a maximum detection window size according to the size range of the image to be detected; generating multiple window sizes with increasing sizes based on the minimum detection window size and the maximum detection window size, wherein the size ratio of any two adjacent window sizes is a preset value; calculating the corresponding sliding step size for each window size according to a preset overlap rate; and dividing the image to be detected into multiple detection windows with overlapping regions by sliding window division based on the window size and the corresponding sliding step size.

[0012] By employing the above technical solution, setting the minimum and maximum detection window sizes according to the size range of the image to be detected allows for flexible adjustment of the detection window size range to accommodate the potential target size distribution in the image, covering both small and large targets. Then, by generating multiple incrementally increasing window sizes between the minimum and maximum window sizes at a preset ratio, multi-scale sliding window detection can be achieved with a fixed size step, improving adaptability to targets of different sizes. Next, the sliding step size corresponding to each window size is calculated based on a preset overlap rate. By reasonably setting the overlap rate, a certain amount of redundancy can be introduced between adjacent detection windows, improving detection robustness and preventing targets from being truncated by window boundaries and thus missed. Finally, by dividing the image into sliding windows based on the generated window sizes and sliding step sizes, a series of detection windows of varying sizes and overlapping each other, covering the entire image region, can be obtained, providing rich candidate regions for subsequent local detection and improving the detection rate of small targets.

[0013] Optionally, the step of inputting each of the detection windows into the target detection network to obtain the preliminary detection results corresponding to each detection window specifically includes: traversing each detection window and sequentially using each detection window as input data, feeding it into the target detection network to perform forward propagation calculation, wherein the forward propagation calculation includes feature extraction, feature fusion, target classification, and bounding box regression; parsing out the category information, confidence score, and bounding box position information of the potential target within each detection window; and summarizing the category information, confidence score, and bounding box position information parsed from all detection windows to form the preliminary detection results corresponding to each detection window.

[0014] By employing the aforementioned technical solution, each detection window generated by the sliding window division is traversed and sequentially input into the target detection network for forward propagation computation. This fully leverages the network's strong representation capability of small targets within local regions, obtaining key information such as the category, confidence score, and location of potential targets within each window. The forward propagation computation includes feature extraction, feature fusion, target classification, and bounding box regression. By performing feature transformation and semantic extraction layer by layer on the input image and combining feature information at different scales, robust and accurate detection results can be obtained. Simultaneously, by analyzing the network output, the category label, confidence score, and bounding box coordinates of potential targets within each detection window can be clearly obtained, providing crucial information for subsequent detection result selection and fusion. Finally, by summarizing the network outputs of all detection windows, a preliminary detection result covering the entire image region to be detected can be obtained, containing target information that may exist at different locations and scales, preparing for further non-maximum suppression processing.

[0015] Optionally, performing non-maximum suppression processing on the preliminary detection results of each detection window to obtain a first detection result containing small-sized targets specifically includes: traversing the preliminary detection results of each detection window, extracting target bounding box information, the target bounding box information including position coordinates, size, and confidence score, and transforming the position coordinates from the local coordinate system of the detection window to the global coordinate system of the image to be detected to generate a set of target bounding boxes in the global coordinate system; grouping the target bounding box set according to the category information of each target bounding box in the target bounding box set to generate multiple target subsets to be merged; selecting the target bounding box with the highest confidence score as the reference box for each target subset to be merged; calculating the intersection-union ratio (IUGR) between the reference box and any target bounding box in the target subset to be merged other than the reference box; determining the remaining target bounding boxes with IUGR greater than a preset first non-maximum suppression threshold as redundant boxes and removing them from the target subset to be merged to obtain multiple suppressed subsets; merging the multiple suppressed subsets to form the first detection result.

[0016] By employing the aforementioned technical solution, the preliminary detection results of each detection window are traversed, and the target bounding box information is extracted. This allows for the acquisition of key attributes such as the precise location, size, and confidence score of each potential target. Simultaneously, by transforming the bounding box coordinates from the local coordinate system of the detection window to the global coordinate system of the image to be detected, the positional standards between different detection windows are standardized, facilitating subsequent bounding box filtering and merging. Then, the target bounding boxes are grouped according to their category information, generating multiple target subsets to be merged. Each subset contains potential targets predicted to be of the same category, preparing for further non-maximum suppression (NMS) processing. Next, NMS is applied to each target subset based on the confidence score and a preset first NMS threshold. By calculating the intersection-union ratio (CUI) between bounding boxes, bounding boxes with high redundancy and low confidence are removed, effectively eliminating redundant detections between different detection windows and improving the accuracy and reliability of the detection results. Finally, the suppressed target subsets are merged to obtain a refined detection result containing small-sized targets, i.e., the first detection result, providing a crucial component for subsequent detection result fusion.

[0017] Optionally, the step of inputting the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing regular targets specifically includes: inputting the complete image of the image to be detected into the preset benchmark target detection network, performing forward propagation calculation, and parsing the original full-image detection result based on the result of the forward propagation calculation; comparing the pixel size of each detected target in the original full-image detection result with a preset pixel size threshold, filtering out targets with pixel sizes larger than the preset pixel size threshold to form a preliminary set of regular targets; and performing non-maximum suppression processing on the preliminary set of regular targets based on the confidence score of each target in the preliminary set of regular targets and a preset second non-maximum suppression threshold to remove redundant detection boxes and obtain the second detection result.

[0018] By employing the aforementioned technical solution, the entire image to be detected is input into a pre-defined benchmark target detection network and forward propagation computation is performed. This fully leverages the benchmark network's excellent performance in conventional target detection tasks, rapidly acquiring salient target information from the image. Based on the results of the forward propagation computation, the original full-image detection result can be obtained by parsing the network output. This result includes attributes such as the category, location, and confidence score of each target predicted by the network across the entire image. Next, by comparing the pixel size of each detected target with a pre-defined threshold, targets larger than the threshold can be filtered out, forming a preliminary set of conventional targets, effectively eliminating potentially small targets and noise interference. Then, targets in the preliminary set of conventional targets are suppressed based on their confidence scores and a pre-defined non-maximum suppression threshold. By comparing the overlap between target bounding boxes, redundant and overlapping detection boxes can be removed, resulting in a refined conventional target detection result, i.e., the second detection result. This result complements the first detection result, together constituting a comprehensive detection of targets at different scales across the entire image, providing crucial input for the final detection result fusion. Meanwhile, since the benchmark network is pre-trained and optimized on conventional target detection tasks, it can ensure good detection results for targets of moderate size and clear appearance in the image, making up for the possible missed detection problem in small target detection methods and improving the completeness and accuracy of the detection results.

[0019] A second aspect of this application provides a joint detection device for small and regular targets based on YOLOv5. The device includes a target network construction module, a sliding window module, a small target detection module, a regular target detection module, and a result fusion module. The target network construction module is used to construct a target detection network based on a preset benchmark target detection network. The sliding window module is used to divide the image to be detected into multiple detection windows with overlapping regions. The small target detection module is used to input each of the detection windows into the target detection network to obtain preliminary detection results corresponding to each detection window. Non-maximum suppression processing is performed on the preliminary detection results of each detection window to obtain a first detection result containing small-sized targets. The regular target detection module is used to input the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing regular targets. The result fusion module is used to fuse the first detection result and the second detection result to generate a joint detection result and output the joint detection result.

[0020] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0022] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages:

[0023] 1. This technical solution preserves the detailed features of minute targets through sliding window segmentation and removes redundant detection boxes by combining non-maximum suppression, thus solving the problem of missed detection of minute targets in high-resolution images due to scaling. Simultaneously, it utilizes the complete image input to detect regular targets, fully leveraging YOLOv5's global perception capabilities. Finally, by fusing the detection results of minute and regular targets, it achieves joint detection of targets at multiple scales. It maintains both detection accuracy and efficiency, making it suitable for complex scenarios such as UAV aerial photography and industrial quality inspection. Attached Figure Description

[0024] Figure 1 This is a schematic flowchart of a method for joint detection of small-sized and conventional targets based on YOLOv5 disclosed in an embodiment of this application;

[0025] Figure 2 This is another flowchart illustrating a method for joint detection of small-sized and conventional targets based on YOLOv5 disclosed in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of a module of a small-sized and conventional target joint detection device based on YOLOv5 disclosed in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0028] Explanation of reference numerals in the attached figures: 301, Target network construction module; 302, Sliding window module; 303, Small target detection module; 304, Regular target detection module; 305, Result fusion module; 400, Electronic device; 401, Processor; 402, Communication bus; 403, User interface; 404, Network interface; 405, Memory. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0030] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0031] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0032] This application provides a method for joint detection of small-sized and conventional targets based on YOLOv5, referring to... Figure 1 , Figure 1This is a flowchart illustrating a method for joint detection of small and regular targets based on YOLOv5, provided in an embodiment of this application. The method is applied to a server, which executes the YOLOv5-based joint detection program for small and regular targets. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The method includes steps S101 to S106, as follows:

[0033] Step S101: Construct a target detection network based on a preset benchmark target detection network.

[0034] In step S101, the server constructs a target detection network optimized for small-sized targets based on a preset benchmark target detection network (such as a conventional YOLOv5 network). Specifically, the server introduces a channel attention mechanism and a feature pyramid structure into the original YOLOv5 network to enhance the network's ability to extract and fuse features of small-sized targets. Simultaneously, the server also makes appropriate adjustments and optimizations to the backbone network and detection head of the YOLOv5 network to meet the needs of small-sized target detection tasks.

[0035] In one possible implementation, please refer to Figure 2 , combined Figure 2 One embodiment of step S101 will be described in detail, including steps S201-S209, as follows:

[0036] Step S201: In the feature pyramid of the preset benchmark target detection network, add a feature map with a downsampling factor of the first preset value as a shallow feature map, and remove the feature map with a downsampling factor of the second preset value. The first preset value is less than the second preset value.

[0037] In step S201, the server modifies and optimizes the feature pyramid structure of the preset benchmark target detection network (such as the conventional YOLOv5 network) to enhance the network's ability to detect small-sized targets.

[0038] Specifically, the server first adds shallow feature maps with smaller downsampling factors to the existing YOLOv5 feature pyramid as feature representations specifically for detecting small targets. Here, the server sets the downsampling factor to a first preset value, which is smaller than the smallest downsampling factor in the original feature pyramid. By introducing these shallow feature maps with higher resolution, the network can better preserve the detailed information of small targets in the input image, providing richer and more refined feature representations for subsequent detection tasks.

[0039] Meanwhile, to further highlight the role of shallow feature maps in small target detection, the server removed deep feature maps with large downsampling factors from the original feature pyramid. These deep feature maps, after multiple downsampling and pooling operations, have relatively low resolution and are mainly suitable for detecting large targets, with limited expressive power for small targets. By removing these deep feature maps, the server reduces the computational load and parameter size of the network, improving detection efficiency; on the other hand, it also strengthens the weight and contribution of shallow feature maps in the overall feature pyramid, making the network more focused on the detection needs of small targets.

[0040] For example, suppose the original YOLOv5 feature pyramid contains three feature maps, P3, P4, and P5, with downsampling factors of 8, 16, and 32, respectively. The server adds two shallow feature maps, P1 and P2, to the optimized feature pyramid, with downsampling factors of 2 and 4, respectively, significantly higher than the resolution of the original feature maps. Simultaneously, the server removes the original P5 feature map, retaining only the three feature maps P1, P2, and P3 with downsampling factors of 2, 4, and 8 for subsequent detection tasks. In this way, the modified feature pyramid can extract more refined and complete target feature information within the same receptive field, especially significantly enhancing its ability to represent small-sized targets.

[0041] It should be noted that the specific values ​​of the first and second preset values ​​can be flexibly set according to the actual task requirements and network structure characteristics. Typically, the first preset value can be a smaller downsampling factor such as 2 or 4 to obtain sufficiently high-resolution shallow feature maps; while the second preset value can be a larger downsampling factor such as 16 or 32 to remove deep feature maps with insufficient resolution. The server can determine the optimal combination of the first and second preset values ​​through experimental comparison and parameter tuning, thereby maximizing the performance of small target detection.

[0042] Step S202: Perform global average pooling on the shallow feature map to obtain average pooling feature vectors, and perform global max pooling on the shallow feature map to obtain max pooling feature vectors.

[0043] In step S202, the server performs global average pooling and global max pooling operations on the shallow feature map, respectively, to obtain two independent feature vectors: the average pooling feature vector and the max pooling feature vector. Global average pooling, by averaging the values ​​of each channel in the shallow feature map, provides an overall representation of all features within that channel; while global max pooling, by extracting the maximum value within each channel, highlights the most significant and critical feature representations within that channel. These two types of pooling operations compress and summarize the shallow feature map from different perspectives, providing the necessary feature input for the subsequent attention mechanism.

[0044] Step S203: Input the average pooling feature vector and the max pooling feature vector into the first fully connected layer for channel dimensionality reduction to generate the corresponding first intermediate vector and second intermediate vector.

[0045] In step S203, the server inputs the average pooling feature vector and the max pooling feature vector into the first fully connected layer for channel dimensionality reduction. Through the nonlinear transformation and parameter mapping of this layer, the high-dimensional pooling feature vector is compressed into a lower-dimensional space, generating the corresponding first and second intermediate vectors. The purpose of channel dimensionality reduction is to reduce the complexity of subsequent calculations, and it also plays a role in feature selection and refinement, allowing the network to focus more on key features related to the small target detection task.

[0046] Step S204: Input the first intermediate vector and the second intermediate vector into the second fully connected layer for channel upscaling to restore the original number of channels of the shallow feature map, thereby generating the first processed vector and the second processed vector respectively.

[0047] In step S204, the server inputs the first and second intermediate vectors into the second fully connected layer for channel upscaling. Through the reverse mapping and dimension restoration of this layer, the compressed intermediate vectors are expanded back to the same channel number dimension as the shallow feature map, thereby generating the first and second processed vectors respectively. The purpose of channel upscaling is to perform element-wise multiplication with the original shallow feature map and apply attention weights to the corresponding channels to achieve adaptive feature enhancement and suppression.

[0048] For example, assuming the shallow feature map has a shape of [batch_size, channels, height, width], the server first uses global average pooling and global max pooling to obtain average pooling and max pooling feature vectors with shapes of [batch_size, channels]. Then, the server designs an attention module containing two fully connected layers. The first layer performs channel dimensionality reduction, compressing the number of channels in the input feature vector to 1 / 16 of its original value, resulting in an intermediate vector with a shape of [batch_size, channels / 16]. The second layer performs channel dimensionality enhancement, restoring the number of channels in the intermediate vector to the same as the shallow feature map, resulting in a processed vector with a shape of [batch_size, channels]. Finally, by multiplying the processed vector element-wise with the original shallow feature map, an adaptive attention weight is assigned to each pixel in the spatial dimension. This allows the network to focus more on the feature regions of small targets, suppressing background interference and thus improving detection accuracy.

[0049] It should be noted that the specific structure and parameter settings of the fully connected layer in the attention module can be adjusted and optimized according to actual needs. The ratio of channel dimensionality reduction to dimensionality increase can also be appropriately balanced, taking into account both computational efficiency and ensuring the effectiveness of the attention mechanism; this application does not impose any restrictions on this.

[0050] Step S205: Perform element-wise addition on the first processed vector and the second processed vector to generate a merged feature vector.

[0051] In step S205, the server performs element-wise addition on the first and second processed vectors, fusing their feature information in the corresponding channels to generate a merged feature vector. The purpose of this step is to comprehensively utilize the feature representations extracted by both average pooling and max pooling, considering both the overall distribution of all features within a channel and highlighting the most significant and critical feature responses within that channel. This allows the subsequently generated attention weights to more comprehensively and accurately reflect the important regions in the shallow feature map.

[0052] Step S206: Perform the Sigmoid activation function operation on the merged feature vector to map the values ​​of each element of the merged feature vector to the range of 0 to 1, thereby generating channel attention weights.

[0053] In step S206, the server performs a Sigmoid activation function operation on the merged feature vector, mapping each element value to the range of 0 to 1. Through the non-linear transformation of the Sigmoid function, the merged feature vector is transformed into a weight vector with the same number of channels as the shallow feature map, where each element represents the importance of the corresponding channel. Since the output range of the Sigmoid function is between 0 and 1, the generated weight vector can be directly used as channel attention weights for adaptive scaling and modulation of the shallow feature map.

[0054] Step S207: Multiply the channel attention weights by the shallow feature map to obtain the first feature map after channel attention adjustment.

[0055] In step S207, the server multiplies the channel attention weights element-wise with the shallow feature map to obtain the first feature map after channel attention adjustment. Specifically, the server multiplies each channel of the shallow feature map by its corresponding channel attention weight, thereby achieving adaptive enhancement or suppression of features in different channels. For channels closely related to the small target detection task, the corresponding attention weights are larger, making the features in these channels more amplified and highlighted; while for channels irrelevant to the task or that interfere with it, the corresponding attention weights are smaller, making the features in these channels correspondingly suppressed and weakened. Through this channel modulation based on the attention mechanism, the regions and details related to small targets in the first feature map are more vividly and clearly expressed, while background interference and irrelevant information are effectively filtered and suppressed, thus providing more refined and effective feature input for subsequent detection tasks.

[0056] For example, suppose the shape of the shallow feature map is [batch_size, channels, height, width]. The first and second processed vectors obtained in step S204 both have the shape [batch_size, channels]. In step S205, the server adds these two vectors element-wise to obtain a merged feature vector with the shape [batch_size, channels]. Then, in step S206, the server applies a sigmoid activation function to the merged feature vector, mapping each element to between 0 and 1, generating channel attention weights with the shape [batch_size, channels]. Finally, in step S207, the server copies and expands the channel attention weights along the channel dimension to the same shape [batch_size, channels, height, width] as the shallow feature map, and then multiplies it element-wise with the shallow feature map to obtain the first feature map adjusted by channel attention, whose shape is consistent with the shallow feature map.

[0057] Step S208: Perform spatial attention operation on the first feature map to generate spatial attention weights.

[0058] In step S208, the server performs a spatial attention operation on the first feature map to generate corresponding spatial attention weights. The purpose of the spatial attention mechanism is to adaptively adjust the importance of each pixel position in the feature map by learning a weight matrix of the same size as the input feature map, so that regions associated with small targets receive higher weights, while the weights of background regions are reduced accordingly.

[0059] To generate spatial attention weights, the server can adopt a design approach similar to channel attention mechanisms. First, the server performs max pooling and average pooling operations along the channel dimension on the first feature map, obtaining two two-dimensional spatial feature maps that highlight the most salient regions and the overall distribution, respectively. Then, the server adds these two spatial feature maps element-wise at corresponding positions to obtain a fused spatial feature map. Next, the server processes the fused spatial feature map through a convolutional layer to reduce its spatial size and extract high-level semantic information. Finally, the server uses another convolutional layer to restore the processed spatial feature map to the same size as the first feature map and applies a sigmoid activation function to convert it into spatial attention weights between 0 and 1.

[0060] Step S209: Adjust the first feature map based on the spatial attention weights to obtain the second feature map, and then obtain the object detection network.

[0061] In step S209, the server adjusts the first feature map based on spatial attention weights to obtain a second feature map, thereby completing the construction of the object detection network. Specifically, the server multiplies the spatial attention weights element-wise with the first feature map, assigning an adaptive weight value to each pixel in the first feature map. For pixel regions related to small targets, the corresponding spatial attention weights are larger, enhancing and highlighting the feature responses of these regions; while for background regions or irrelevant pixels, the corresponding spatial attention weights are smaller, suppressing and weakening the feature responses of these regions accordingly. Through this spatial dimension attention modulation, the regions of small targets in the second feature map become more focused and distinct, while background interference and irrelevant information are effectively filtered and suppressed, resulting in a feature representation optimized for small targets.

[0062] For example, suppose the shape of the first feature map is [batch_size, channels, height, width]. In step S208, the server first performs max pooling and average pooling on the first feature map along the channel dimension, obtaining two spatial feature maps with shapes [batch_size, 1, height, width]. Then, the server adds these two spatial feature maps at corresponding positions to obtain a fused spatial feature map with a shape of [batch_size, 1, height, width]. Next, the server uses a 1x1 convolutional layer to process the fused spatial feature map into a compressed feature map with a shape of [batch_size, 1, height / r, width / r], where r is a shrinkage factor. Finally, the server uses another 1x1 convolutional layer to restore the compressed feature map to the same shape [batch_size, 1, height, width] as the first feature map, and applies the Sigmoid function to obtain the spatial attention weights. In step S209, the server multiplies the spatial attention weights element-wise with the first feature map to obtain a second feature map with shape [batch_size, channels, height, width], which is used as the output of the object detection network.

[0063] Step S102: Divide the image to be detected into sliding windows to generate multiple detection windows with overlapping regions.

[0064] In step S102, the server divides the image to be detected into sliding windows, generating multiple detection windows with overlapping regions. Specifically, the server first determines the number and position of sliding windows based on the size of the image to be detected and preset sliding window parameters (such as sliding window size and sliding window step size). Then, the server slides the windows sequentially across the image to be detected from left to right and from top to bottom, treating each sliding window region as an independent detection window. There is a certain overlap between adjacent detection windows to ensure complete coverage of the entire image area.

[0065] In one possible implementation, the image to be detected is divided into sliding window segments to generate multiple detection windows with overlapping regions, specifically including steps S1021-S1024, as follows:

[0066] Step S1021: Set the minimum detection window size and the maximum detection window size according to the size range of the image to be detected.

[0067] In step S1021, the server sets the minimum and maximum detection window sizes based on the size range of the image to be detected. These two size parameters determine the range of the sliding window size and need to be set reasonably according to the actual task requirements and the target scale distribution. Generally, the minimum detection window size should be small enough to ensure that it can cover small targets in the image; while the maximum detection window size should be large enough to accommodate larger targets that may exist in the image. The server can determine the optimal minimum and maximum detection window sizes by statistically analyzing the size distribution of targets in the set of images to be detected, or by referring to commonly used empirical values ​​in the field.

[0068] Step S1022: Based on the minimum detection window size and the maximum detection window size, generate multiple window sizes with increasing dimensions, wherein the size ratio of any two adjacent window sizes is a preset value.

[0069] In step S1022, the server generates a sequence of progressively increasing window sizes based on the minimum and maximum detection window sizes. Specifically, starting with the minimum detection window size, the server gradually increases the window size using a preset size ratio as the step size until the maximum detection window size is reached. This preset size ratio is a hyperparameter that determines the scaling factor between two adjacent window sizes; commonly chosen values ​​are 1.2, 1.5, etc. By generating a series of progressively increasing window sizes, the server can adaptively adjust the size of the sliding window during the detection process to accommodate targets of different scales.

[0070] Step S1023: For each window size, calculate the corresponding sliding step length based on the preset overlap rate.

[0071] In step S1023, the server calculates the corresponding sliding step size for each window size based on a preset overlap rate. The overlap rate is an important parameter, representing the proportion of the window area where the overlapping region between two adjacent sliding windows occupies. Introducing the overlapping region aims to prevent targets from being truncated by window boundaries, thus avoiding missed detections. Based on the preset overlap rate (e.g., 0.2, 0.5, etc.) and the current window size, the server calculates the sliding window step size in both the horizontal and vertical directions, ensuring sufficient overlap between adjacent windows.

[0072] Step S1024: Based on the window size and the corresponding sliding step, the image to be detected is divided into sliding windows to obtain multiple detection windows with overlapping regions.

[0073] In step S1024, the server performs scale-wise sliding window division on the image to be detected based on the generated window size sequence and the corresponding sliding step size. Specifically, based on each window size, the server starts from the top left corner of the image to be detected and moves the window horizontally and vertically with the corresponding sliding step size, extracting image sub-regions sequentially until the entire image is covered. During the sliding process, due to overlapping areas, there will be a certain degree of pixel redundancy between adjacent detection windows, but this redundancy is necessary to improve the robustness and accuracy of detection. After sliding window division, the server finally obtains a set of detection windows of different sizes that overlap with each other, preparing for subsequent local target detection.

[0074] For example, assuming the image to be detected is 1920x1080 pixels, the server sets the minimum detection window size to 32x32, the maximum detection window size to 512x512, the preset size ratio to 1.5, and the preset overlap rate to 0.5. In step S1022, the server generates a sequence of window sizes: [32x32, 48x48, 72x72, 108x108, 162x162, 243x243, 364x364, 512x512]. In step S1023, the server calculates a sequence of sliding steps: [16, 24, 36, 54, 81, 121, 182, 256]. Finally, in step S1024, the server uses each window size and its corresponding sliding step to divide the image into sliding windows, resulting in a set of detection windows. Taking a 108x108 window size as an example, the server starts from the top left corner of the image and slides the window horizontally and vertically in steps of 54 pixels, extracting sub-regions of the image sequentially until the entire image is covered, resulting in a total of 720 detection windows of size 108x108. Detection windows of other sizes are generated in a similar manner, ultimately resulting in a set of 3240 overlapping detection windows of different sizes, which serve as input for subsequent local object detection.

[0075] Step S103: Input each detection window into the target detection network to obtain the preliminary detection results corresponding to each detection window.

[0076] In step S103, the server first preprocesses the image data of each detection window to the input format and size required by the network. Then, the server sequentially inputs each detection window into the network. After a series of operations, including feature extraction from the backbone network, feature fusion from the feature pyramid, and target classification and location regression by the detection head, information such as the category, confidence level, and bounding box position of the detected target within the detection window is obtained, serving as the preliminary detection result for that window. The server repeats the above process until all detection windows have obtained their corresponding preliminary detection results.

[0077] In one possible implementation, each detection window is input into the target detection network to obtain the preliminary detection result corresponding to each detection window, specifically including steps S1031-S1033, as follows:

[0078] Step S1031: Traverse each detection window and sequentially use each detection window as input data to feed into the object detection network to perform forward propagation calculation. Forward propagation calculation includes feature extraction, feature fusion, object classification, and bounding box regression.

[0079] In step S1031, the server iterates through all the detection windows obtained in step S1024 and sequentially inputs them into the improved YOLOv5-xs object detection network to perform forward propagation computation. Specifically, for each detection window, the server feeds its pixel data into the input layer of the YOLOv5-xs network. After a series of operations such as convolution, pooling, and attention mechanisms, multi-scale feature representations of the target within the window are extracted. During feature extraction, the image data within the detection window passes through the feature pyramid and attention module constructed in steps S201-S209, resulting in a pyramid-shaped feature map that integrates shallow high-resolution features and deep high-level semantic features. Next, the network combines feature maps of different scales through a feature fusion layer to form a unified feature representation containing multi-scale target information. Finally, the network uses fully connected layers and convolutional layers to complete the target classification and bounding box regression tasks, respectively, predicting the category, confidence, and bounding box position of each potential target within the detection window.

[0080] Step S1032: Parse the category information, confidence score, and bounding box position information of the potential targets within each detection window.

[0081] In step S1032, the server parses and extracts the output of the YOLOv5-xs network for each detection window. Specifically, the server first performs Softmax normalization on the network's classification output to obtain the probability distribution of the category to which each predicted box belongs within the detection window, and selects the category with the highest probability as the category label of that predicted box. Next, the server multiplies the confidence output of the predicted box by the category probability to obtain the final confidence score of each predicted box, representing the probability that the predicted box contains the target. Finally, based on the network's bounding box regression output, the server calculates the precise position and size of each predicted box in the detection window coordinate system, typically represented as the center coordinates, width, and height of the predicted box. Through this series of parsing operations, the server extracts key information about each potential target within the detection window from the output of the YOLOv5-xs network, including its category label, confidence score, and bounding box position.

[0082] Step S1033: Summarize the category information, confidence score and bounding box position information parsed from all detection windows to form the preliminary detection results corresponding to each detection window.

[0083] In step S1033, the server summarizes and integrates the parsing results of all detection windows to form a preliminary detection result set. Specifically, the server iterates through the parsing results of each detection window, retains the predicted bounding boxes with confidence scores higher than a preset threshold, and records their category information, confidence scores, and bounding box position information. Simultaneously, the server transforms the position coordinates of the predicted boxes from the local coordinate system of the detection window to the global coordinate system of the original image for subsequent result merging and visualization. After summarizing, the server obtains a preliminary detection result set containing all possible targets in the detection windows, preparing for subsequent non-maximum suppression and result fusion.

[0084] For example, suppose the server generates 3240 detection windows of different sizes in step S1024. In step S1031, the server sequentially inputs these detection windows into the YOLOv5-xs network for forward propagation calculation, obtaining the network output corresponding to each detection window. Taking a 108x108 detection window as an example, the network output may contain 20 predicted boxes, each with an 80-dimensional class probability vector and a 4-dimensional bounding box regression vector. In step S1032, the server parses the output of these 20 predicted boxes to obtain their class labels, confidence scores, and bounding box location coordinates. Suppose that after parsing, it is found that 5 of the predicted boxes have confidence scores higher than the threshold of 0.5, then these 5 predicted boxes will be retained, and their class information, confidence scores, and bounding box location information will be recorded. The server performs similar parsing and filtering on the output of 3240 detection windows, and finally obtains a preliminary detection result set containing about 15000 prediction boxes in step S1033, which serves as the input for subsequent processing.

[0085] Step S104: Perform non-maximum suppression processing on the preliminary detection results of each detection window to obtain a first detection result containing small-sized targets.

[0086] In step S104, the server first transforms the target bounding boxes in the preliminary detection results of each detection window from the window coordinate system to the original image coordinate system, and merges the bounding boxes of the same target detected by different windows. Then, the server sets a confidence threshold and an intersection-union (IU) threshold for non-maximum suppression (NUS), and performs NUS suppression processing on all bounding boxes in the preliminary detection results based on their categories. For each category, the server first sorts them by confidence from high to low, then selects the bounding box with the highest confidence as the benchmark, calculates its IU with other bounding boxes, and considers bounding boxes with IU higher than the IU threshold as redundant and deletes them. This process is repeated until no more redundant boxes can be deleted. Finally, the server uses the remaining bounding boxes after NUS suppression processing as the detection result for small-sized targets, i.e., the first detection result.

[0087] For example, suppose the image to be detected is a 1000×1000 aerial image, and the server sets the detection window size to 500×500 with a stride of 250. This allows the server to divide the image into 16 detection windows. The server sequentially inputs these 16 windows into the object detection network for inference, obtaining 16 preliminary detection results. Then, the server applies non-maximum suppression to these detection results using a confidence threshold of 0.5 and an IoU threshold of 0.6, ultimately obtaining the precise location and category information of small targets such as cars and pedestrians in the image, which is output as the first detection result.

[0088] In one possible implementation, non-maximum suppression processing is performed on the preliminary detection results of each of the detection windows to obtain a first detection result containing small-sized targets, specifically including steps S1041-S1046, as follows:

[0089] Step S1041: Traverse the preliminary detection results of each detection window, extract the target bounding box information, which includes the position coordinates, size and confidence score, and transform the position coordinates from the local coordinate system of the detection window to the global coordinate system of the image to be detected, so as to generate a set of target bounding boxes in the global coordinate system.

[0090] In step S1041, the server iterates through the preliminary detection results of all detection windows, extracting key information for each detection box, including its position coordinates, size, and confidence score. Due to differences in size and position between different detection windows, the server needs to transform the position coordinates of the detection boxes from their respective window's local coordinate system to the global coordinate system of the image to be detected, so that subsequent unified result merging and comparison can be performed. Specifically, the server performs a translation transformation on the center coordinates of each detection box based on the position offset of the detection window in the image to be detected, mapping it to the global coordinate system. Simultaneously, the server also needs to scale the width and height of the detection boxes according to the scaling ratio of the detection windows to ensure that their size in the global coordinate system matches the actual target. After coordinate transformation, the server obtains a set of target bounding boxes represented in the global coordinate system containing all detection boxes, preparing for subsequent grouping and suppression processing.

[0091] Step S1042: Based on the category information of each target bounding box in the target bounding box set, group the target bounding box set to generate multiple target subsets to be merged.

[0092] In step S1042, the server divides the target bounding box set into multiple target subsets to be merged based on the category information of each detection box. Since the distribution and features of different target categories in the image may differ, post-processing is required for detection boxes of different categories separately. The server iterates through the target bounding box set, grouping detection boxes belonging to the same category into a single target subset based on the category label of each detection box. After grouping, the server obtains several target subsets to be merged, where the detection boxes in each subset belong to the same category, thus preparing for subsequent non-maximum suppression processing.

[0093] Step S1043: For each subset of targets to be merged, select the target bounding box with the highest confidence score as the reference box.

[0094] Step S1044: Calculate the intersection-union ratio between the reference box and any target bounding box other than the reference box in the target subset to be merged.

[0095] Step S1045: Determine the remaining target bounding boxes whose intersection-union ratio is greater than the preset first non-maximum suppression threshold as redundant boxes, and remove them from the target subset to be merged to obtain multiple subsets after suppression processing;

[0096] In steps S1043-S1045, the server performs a non-maximum suppression algorithm on each target subset to be merged, removing redundant detection boxes within the subset. The basic idea of ​​non-maximum suppression is that for multiple overlapping detection boxes of the same target, only the one with the highest confidence score is retained, while the rest are considered redundant boxes and discarded. The server first sorts the detection boxes in each target subset in descending order according to their confidence scores. Then, starting with the detection box with the highest confidence score, the server calculates its intersection-union ratio (IoU) with other detection boxes in the subset, which is the area of ​​the intersection of two detection boxes divided by the area of ​​their union. If the IoU of a detection box with the current highest confidence detection box is greater than a preset first non-maximum suppression threshold (e.g., 0.5), it is marked as a redundant box. The server processes each detection box in the subset sequentially, finally obtaining a refined subset with redundant boxes removed. This non-maximum suppression process is performed on all target subsets, and the server obtains multiple refined subsets after suppression.

[0097] Step S1046: Merge multiple suppressed subsets to form the first detection result.

[0098] In step S1046, the server merges all refined subsets processed by non-maximum suppression into a final detection result set, namely the first detection result. The detection boxes in this result set have had redundancy and overlap between different detection windows and different categories removed, and each detection box corresponds to an independent tiny target instance in the image. The first detection result contains the category information, confidence score, and position and size information in the global coordinate system for each detection box, which can be used for subsequent target filtering, position correction, and visualization.

[0099] For example, suppose the server obtains a preliminary detection result set containing 15,000 detection boxes in step S1033. In step S1041, the server extracts the position coordinates, size, and confidence score of these detection boxes, and transforms their coordinates from the local coordinate system of their respective detection windows to the global coordinate system of the image to be detected, forming a global target bounding box set. In step S1042, the server divides the target bounding box set into 80 target subsets to be merged according to the category labels of the detection boxes, with the detection boxes in each subset belonging to the same category. In steps S1043-S1045, the server performs non-maximum suppression processing on each target subset, setting the first non-maximum suppression threshold to 0.5. Taking a target subset of one face category as an example, suppose this subset contains 1,000 detection boxes. After non-maximum suppression, 200 refined detection boxes with high confidence and low overlap remain. The server performs similar suppression processing on 80 target subsets. Finally, in step S1046, all refined detection boxes are merged into a first detection result set containing 3000 detection boxes, with each detection box corresponding to an independent tiny target instance in the image.

[0100] Step S105: Input the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing regular targets.

[0101] In step S105, the server inputs the complete image of the image to be detected into a preset benchmark target detection network (such as a conventional YOLOv5 network) to obtain a second detection result containing regular targets. Specifically, the server first preprocesses the complete image to be detected to the input format and size required by the benchmark network. Then, the server inputs the preprocessed image data into the benchmark network. After a series of operations, including feature extraction from the backbone network, feature fusion from the feature pyramid, and target classification and position regression by the detection head, the server obtains information such as the category, confidence level, and bounding box position of the detected targets throughout the entire image, which serves as the original full-image detection result. Next, the server sets a pixel size threshold (such as 50×50 pixels) to distinguish between regular targets and small targets. The server iterates through each target bounding box in the full-image detection result, calculating the number of pixels corresponding to its width and height. If the width and height of the target bounding box are both greater than the size threshold, it is considered a regular target and retained; otherwise, it is considered a small target and discarded. Finally, the server obtains a second detection result containing only regular targets.

[0102] For example, given a 1000×1000 input image, the server inputs it into a standard YOLOv5 network for a complete forward inference, obtaining all detected targets in the image along with their corresponding bounding boxes and category information. Then, the server sets a size threshold of 50×50 pixels for regular targets and judges the bounding box size of each detected target. Assuming there are 100 targets in the full image detection result, with 70 targets having bounding box sizes larger than 50×50 and 30 smaller than the threshold, the server considers these 70 targets as regular targets and outputs their category and location information as the second detection result; the remaining 30 small targets are discarded.

[0103] In one possible implementation, the complete image of the image to be detected is input into a preset benchmark target detection network to obtain a second detection result containing conventional targets, specifically including steps S1051-S1053, as follows:

[0104] Step S1051: Input the complete image of the image to be detected into the preset benchmark target detection network, perform forward propagation calculation, and parse the original full image detection result based on the result of the forward propagation calculation.

[0105] In step S1051, the server inputs the complete image data of the image to be detected into a preset benchmark target detection network and performs a forward propagation calculation. Similar to the processing of the detection window in step S1031, the benchmark target detection network extracts multi-scale feature representations of targets in the image through a series of convolution and pooling operations, and generates the original full-image detection result through tasks such as feature fusion, target classification, and bounding box regression. This original full-image detection result contains all target instances predicted by the network on the complete image, as well as their category information, confidence scores, and key attributes such as bounding box positions. Since the benchmark target detection network is pre-trained on a conventional target dataset, it has good detection capabilities for targets of moderate size and clear appearance in the image, but it may have omissions or false detections for small targets.

[0106] Step S1052: Compare the pixel size of each detected target in the original full-image detection result with the preset pixel size threshold, filter out targets whose pixel size is greater than the preset pixel size threshold, and form a preliminary set of regular targets.

[0107] In step S1052, the server performs preliminary screening on the original full-image detection results to filter out targets with smaller sizes. Specifically, the server iterates through each detection box in the original full-image detection results and calculates its pixel size, i.e., the number of pixels corresponding to the width and height of the detection box. Then, the server compares the pixel size of the detection box with a preset pixel size threshold. If the pixel size of the detection box is greater than the threshold, it is retained and regarded as a preliminary regular target; otherwise, if the pixel size of the detection box is less than or equal to the threshold, it is discarded and regarded as a possible small target or noise detection. Through this size screening process, the server obtains a preliminary set of regular targets, in which the detection boxes correspond to significant target instances with larger pixel sizes in the image, laying the foundation for subsequent non-maximum suppression processing.

[0108] Step S1053: Based on the confidence scores of each target in the preliminary regular target set and the preset second non-maximum suppression threshold, perform non-maximum suppression processing on the preliminary regular target set, remove redundant detection boxes, and obtain the second detection result.

[0109] In step S1053, the server performs non-maximum suppression (NMS) on the initial set of regular targets to further remove redundant and overlapping detection boxes, obtaining the final second detection result. Similar to step S1043, the server first sorts the detection boxes in the initial set of regular targets in descending order based on their confidence scores. Then, starting with the detection box with the highest confidence score, the server calculates its intersection-union (IoU) with other detection boxes in the set. If the IoU of a detection box with the current highest confidence detection box is greater than a preset second NMS threshold (e.g., 0.5), it is marked as a redundant box and removed. The second NMS threshold may differ from the first NMS threshold; its value can be adjusted appropriately based on the characteristics and distribution of regular targets to balance the precision and recall of the detection results. After NMS processing, redundant and overlapping detection boxes in the initial set of regular targets are effectively removed, resulting in a refined regular target detection result, i.e., the second detection result.

[0110] Combining the first detection result obtained in step S1044 and the second detection result obtained in step S1053, the server finally outputs a comprehensive set of target detection results, which includes both small targets in the image and regular targets of moderate size. This method of fusing local analysis of the detection window and overall detection of the entire image can effectively make up for the shortcomings of single detection methods, improve the detection capability of targets of different sizes, and achieve more robust and comprehensive target detection.

[0111] For example, suppose the server uses YOLOv3 as the preset baseline object detection network. In step S1051, the server inputs the complete image data of the image to be detected into the YOLOv3 network, performs one forward propagation calculation, and obtains an original full-image detection result containing 10,000 detection boxes. In step S1052, the server sets a preset pixel size threshold of 32x32, that is, detection boxes with a pixel size greater than 32x32 are considered regular targets. After size filtering, 8,000 detection boxes in the original full-image detection result meet the pixel size condition and are retained to form a preliminary set of regular targets. In step S1053, the server performs non-maximum suppression processing on these 8,000 detection boxes, setting a second non-maximum suppression threshold of 0.5. After non-maximum suppression, redundant detection boxes in the preliminary set of regular targets are removed, and a second detection result containing 2,000 detection boxes is finally obtained. The second detection result is merged with the first detection result obtained in step S1044, which contains 3,000 small target detection boxes. The server finally outputs a comprehensive target detection result set containing 5,000 detection boxes, covering target instances of various sizes in the image to be detected.

[0112] Step S106: Merge the first detection result and the second detection result to generate a joint detection result, and output the joint detection result.

[0113] In step S106, the server fuses the first detection result and the second detection result to generate a joint detection result, and outputs the joint detection result. Specifically, the server first aligns the position and size of the target bounding boxes in the first and second detection results to unify their coordinate systems. Then, the server compares the target bounding boxes in the two results one by one to determine whether they are duplicate detections of the same target. If the intersection-union ratio (IoU) of the two bounding boxes is greater than a preset threshold (e.g., 0.5), they are considered to correspond to the same target. For these overlapping bounding boxes, the server compares their confidence scores, retains the bounding boxes with higher confidence scores, and discards the bounding boxes with lower confidence scores. If the confidence scores are the same, the average of their position and size is taken as the final output. For bounding boxes that are not determined to be overlapping, the server directly adds them to the joint detection result. Finally, the server adds all the remaining small-sized target bounding boxes in the first detection result and all the remaining regular target bounding boxes in the second detection result to the joint detection result. As needed, the server can sort all bounding boxes in the joint detection results in descending order according to their confidence scores, and retain only the top N detection results with the highest confidence scores. N can be set according to actual requirements. At this point, the joint detection results are complete, containing complete information such as the category, location, size, and confidence score of all regular and small targets detected in the input image. The server outputs this joint detection result, which can be used for subsequent tasks such as target tracking and behavior analysis.

[0114] For example, suppose for the same 1000×1000 input image, the server obtains 50 small targets (mainly pedestrians, bicycles, etc.) in the first detection result and 20 regular targets (mainly cars, trucks, etc.) in the second detection result. The server first unifies the coordinate systems of the two results, and then uses an IoU threshold of 0.5 to determine overlapping bounding boxes. After processing, it is found that 5 small targets overlap with regular targets, so the detection result of the regular targets is used, and the corresponding small targets are discarded. Finally, the joint detection result contains a total of 65 targets (45 small targets + 20 regular targets). The server can further sort them by confidence and output the top 50 high-confidence targets as the final detection output.

[0115] Reference Figure 3This application also provides a joint detection device for small-sized and regular targets based on YOLOv5. This device is a server, comprising a target network construction module 301, a sliding window module 302, a small target detection module 303, a regular target detection module 304, and a result fusion module 305. Specifically: the target network construction module 301 constructs a target detection network based on a preset benchmark target detection network; the sliding window module 302 divides the image to be detected into multiple detection windows with overlapping regions; the small target detection module 303 inputs each detection window into the target detection network to obtain preliminary detection results corresponding to each detection window; performs non-maximum suppression processing on the preliminary detection results of each detection window to obtain a first detection result containing small-sized targets; the regular target detection module 304 inputs the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing regular targets; and the result fusion module 305 fuses the first and second detection results to generate a joint detection result and outputs the joint detection result.

[0116] In one possible implementation, the target network construction module 301 constructs a target detection network based on a preset benchmark target detection network. Specifically, the target network construction module 301 adds a feature map with a downsampling factor of a first preset value as a shallow feature map in the feature pyramid of the preset benchmark target detection network, and removes a feature map with a downsampling factor of a second preset value, where the first preset value is less than the second preset value; the target network construction module 301 performs a channel attention operation on the shallow feature map to generate channel attention weights; the target network construction module 301 multiplies the channel attention weights with the shallow feature map to obtain a first feature map adjusted by channel attention; the target network construction module 301 performs a spatial attention operation on the first feature map to generate spatial attention weights; the target network construction module 301 adjusts the first feature map based on the spatial attention weights to obtain a second feature map, thereby obtaining the target detection network.

[0117] In one possible implementation, the target network construction module 301 performs channel attention operations on the shallow feature map to generate channel attention weights. Specifically, this includes: the target network construction module 301 performs global average pooling on the shallow feature map to obtain an average pooling feature vector, and performs global max pooling on the shallow feature map to obtain a max pooling feature vector; the target network construction module 301 inputs the average pooling feature vector and the max pooling feature vector into the first fully connected layer for channel dimensionality reduction processing to generate corresponding first intermediate vectors and second intermediate vectors; the target network construction module 301 inputs the first intermediate vector and the second intermediate vector into the second fully connected layer for channel dimensionality increase processing to restore the original number of channels of the shallow feature map, thereby generating a first processed vector and a second processed vector respectively; the target network construction module 301 performs element-wise addition on the first processed vector and the second processed vector to generate a merged feature vector; the target network construction module 301 performs a Sigmoid activation function operation on the merged feature vector to map the values ​​of each element of the merged feature vector to the interval between 0 and 1, thereby generating channel attention weights.

[0118] In one possible implementation, the sliding window module 302 divides the image to be detected into multiple detection windows with overlapping regions. Specifically, the sliding window module 302 sets a minimum detection window size and a maximum detection window size according to the size range of the image to be detected; the sliding window module 302 generates multiple window sizes with increasing sizes based on the minimum and maximum detection window sizes, wherein the size ratio of any two adjacent window sizes is a preset value; for each window size, the sliding window module 302 calculates the corresponding sliding step size according to a preset overlap rate; the sliding window module 302 divides the image to be detected into multiple detection windows with overlapping regions based on the window size and the corresponding sliding step size.

[0119] In one possible implementation, the small target detection module 303 inputs each detection window into the target detection network to obtain preliminary detection results corresponding to each detection window. Specifically, the small target detection module 303 traverses each detection window and sequentially uses each detection window as input data, sending it into the target detection network to perform forward propagation calculation. The forward propagation calculation includes feature extraction, feature fusion, target classification, and bounding box regression. The small target detection module 303 parses the category information, confidence score, and bounding box position information of the potential target within each detection window. The small target detection module 303 summarizes the category information, confidence score, and bounding box position information parsed from all detection windows to form preliminary detection results corresponding to each detection window.

[0120] In one possible implementation, the small target detection module 303 performs non-maximum suppression processing on the preliminary detection results of each of the detection windows to obtain a first detection result containing small-sized targets. Specifically, the small target detection module 303 traverses the preliminary detection results of each of the detection windows, extracts target bounding box information, the target bounding box information including position coordinates, size, and confidence score, and transforms the position coordinates from the local coordinate system of the detection window to the global coordinate system of the image to be detected, so as to generate a set of target bounding boxes in the global coordinate system; the small target detection module 303, based on the category information of each target bounding box in the target bounding box set, performs non-maximum suppression processing on the target bounding box. The target bounding box set is grouped to generate multiple target subsets to be merged; the small target detection module 303 selects the target bounding box with the highest confidence score as the reference box for each target subset to be merged; the small target detection module 303 calculates the intersection-union ratio (IUR) between the reference box and any target bounding box in the target subset to be merged, excluding the reference box; the small target detection module 303 determines the remaining target bounding boxes whose IUR is greater than a preset first non-maximum suppression threshold as redundant boxes and removes them from the target subset to be merged, obtaining multiple suppressed subsets; the small target detection module 303 merges the multiple suppressed subsets to form the first detection result.

[0121] In one possible implementation, the conventional target detection module 304 inputs the complete image of the image to be detected into a preset benchmark target detection network to obtain a second detection result containing conventional targets. Specifically, the conventional target detection module 304 inputs the complete image of the image to be detected into the preset benchmark target detection network, performs forward propagation calculation, and parses the original full-image detection result based on the result of the forward propagation calculation; the conventional target detection module 304 compares the pixel size of each detected target in the original full-image detection result with a preset pixel size threshold, filters out targets with pixel sizes larger than the preset pixel size threshold, and forms a preliminary set of conventional targets; the conventional target detection module 304 performs non-maximum suppression processing on the preliminary set of conventional targets based on the confidence score of each target in the preliminary set of conventional targets and a preset second non-maximum suppression threshold, removes redundant detection boxes, and obtains the second detection result.

[0122] It should be noted that the devices provided in the above embodiments are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the devices and methods provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0123] This application also provides an electronic device. (See reference...) Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 400 may include: at least one processor 401, at least one network interface 404, a user interface 403, a memory 405, and at least one communication bus 402.

[0124] The communication bus 402 is used to enable communication between these components.

[0125] The user interface 403 may include a display screen and a camera. Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0126] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0127] The processor 401 may include one or more processing cores. The processor 401 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 405, and by calling data stored in memory 405. Optionally, the processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 401 and may be implemented as a separate chip.

[0128] The memory 405 may include random access memory (RAM) or read-only memory. Optionally, the memory 405 may include a non-transitory computer-readable storage medium. The memory 405 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 405 may also be at least one storage device located remotely from the aforementioned processor 401. (Refer to...) Figure 4 The memory 405, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a joint detection method of small-sized and conventional targets based on YOLOv5.

[0129] exist Figure 4 In the illustrated electronic device 400, the user interface 403 is mainly used to provide an input interface for the user and acquire user input data; while the processor 401 can be used to call an application program stored in the memory 405 that is a joint detection method for small-sized and conventional targets based on YOLOv5. When executed by one or more processors 401, the electronic device 400 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0130] This application also provides a computer-readable storage medium storing instructions. When executed by one or more processors 401, these instructions cause the electronic device 400 to perform one or more of the methods described in the above embodiments.

[0131] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0132] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0136] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0137] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for joint detection of small-sized and conventional targets based on YOLOv5, characterized in that, The method includes: Construct a target detection network based on a pre-defined benchmark target detection network; The image to be detected is divided into sliding windows to generate multiple detection windows with overlapping regions. Each of the detection windows is input into the target detection network to obtain the preliminary detection results corresponding to each of the detection windows; Non-maximum suppression processing is performed on the preliminary detection results of each detection window to obtain a first detection result containing small-sized targets; The complete image of the image to be detected is input into the preset benchmark target detection network to obtain a second detection result containing conventional targets; The first detection result and the second detection result are combined to generate a joint detection result, which is then output. The step of inputting each of the detection windows into the target detection network to obtain the preliminary detection results corresponding to each detection window specifically includes: Each of the detection windows is traversed, and each of the detection windows is used as input data in turn to perform forward propagation calculation in the target detection network. The forward propagation calculation includes feature extraction, feature fusion, target classification, and bounding box regression. The category information, confidence score, and bounding box position information of the potential targets within each detection window are parsed out. The category information, confidence score, and bounding box position information parsed from all detection windows are summarized to form the preliminary detection results corresponding to each detection window; The preliminary detection results of each of the detection windows are subjected to non-maximum suppression processing to obtain a first detection result containing small-sized targets, specifically including: The preliminary detection results of each detection window are traversed to extract target bounding box information, which includes position coordinates, size and confidence score. The position coordinates are then transformed from the local coordinate system of the detection window to the global coordinate system of the image to be detected to generate a set of target bounding boxes in the global coordinate system. Based on the category information of each target bounding box in the target bounding box set, the target bounding box set is grouped to generate multiple target subsets to be merged; For each subset of targets to be merged, the bounding box with the highest confidence score is selected as the reference box; Calculate the intersection-union ratio (IUU) between the reference bounding box and any target bounding box other than the reference bounding box within the target subset to be merged; The remaining target bounding boxes whose intersection-union ratio is greater than the preset first non-maximum suppression threshold are identified as redundant boxes and removed from the target subset to be merged, resulting in multiple suppressed subsets. The first detection result is formed by merging multiple subsets that have undergone suppression processing. The step of inputting the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing conventional targets specifically includes: The complete image of the image to be detected is input into the preset benchmark target detection network, forward propagation calculation is performed, and the original full image detection result is parsed based on the result of the forward propagation calculation. The pixel size of each detected target in the original full-image detection result is compared with a preset pixel size threshold, and targets with pixel sizes larger than the preset pixel size threshold are filtered out to form a preliminary set of regular targets; Based on the confidence scores of each target in the preliminary set of regular targets and the preset second non-maximum suppression threshold, non-maximum suppression processing is performed on the preliminary set of regular targets to remove redundant detection boxes and obtain the second detection result.

2. The method of claim 1, wherein, The construction of the target detection network based on the preset benchmark target detection network specifically includes: In the feature pyramid of the preset benchmark target detection network, a new feature map with a downsampling factor of a first preset value is added as a shallow feature map, and a feature map with a downsampling factor of a second preset value is removed. The first preset value is less than the second preset value. Perform channel attention operation on the shallow feature map to generate channel attention weights; Multiply the channel attention weights by the shallow feature map to obtain the first feature map after channel attention adjustment; Perform a spatial attention operation on the first feature map to generate spatial attention weights; The first feature map is adjusted based on the spatial attention weights to obtain the second feature map, and then the target detection network is obtained.

3. The method according to claim 2, characterized in that, The step of performing a channel attention operation on the shallow feature map to generate channel attention weights specifically includes: Perform a global average pooling operation on the shallow feature map to obtain an average pooling feature vector, and perform a global max pooling operation on the shallow feature map to obtain a max pooling feature vector. The average pooling feature vector and the max pooling feature vector are respectively input into the first fully connected layer for channel dimensionality reduction processing to generate the corresponding first intermediate vector and second intermediate vector. The first intermediate vector and the second intermediate vector are respectively input into the second fully connected layer for channel upsizing to restore the original number of channels of the shallow feature map, thereby generating the first processed vector and the second processed vector respectively. Perform element-wise addition on the first processed vector and the second processed vector to generate a merged feature vector; The sigmoid activation function is applied to the merged feature vector to map the values ​​of each element of the merged feature vector to the range of 0 to 1, thereby generating the channel attention weights.

4. The method of claim 1, wherein, The process of dividing the image to be detected into multiple detection windows with overlapping regions specifically includes: Based on the size range of the image to be detected, set the minimum detection window size and the maximum detection window size; Based on the minimum detection window size and the maximum detection window size, multiple window sizes with increasing dimensions are generated, wherein the size ratio of any two adjacent window sizes is a preset value; For each window size, the corresponding sliding step size is calculated based on the preset overlap rate; Based on the window size and the corresponding sliding step size, the image to be detected is divided into sliding windows to obtain multiple detection windows with overlapping regions.

5. A small size and general target joint detection device based on YOLOv5, characterized in that, The apparatus is used to perform the method as described in any one of claims 1-4, the apparatus comprising a target network construction module, a sliding window module, a small target detection module, a regular target detection module, and a result fusion module, wherein: The target network construction module is used to construct a target detection network based on a preset benchmark target detection network; The sliding window module is used to divide the image to be detected into sliding windows and generate multiple detection windows with overlapping regions. The small target detection module is used to input each of the detection windows into the target detection network to obtain the preliminary detection results corresponding to each detection window; and to perform non-maximum suppression processing on the preliminary detection results of each detection window to obtain a first detection result containing small-sized targets. The conventional target detection module is used to input the complete image of the image to be detected into the preset benchmark target detection network to obtain a second detection result containing conventional targets; The result fusion module is used to fuse the first detection result and the second detection result to generate a joint detection result and output the joint detection result.

6. An electronic device, comprising: The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-4.

Images