A dense small target detection method based on IDT-YOLOv5-CBAM hybrid algorithm

Abstract

The present invention discloses a dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm. The method optimizes the k-means algorithm using fuzzy ISODATA dynamic clustering to avoid the occurrence of local optimal solutions. A detection layer is embedded to make the receptive field of each gridcell corresponding to the original image smaller, thereby enhancing the detection capability of small targets. At the same time, a parallel strategy of add is adopted to fuse multi-scale features, so that they retain more detailed information while having high-level semantic information. The CBAM attention mechanism module is introduced into the Neck network in the model to achieve a comprehensive balance between spatial and channel dimensions. At the same time, upsampling is replaced by deconvolution. The learning ability of deconvolution is used to allow the network to learn how to upsample, thereby enhancing the detection effect of small targets in the target image. The present invention can quickly and accurately identify as many detection targets as possible, greatly reducing the missed detection rate and false detection rate of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision target detection, and in particular relates to a dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm. Background Art

[0002] With the continuous development of deep learning, object detection technology has gradually shifted from traditional manual detection methods to detection methods based on deep neural networks. Among the many deep learning-based object detection methods, single-stage deep learning-based object detection methods are widely used due to their simple network structure, fast operation speed, and higher detection efficiency.

[0003] The existing single-stage target detection method based on deep learning has unsatisfactory detection effects on small target objects during the detection process due to the fact that small target objects contain less feature information, lower resolution, more complex background information, unclear detail information and higher positioning accuracy requirements, which reduces the model detection accuracy.

[0004] In general, currently widely used deep learning object detection methods achieve good results for detecting large, lightly occluded objects. However, detecting smaller, densely packed, and heavily occluded objects, such as fruit on trees, vehicles in aerial images, and crowds, remains challenging. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm. To address the problem that the YOLOv5 algorithm has low accuracy in dense target detection, the fuzzy ISODATA dynamic clustering technology is introduced on the basis of the original YOLOv5 model, a detection layer is added, and a CBAM attention mechanism is introduced, so that the model can quickly and accurately identify as many detection targets as possible, greatly reducing the missed detection rate and false detection rate of the model.

[0006] Technical solution: A dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm, including the following steps:

[0007] (1) Preprocess the pre-acquired small target images to be detected and create a data set:

[0008] (2) Improve the original YOLOv5 network structure to obtain the improved IDT-YOLOv5-CBAM network structure;

[0009] (3) Adjust the parameters of the improved network model: In the IDT-YOLOv5-CBAM network model, set the size of the convolutional neural network input image, the number of recognition types, and the number of iterations according to the recognition effect and training speed required by the user;

[0010] (4) The improved model is evaluated on the trained IDT-YOLOv5-CBAM network model in terms of accuracy, recall rate and Map value.

[0011] Furthermore, the step (1) includes the following steps:

[0012] (11) Image acquisition: The user uses a mobile phone, digital camera or other image acquisition device to capture images of the small target to be detected;

[0013] (12) Image annotation: annotate the collected images and mark the category and location information of small targets;

[0014] (13) Image augmentation: Use the Augmentor image data augmentation library to augment the image; by adding noise, horizontal mirroring, vertical mirroring, cropping, rotating, translating, and adjusting brightness, the sample is augmented to meet the recognition requirements;

[0015] (14) Dataset division: The amplified images are divided into training set and validation set in a ratio of 8:2.

[0016] Furthermore, the step (2) includes the following steps:

[0017] (21) Add a detection layer based on the original three detection layers of YOLOv5;

[0018] (22) Based on the embedding of the fourth detection layer, the 20th layer is concat-fused with the 2nd layer, the 25th layer is concat-fused with the 4th layer, and the 30th layer is concat-fused with the 6th layer;

[0019] (23) The CBAM attention mechanism module is introduced into the Neck network in YOLOv5 to achieve full consideration of both spatial and channel dimensions;

[0020] (24) The original upsampling in the Neck layer of the original model is replaced with deconvolution, and the deconvolution module is used to enhance the resolution of the feature map;

[0021] (25) Considering the characteristics of low-level features with low semantics and high details and high-level features with high semantics and low details, the parallel strategy add is adopted to replace the Concat fusion operation in the Neck layer;

[0022] (26) The k-means algorithm is optimized by using the fuzzy ISODATA dynamic clustering method. By adding a fuzzy matrix and adding the merging and splitting steps in the iterative process, the size of the K value is dynamically determined to avoid the occurrence of local optimal solutions.

[0023] Furthermore, the implementation process of step (3) is as follows:

[0024] Image size imgsz = [640, 640], confidence threshold conf_thres = 0.5, Iou threshold iou_thres = 0.45, initial learning rate Ir = 0.0032, learning rate momentum momentum = 0.843, weight decay coefficient weight_decay = 0.00036, set epoch = 200.

[0025] Furthermore, the added detection layer described in step (21) specifically corresponds to the 17th to 21st layers of the overall structure. It first passes through a CSP2_1 module and a CBS module, and then a deconvolution operation is performed by the 19th layer. The output result is Add fused with the second layer, and finally the CSP2_1 module of the 21st layer outputs the result of the Add fusion, so that the output feature map becomes a 4-fold downsampling operation of the original input.

[0026] Furthermore, the implementation process of step (22) is as follows:

[0027] The 80×80×64 feature map outputted at the 18th layer is upsampled to high-level feature information of 160×160×64, which is then concat-fused with the 160×160×64 low-level feature information of the second layer. The 80×80×128 feature map outputted at the 24th layer by the CSP2_1 module is concat-fused with the low-level feature information of the 4th layer of the same size. The 40×40×256 feature map outputted at the 29th layer by the CSP2_1 module is concat-fused with the low-level feature information of the 6th layer of the same size.

[0028] Furthermore, the implementation process of step (25) is as follows:

[0029] Assume that the two input channels are A1, A2...A c and B1, B2…B c , the formula is as follows

[0030]

[0031]

[0032] Where, c represents the number of channels; A i 、B i Respectively represent the two input channels; K i Represents the convolution kernel of the corresponding channel.

[0033] Furthermore, the step (26) includes the following steps:

[0034] S1: Set the expected number of anchors K0 = 12, randomly select cluster centers, and initialize 12 anchors;

[0035] S2: Calculate the membership matrix U:

[0036]

[0037] Among them, K means there are K cluster centers; ij is an element in the membership matrix U; d ij represents the distance from the jth sample to the i-th cluster center; d pj represents the distance from the jth sample to the pth cluster center; m represents the degree of inter-class fuzziness;

[0038] S3: Calculate cluster centers:

[0039]

[0040] Among them, Z i represents the i-th cluster center; N represents the total number of samples; L represents the number of iterations;

[0041] S4: Determine whether to split. If one of the following conditions is met, split is performed: ① The final number of clusters is less than 1 / 2 of the expected value or the final number of clusters is greater than 2 times the expected value; ② The number of iterative operations is an even number; ③ The maximum value of the standard deviation vector of the sample distances in the same category is greater than the threshold of the sample standard deviation, that is, σ jmax >θ s , where θ s Is the threshold of the sample standard deviation, split into new cluster centers Z1 and Z2:

[0042] Z1=Z i +fσ jmax Z1=Z i -fσ jmax (14)

[0043] Among them, σ jmax Represents the maximum value of the sample distance standard deviation vector; f represents the splitting coefficient, which satisfies 0 <f<1;

[0044] S5: Determine whether to merge. If one of the following conditions is met, merge: ① Calculate the distance D between each class. ij , if it is less than the threshold value of the distance between the centers of various samples, that is, D ij <θ k ,θ k is the threshold value of the distance between the centers of each type of samples; ② If the number of samples in a certain class is less than the specified number threshold θ n , the new cluster center is Z k :

[0045] Z k =(N i Z i +N j Z j ) / (N i +N j ) (15)

[0046] Among them, Z i , Z j Represents the sample center of class i and class j respectively; N i 、N j Represents the number of samples of class i and class j respectively;

[0047] S6: Calculate the new clustering results and calculate the distance d from each sample to each cluster center ij ;

[0048] S7: Calculate the new membership matrix according to formula (12);

[0049] S8: Return to step S3 and repeat the iteration until the iteration is completed.

[0050] Compared with the prior art, the present invention has the following beneficial effects: 1. To address the problem that the size and number of the original anchorbox are not suitable for the self-made data set, the present invention adopts the fuzzy ISODATA dynamic clustering method to optimize the k-means algorithm. By adding a fuzzy matrix and adding two steps of merging and splitting in the iterative process, the size of the K value is dynamically determined to avoid the occurrence of local optimal solutions as much as possible; 2. To address the defect that the original YOLOv5 model has poor detection effect on small target objects, the present invention adds a detection layer on the basis of the original three detection layers, so that the receptive field of each gridcell corresponding to the original image is smaller. Enhance the detection capability of small targets; 3. In response to the problem that the original YOLOv5 model has low detection accuracy for small target objects, the present invention uses the parallel strategy add to replace the Concat fusion operation. On the one hand, it increases the amount of information in each dimension; on the other hand, the corresponding feature maps share a convolution kernel, which greatly reduces the number of parameters during calculation; while effectively improving the detection accuracy, it ensures the detection speed; 4. In response to the defect that the original YOLOv5 model has poor effect on image feature extraction, the present invention introduces the CBAM attention mechanism module in the Neck network in the model to achieve full consideration of both spatial dimension and channel dimension, thereby improving the effect of feature fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the present invention;

[0052] Figure 2 Schematic diagram of the IDT-YOLOv5-CBAM model structure;

[0053] Figure 3 This is the structural diagram of the CBAM attention mechanism;

[0054] Figure 4 This is the recognition effect of the original YOLOv5 model on an apple on a single fruit tree;

[0055] Figure 5 This is the recognition effect of the IDT-YOLOv5-CBAM model on apples on a single fruit tree;

[0056] Figure 6 This is the recognition effect of the original YOLOv5 model on a large cluster of apples on multiple fruit trees;

[0057] Figure 7 This is the recognition effect of the IDT-YOLOv5-CBAM model on large-scale clustered apples on multiple fruit trees. DETAILED DESCRIPTION

[0058] The present invention will be described in further detail below with reference to the accompanying drawings.

[0059] The purpose of this invention is to provide a dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm. In order to solve the problem that the YOLOv5 algorithm has low detection accuracy for dense targets, the fuzzy ISODATA dynamic clustering technology, the increase of detection layers, and the introduction of the CBAM attention mechanism are used on the basis of the original YOLOv5 model to enable it to quickly and accurately identify as many detection targets as possible, greatly reducing the missed detection rate and false detection rate of the model. Figure 1 As shown, the specific steps include:

[0060] Step 1: Make a dataset.

[0061] Use a mobile phone, digital camera, or other image acquisition device to capture images of apples on a fruit tree. Name the images according to the format specified by the YOLOv5 dataset. Create two folders, named images and labels, and within each folder, create two folders, named train and val. Use LabelImg to label the apples in the captured images. First, enter conda activate pyxxx in the command prompt to activate the virtual environment; then, enter LabelImg to start the program. Next, label the apples' categories and locations. Finally, save the annotations.

[0062] The following rules should be followed when annotating images:

[0063] 1) When the box is selected as apple, the label can be named apple.

[0064] 2) When selecting densely clustered and overlapping apples, select them one by one accurately by hand.

[0065] 3) If the occlusion of the frame exceeds 90%, the current target is discarded.

[0066] 4) If the target area is smaller than 8×8 pixels, the current target is discarded.

[0067] If the images collected by the user cannot meet the requirement of 2,000 images required to identify a single variety, in the case of a small number of samples, in order to meet the training of deep convolutional neural networks and improve the generalization of the model, data augmentation is undoubtedly the preferred means to solve this problem. On the one hand, data amplification can expand the sample size of the data, and on the other hand, it can increase the diversity of samples. Users can use the Augmentor image data enhancement library to amplify images; users select the image save path and the tag information txt file path, and formulate the amplified image and txt file output path. The present invention mainly achieves sample amplification by adding noise, horizontal mirroring, vertical mirroring, cropping, rotation, translation and brightness adjustment to the sample to meet the required recognition requirements.

[0068] In the images folder, store images for testing, training, and validation in the "train" and "val" folders, respectively, with a ratio of 80% and 20%. The corresponding "labels" folder stores the labels, which should be in the form of a txt file.

[0069] Step 2: Improve the original YOLOv5 network structure to obtain the improved IDT-YOLOv5-CBAM network structure, as shown in Figure 2 shown.

[0070] (1) A detection layer is embedded in the original YOLOv56.0 version to improve the detection capability of small-sized objects. The original YOLOv5 network structure has three detection layers, and the output size is 8x, 16x, and 32x downsampling of the input image respectively. If the original image size input is 640×640, then the feature maps will be 80×80, 40×40, and 20×20 respectively. By embedding a detection layer, the output feature map becomes 4x, 8x, 16x, and 32x downsampling of the original input image. The 160×160 feature map divides the image to be detected into 160×160 grid cells. The receptive field of each grid cell corresponding to the original image is the smallest, which is conducive to the detection of extremely small objects. Similarly, small objects are processed by the 80×80 feature map, medium objects by the 40×40 feature map, and large objects by the 20×20 feature map.

[0071] Specifically, layers 17 to 21 of the overall structure first pass through a CSP2_1 module and a CBS module. Next, a deconvolution operation is performed in the 19th layer, and the output result is fused with the Add operation in the second layer. Finally, the CSP2_1 module in the 21st layer outputs the Add fusion result, achieving a feature map that is downsampled four times the original input.

[0072] (2) Based on the embedding of the fourth detection layer, the present invention performs concat multi-scale fusion on the 20th layer with the 2nd layer, the 25th layer with the 4th layer, and the 30th layer with the 6th layer. Because the resolution of low-level features is higher, they contain more information such as position and details, but due to the small number of convolution layers, their semantics is lower and there is more noise. High-level features carry a lot of semantic information, but the resolution is too low, so the perception of details is poor. If only the last feature map is used for detection, a lot of detail information will be lost, but if low-level features are used for detection, semantics will be lost. Therefore, no matter which feature map is used, the detection effect will be affected. Therefore, the present invention performs an upsampling operation on the 80×80×64 feature map output by the 18th layer to convert it into high-level feature information of 160×160×64, and then concat fusion is performed with the low-level feature information of 160×160×64 in the second layer. Similarly, the present invention concatenates the 80×80×128 feature map of the 24th layer, which passes through the CSP2_1 module, with the lower-level feature information of the 4th layer. The 40×40×256 feature map of the 29th layer, which passes through the CSP2_1 module, is concatenated with the lower-level feature information of the 6th layer. This allows them to retain more detailed information while retaining high-level semantic information, effectively improving detection accuracy.

[0073] (3) The Convolutional Block Attention Module (CBAM) is introduced into the Neck network in the YOLOv5 model; Figure 3 As shown in the figure, based on the above model improvements, the CBAM attention mechanism is introduced in layers 11, 16, and 21, respectively. It is introduced in the Neck layer because the weights of the CBAM attention mechanism module are randomly initialized. If introduced into the backbone network, it may destroy the backbone network weights. The backbone network features are the basis for subsequent feature extraction, which may ultimately make the extracted features less useful. CBAM is an attention mechanism based on convolutional modules. It combines spatial and channel-wise weighted fusion of input features, and is more effective in feature fusion than a single channel-wise attention mechanism (SENet).

[0074] The convolution operation essentially discovers the characteristics of each region of the input image. This characteristic is a high-dimensional semantic feature, with each channel responsible for its own information. The channel attention mechanism adaptively determines which parts are more important and increases their corresponding weights, while reducing the weights of less important parts. The spatial attention mechanism focuses more on which areas are more important in the spatial dimension. For example, in face recognition, all parts of an image other than the face are obviously unimportant, so the spatial attention mechanism automatically reduces their weights. The CBAM attention mechanism comprehensively considers both spatial and channel dimensions, focusing not only on the weight of channels but also on the weight of each pixel.

[0075] The CBAM attention mechanism is a combination of the channel attention mechanism and the spatial attention mechanism. First, the feature map of size h×w×c is input into the channel attention mechanism module, and the most meaningful feature F1 is obtained through formula (1). Then, the enhanced feature is input into the spatial attention mechanism module, and the most informative feature F2 is obtained through formula (2).

[0076]

[0077]

[0078] Among them, F represents the feature map of size h×w×c; F1 represents the feature after the channel attention mechanism is enhanced for the input feature F; F2 represents the feature after the spatial attention mechanism is enhanced for the enhanced feature F1; M c (F) represents the weight parameter learned after the channel attention mechanism is strengthened on the input feature F; M s (F1) represents the weight parameter learned after the spatial attention mechanism is strengthened on the enhanced feature F1; Represents element-wise dot product.

[0079] In the channel attention mechanism, for a feature map with an input and output of h×w×c, the MaxPool (global maximum pooling) and AvgPool (global average pooling) operations are first performed on each pixel in the spatial dimension, respectively, and feature compression in the spatial dimension is achieved through formulas (3) and (4), respectively. The original h×w×c feature map is converted into a 1×1×c tensor, resulting in c scalars.

[0080]

[0081]

[0082] Among them, Zc Represents the resulting tensor; h represents the high dimension of the input feature; w represents the wide dimension of the input feature; F c represents the input feature with c channels; F c (i,j) represents the position in the feature.

[0083] The two resulting 1×1×c tensors are fed into the SharedMLP (a shared fully connected layer), where they undergo two full connections: FC -> ReLu -> FC -> Sigmoid. This achieves a dimensionality reduction followed by dimensionality increase. The goal of dimensionality reduction is to reduce the number of parameters while maintaining good performance and improving computational speed. The goal of dimensionality increase is to ultimately obtain more informative features. The ReLu activation function is used after the first full connection, and the Sigmoid activation function is used after the second full connection.

[0084] The two 1×1×c tensors output by the above results are first added together, and then activated with the Sigmoid function to generate the weight M c , as shown in formula (5).

[0085] M c (F) = σ(MLP(F max )+MLP(F avg )) (5)

[0086] Among them, M c (F) represents the weight parameter learned after the channel attention mechanism is strengthened on the input feature F; F max Indicates the global maximum pooling operation on the input feature F; F avg Indicates a global average pooling operation on the input feature F; MLP(F) indicates a shared full connection operation on the input feature F; σ indicates a Sigmoid activation operation.

[0087] The M learned from the above operation c (The weight parameters corresponding to each channel) are multiplied by all pixels of the corresponding channel of the input to enhance important features and weaken unimportant features.

[0088] In the spatial attention mechanism, for a feature map with an input and output of h×w×c, MaxPool (global maximum pooling) and AvgPool (global average pooling) operations are first performed on each pixel in the channel dimension to achieve feature compression in the channel dimension. The original h×w×c feature map is converted into a w×h×1 tensor.

[0089] The two w×h×1 tensors formed above are first concat-fused to double the number of channels. Then a convolution operation is performed to halve the number of channels, and finally a Sigmoid function is used for activation to generate the weight M. s , as shown in formula (6).

[0090] M s (F1)=σ(Concat 7×7 [F 1,max ; F 1,avg ]) (6)

[0091] Among them, M s (F1) represents the weight parameter learned after the spatial attention mechanism is strengthened for the enhanced feature F1; F 1,max Indicates the global maximum pooling operation on the input feature F1; F 1,avg Indicates a global average pooling operation on the input feature F1; Concat 7×7 [F 1,max ; F 1,avg ] indicates the use of 7×7 convolution kernel for F 1,max and F 1,avg Perform Concat fusion.

[0092] The M learned from the above operation s (The weight parameters corresponding to each channel) are multiplied by the input feature map.

[0093] (4) Based on the above improvements, the present invention replaces the original upsampling in the Neck layer with deconvolution. This is because a lot of contextual information is included in the detection process. We replace the original upsampling in the YOLOv5 Neck layer with deconvolution, and use the deconvolution module to enhance the resolution of the feature map. Then, in the process of fusing with the underlying convolutional network, a higher-resolution feature map can be obtained. The learning ability of deconvolution allows the network to learn how to upsample, thereby enhancing the detection effect of small targets in the target image. Compared with the upsampling method, deconvolution not only realizes pixel filling, but also adds a parameter learning process, which can obtain more small target information while increasing the resolution of the feature map.

[0094] The present invention uses dilated convolution to perform deconvolution. Dilated convolution is to add intervals to the basic convolution kernel. For example, in conventional convolution, a 3×3 convolution kernel has a dilationrate of 1 and an interval of 0. If a convolution with an interval of 1 is performed, the corresponding dilationrate becomes 2, and the corresponding image block is expanded from 3×3 to 7×7. The receptive field will increase with the increase of the convolution kernel. Taking the 12th layer as an example, the 20×20 feature map passed in by the previous layer is deconvolved twice and then fused with the 40×40 feature map of the 6th layer to obtain more feature semantic information of small targets. The formula is as follows, where the convolution kernel size is 4×4, the step size is 2, and the padding is 1.

[0095] y=(x-1)s+k-2p (7)

[0096] Where x represents the input feature map size; y represents the output feature map size; s represents the stride; k represents the convolution kernel size; and p represents padding.

[0097] (5) The present invention adopts the parallel strategy add to replace the Concat fusion operation in the Neck layer. This is because Concat fusion only merges the number of channels, doubling the number of channels in the feature map, while the information under each channel does not increase. Although the dimension does not increase after add performs feature fusion, the amount of information under each dimension is increased. At the same time, each channel of Concat corresponds to a corresponding convolution kernel. The add form adds the corresponding feature maps and then performs the next convolution operation, which is equivalent to adding a priori: the feature maps of the corresponding channels are semantically similar, so the corresponding feature maps share a convolution kernel, which greatly reduces the number of parameters during calculation. When only looking at the output of a single channel, let the channels of the two inputs be A1, A2...A c and B1, B2…B c , the formula is as follows

[0098]

[0099]

[0100] Where c represents the number of channels; A i 、B i Respectively represent the two input channels; K i Represents the convolution kernel of the corresponding channel.

[0101] (6) The present invention uses the fuzzy ISODATA dynamic clustering method to optimize the k-means algorithm. In the original model of YOLOv5, the k-means algorithm was used to cluster the public dataset MS COCO to obtain 9 anchor boxes. This dataset contains 20 categories of targets, ranging from large cars to small cats and dogs, and the target sizes vary greatly. Both the number and size of anchor boxes are not applicable to our self-made dataset. Especially for the new categories in the dataset: large-scale clustered apples and dense crowds of masks, using the original anchor will reduce the accuracy of YOLOv5 model detection and affect model performance.

[0102] The traditional k-means algorithm uses Euclidean distance as a metric, which causes large-sized Boundingboxes to produce more errors than small-sized Boundingboxes. Therefore, we introduce a new distance formula, the IoU distance formula:

[0103] d (box,centriod) =1-IoU (box,cfntriod) (10)

[0104] The objective function is:

[0105]

[0106] Among them, box represents the Bounding box (prediction box); cfntriod represents the center of the Bounding box; IoU represents the intersection over union ratio of the prediction box to the real box; k represents the number of anchors; truth represents the real box.

[0107] The original k-means clustering algorithm has the disadvantage that the k value must be manually set and cannot be changed once it is set. This makes it difficult to estimate an appropriate k value for large and high-dimensional datasets. Furthermore, the random selection of the initial k value in the k-means clustering algorithm can easily lead to the results falling into a local optimum.

[0108] The present invention adopts the method of fuzzy ISODATA dynamic clustering to improve the shortcoming of k-means algorithm that cannot automatically determine the k value. Its core idea is not only to increase the fuzzy matrix, but also to add two steps of merging and splitting in the iterative process. If there are too few samples in a certain class of clustering results, or the distance between classes is too close, then they are merged into one class. If the intra-class variance in a certain class is too large, it means that the samples in the class have different shapes and are not suitable to be placed in the same class, then they are classified into two classes. Of course, the number of categories will not be unlimited. If the expected clustering result is K0, then the range of the final clustering result is

[0109] The steps of fuzzy ISODATA dynamic clustering are as follows:

[0110] S1: Since the improved network model embeds the detection layer, the number of anchors increases from 9 to 12. Set the expected K0 = 12, randomly select the cluster center, and initialize 12 anchors.

[0111] S2: Calculate the membership matrix U according to formula (12).

[0112]

[0113] Among them, K means there are K cluster centers; μ ij is an element in the membership matrix U; d ij represents the distance from the jth sample to the i-th cluster center; d pj represents the distance from the jth sample to the pth cluster center; m represents the degree of inter-class fuzziness.

[0114] S3: Calculate the cluster center based on the membership matrix U and formula (13).

[0115]

[0116] Among them, Z i represents the i-th cluster center; N represents the total number of samples; L represents the number of iterations.

[0117] S4: Determine whether to split. If one of the following conditions is met, split is performed: Condition 1: the final number of clusters is less than 1 / 2 of the expected value or the final number of clusters is greater than 2 times the expected value; Condition 2: the number of iterative operations is an even number; Condition 3: the maximum value of the standard deviation vector of the sample distances in the same category is greater than the threshold of the sample standard deviation, that is, σ jmax >θ s (Threshold of sample standard deviation) split into new cluster centers Z1 and Z2.

[0118] Z1=Z i +fσ jmax Z1=Z i -fσ jmax (14)

[0119] Among them, σ jmax Represents the maximum value of the sample distance standard deviation vector; f represents the splitting coefficient, which satisfies 0 <f<1。

[0120] S5: Determine whether to merge. If one of the following conditions is met, merge: Condition 1: Calculate the distance D between each class. ij , if it is less than the threshold value of the distance between the centers of various samples, that is, D ij <θk (the threshold value of the distance between the centers of each type of samples); Condition 2: if the number of samples in a certain class is less than the specified number threshold θ n , the new cluster center is Z k .

[0121] Z k =(N i Z i +N j Z j ) / (N i +N j ) (15)

[0122] Among them, Z i , Z j Represents the sample center of class i and class j respectively; N i 、N j Represent the number of samples of class i and class j respectively.

[0123] S6: Calculate the new clustering results and calculate the distance d from each sample to each cluster center ij .

[0124] S7: Calculate a new membership matrix according to formula (12).

[0125] S8: Return to step S3 and repeat the iteration until the iteration is completed.

[0126] Step 3: Adjust the parameters of the improved network model: In the IDT-YOLOv5-CBAM network model, set the convolutional neural network input image size, number of recognition types, and number of iterations based on the computer memory and video memory size, the user's desired recognition effect, and training speed. Users must also use a graphics card that supports CUDA acceleration.

[0127] The present invention is based on the versions of PyTorch 1.8 and Python 3.8 for testing. During the experiment, an NVIDIA GeForce RTX 2080 Ti GPU is used for calculation, with 16G memory, 11G video memory, and CUDA version 10.2.

[0128] Before training, the model parameters are configured first, image size imgsz = [640, 640], confidence threshold conf_thres = 0.5, Iou threshold iou_thres = 0.45, initial learning rate Ir = 0.0032, learning rate momentum momentum = 0.843, weight attenuation coefficient weight decay =0.00036. Through multiple experiments, it is proved that the best effect is achieved when training for about 200 epochs.

[0129] We modified the number and size of anchors using fuzzy ISODATA dynamic clustering. After multiple experiments, we selected the cluster with the highest average Iou value (Iou = 0.81590). We then settled on 12 anchors, divided into four groups. The sizes of the anchors in each group were: [18, 20, 29, 32, 42, 44], [55, 55, 60, 74, 72, 68], [86, 91, 111, 110, 135, 141], and [179, 190, 253, 262, 395, 408]. Finally, we modified the relevant parameters in the train.py file, not using the pretrained model, and setting epochs to 200 and batch_size to 10.

[0130] Step 4: Evaluate the trained IDT-YOLOv5-CBAM network model from the three aspects of accuracy, recall rate, and Map value.

[0131] The evaluation metrics shown in Tables 1, 2, and 3 are significantly improved compared to the original model. If one improves while the other decreases, consider whether the improvements are worthwhile. Also, identify the cause of the decrease and adjust the parameters before retraining. Continue training until the performance meets the actual testing requirements.

[0132] The indicators after model training are as follows:

[0133] Table 1 Comparison of indicator results between the target improvement stage and the previous stage

[0134]

[0135] Table 2 Comparison of indicator results between the target improvement stage and the original YOLOv5 model

[0136]

[0137] Table 3 Comparison of indicator results between the target improvement stage and YOLOv5+Aug

[0138]

[0139] Among them, YOLOv5 represents the unimproved original model; +Aug represents data augmentation; +Four represents the embedding of the detection layer; +ISO represents the introduction of the fuzzy ISODATA dynamic clustering algorithm; +Mul represents the fusion of multi-scale features; +Cbam represents the introduction of the CBAM attention mechanism module; +F represents the introduction of dilated convolution to replace upsampling; +Add represents the introduction of a parallel strategy to replace Concat fusion.

[0140] Step 5: Use the trained network model weight file best.pt for recognition: Prepare the image to be detected on the computer, change the configuration file yaml, the trained weights, and the image path to be detected in detect.py, and load the network model configuration file yaml and the network model weights by executing detect.py. Finally, the computer saves the detection results in the form of images.

[0141] In order to test the detection effect of the improved IDT-YOLOv5-CBAM model on dense small targets, the present invention selects apples on a single fruit tree with gradually increasing density of targets to be detected and apples clustered in a large area on multiple fruit trees as test objects by comparing it with the original YOLOv5 model. Figures 4 to 7 As shown in Figure 2, the original YOLOv5 model and the improved IDT-YOLOv5-CBAM model were used to test apples on a single fruit tree. The parameters used in the two models were consistent. The test results show that the original YOLOv5 model has a relatively weak detection effect on small targets and a high false detection rate, as shown in Figure 2. Figure 4 Compared with the original YOLOv5 model, the improved IDT-YOLOv5-CBAM model has greatly improved its performance. In addition to being able to detect more targets, Figure 4 The false detection in the Figure 5 As shown in Figure 2, the original YOLOv5 model and the improved IDT-YOLOv5-CBAM model were used to test apples clustered in large areas on multiple fruit trees. Figure 6 、 Figure 7 As shown in the figure, despite the high density of targets in this scenario, the improved IDT-YOLOv5-CBAM model performs quite well. The test results show that the improved IDT-YOLOv5-CBAM model greatly improves the detection capability of densely packed small targets, avoiding missed detections and false detections.

Claims

1. A dense small target detection method based on IDT-YOLOv5-CBAM hybrid algorithm, characterized in that: The following steps are involved: (1) Preprocess the pre-acquired small target images to be detected and create a data set: (2) Improve the original YOLOv5 network structure to obtain the improved IDT-YOLOv5-CBAM network structure; (3) Adjust the parameters of the improved network model: In the IDT-YOLOv5-CBAM network model, set the size of the convolutional neural network input image, the number of recognition types, and the number of iterations according to the recognition effect and training speed required by the user; (4) The improved model is evaluated from the three aspects of accuracy, recall rate and Map value on the trained IDT-YOLOv5-CBAM network model; The step (2) comprises the following steps: (21) Add a detection layer based on the original three detection layers of YOLOv5; (22) On the basis of embedding the fourth detection layer, the 22nd layer is fused with the 2nd layer, the 27th layer is fused with the 4th layer, and the 32nd layer is fused with the 6th layer by Add multi-scale. (23) The CBAM attention mechanism module is introduced into the Neck network in YOLOv5 to achieve full consideration of spatial and channel dimensions; the CBAM attention mechanism is introduced in the 11th, 16th and 21st layers respectively; (24) The original upsampling in the Neck layer of the original model is replaced with deconvolution, and the deconvolution module is used to enhance the resolution of the feature map; (25) Considering the characteristics of low-level features with low semantics and high details and high-level features with high semantics and low details, the parallel strategy add is adopted to replace the Concat fusion operation in the Neck layer; (26) The k-means algorithm is optimized by using the fuzzy ISODATA dynamic clustering method; by adding a fuzzy matrix and adding two steps, merging and splitting, in the iterative process, the size of the K value is dynamically determined to avoid the occurrence of local optimal solutions; The added detection layers, specifically corresponding to layers 19 to 24 of the overall structure, first undergo parameter learning through the CSP2_1 module, CBS module, and CBAM module. Next, a deconvolution operation is performed on the 22nd layer, and the output result is fused with the Add operation of the second layer. Finally, the CSP2_1 module of the 24th layer outputs the Add fusion result, achieving a feature map that is 4 times downsampled from the original input. The 80×80×64 feature map output of the 21st layer is deconvolved to convert it into high-level feature information of 160×160×64, which is then added and fused with the 160×160×64 low-level feature information of the 2nd layer. The 80×80×128 feature map of the 27th layer, which passes through the CSP2_1 module, is added and fused with the low-level feature information of the 4th layer of the same size. The 40×40×256 feature map of the 32nd layer, which passes through the CSP2_1 module, is added and fused with the low-level feature information of the 6th layer of the same size.

2. The dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm according to claim 1 is characterized in that: The step (1) comprises the following steps: (11) Image acquisition: The user uses a mobile phone, digital camera or other image acquisition device to capture images of the small target to be detected; (12) Image annotation: annotate the collected images and mark the category and location information of small targets; (13) Image augmentation: Use the Augmentor image data augmentation library to augment the image; by adding noise, horizontal mirroring, vertical mirroring, cropping, rotating, translating, and adjusting brightness, the sample is augmented to meet the recognition requirements; (14) Dataset division: The amplified images are divided into training set and validation set in a ratio of 8:

2.

3. The dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm according to claim 1 is characterized in that: The implementation process of step (3) is as follows: Image size imgsz = [640, 640], confidence threshold conf_thres = 0.5, Iou threshold iou_thres = 0.45, initial learning rate Ir = 0.0032, learning rate momentum momentum = 0.843, weight decay coefficient weight_decay = 0.00036, set epoch = 200.

4. The dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm according to claim 1 is characterized in that: The implementation process of step (25) is as follows: Assume that the two input channels are A1, A2...A c and B1, B2…B c , the formula is as follows Where, c represents the number of channels; A i 、B i Respectively represent the two input channels; K i Represents the convolution kernel of the corresponding channel.

5. The dense small target detection method based on the IDT-YOLOv5-CBAM hybrid algorithm according to claim 1 is characterized in that: The step (26) comprises the following steps: S1: Set the expected number of anchors K0 = 12, randomly select cluster centers, and initialize 12 anchors; S2: Calculate the membership matrix U: Among them, K means there are K cluster centers; μ ij is an element in the membership matrix U; d ij represents the distance from the jth sample to the i-th cluster center; d pj represents the distance from the jth sample to the pth cluster center; m represents the degree of inter-class fuzziness; S3: Calculate cluster centers: Among them, Z i represents the i-th cluster center; N represents the total number of samples; L represents the number of iterations; S4: Determine whether to split. If one of the following conditions is met, split is performed: ① The final number of clusters is less than 1 / 2 of the expected value or the final number of clusters is greater than 2 times the expected value; ② The number of iterative operations is an even number; ③ The maximum value of the standard deviation vector of the sample distances in the same category is greater than the threshold of the sample standard deviation, that is, σ jmax >θ s , where θ s Is the threshold of the sample standard deviation, split into new cluster centers Z1 and Z2: Z1=Z i +fσ jmax Z1=Z i -fσ jmax (14) Among them, σ jmax Represents the maximum value of the sample distance standard deviation vector; f represents the splitting coefficient, which satisfies 0 <f<1; S5: Determine whether to merge. If one of the following conditions is met, merge: ① Calculate the distance D between each class. ij , if it is less than the threshold value of the distance between the centers of various samples, that is, D ij <θ k ,θ k is the threshold value of the distance between the centers of each type of samples; ② If the number of samples in a certain class is less than the specified number threshold θ n , the new cluster center is Z k : Z k =(N i Z i +N j Z j ) / (N i +N j ) (15) Among them, Z i 、Z j Represents the sample center of class i and class j respectively; N i 、N j Represents the number of samples of class i and class j respectively; S6: Calculate the new clustering results and calculate the distance d from each sample to each cluster center ij ; S7: Calculate the new membership matrix according to formula (12); S8: Return to step S3 and repeat the iteration until the iteration is completed.

Images