Gated spatial-temporal feature distillation and fusion based yolo v8 cherry fruit detection method

Abstract

The application discloses a cherry fruit detection method based on a gated space-frequency feature distillation and fusion YOLOv8, proposes a multi-path gated interaction and dynamic deformation convolution module, and replaces a C2f module of a YOLOv8 model with the module. First, the module adopts a three-branch network structure, and focuses on extraction of local fine-grained features, spatial correlation features and global context features respectively. Second, an efficient feature extraction module is introduced, local details and important fruit feature channels are strengthened through spatial separable convolution and point-by-point convolution. Then, a gated frequency feature distillation block is adopted to dynamically screen key global features and suppress redundant noise, so that the robustness and precision of fruit target detection are improved through structured lightweight modification and invalid feature filtering. Experimental results show that the cherry fruit detection mAP50 of the application is 83.4%, the false detection rate is less than 3.7%, the model parameter quantity is 2.37M, and the detection speed is 2.6f / s, which can meet the accurate and efficient detection demand of cherry fruits in a complex orchard scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of smart agriculture and computer vision, and in particular to a YOLOv8 cherry fruit detection method capable of effectively suppressing complex background and dynamic light interference, with high accuracy, good robustness, strong scale adaptability and low computational complexity. BACKGROUND

[0002] Cherry is one of the high-value specialty crops in China, with a planting area of over 185,000 hectares and an annual output of nearly 850,000 tons. In particular, major producing areas such as Liaoning, Shandong, and Shaanxi have formed "one village, one product" specialty economies relying on cherry industries, becoming an important pillar of China's rural revitalization. However, cherry fruits have short ripening periods, high pressure for concentrated picking, and high requirements for fresh commodity rates. Traditional agricultural methods of manual planting and management face issues such as low efficiency, high cost, and insufficient precision. The low fruit recognition efficiency results in picking costs accounting for more than 40% of total production costs, making it difficult to meet the needs of modern agriculture for precision and intelligent development. Therefore, there is an urgent need to develop efficient and accurate cherry fruit detection technology.

[0003] Cherry fruit detection aims to use digital image processing, computer vision, and artificial intelligence technologies to automatically identify, locate, and count cherry fruits in natural growing environments, overcoming issues such as dense fruit distribution, light variation, complex backgrounds, and fruit occlusion, thereby achieving efficient and accurate detection of cherry fruits. It can provide key technical support for real-time monitoring of cherry growth conditions, fruit management, yield prediction, automated picking, and quality control for precision planting needs.

[0004] According to different overall technical frameworks, existing cherry fruit detection methods mainly include three categories: traditional image processing-based cherry fruit detection methods, traditional machine learning-based cherry fruit detection methods, and deep learning-based cherry fruit detection methods.

[0005] (1) Traditional image processing-based cherry fruit detection methods

[0006] This type of method mainly relies on classic digital image processing and pattern recognition techniques to extract cherry fruit features such as color, texture, and shape, thereby achieving fruit target detection.

[0007] Yang et al. used thresholding operation in HSV color space to separate cherry targets from background, but its accuracy would be significantly reduced in the case of light changes or fruit occlusion; Wang et al. introduced SLIC superpixel segmentation algorithm to enhance the segmentation performance of cherry fruit boundary, which improved the detection rate of adherent fruits, but at the expense of real-time performance of the algorithm; Liu et al. combined Canny operator with morphological operation to extract the edge and contour of cherry fruit, but it could not effectively suppress the interference of overlapping fruits or complex background; Gupta et al. used morphological watershed algorithm to segment fruits, which could better handle the overlapping of fruits, but its accuracy in complex background still needs to be further improved.

[0008] Overall, although this kind of method performs well in some simple scenes, it has been gradually replaced by methods based on machine learning and deep learning due to its poor adaptability to environmental changes.

[0009] (2) Cherry fruit detection method based on traditional machine learning

[0010] This kind of method aims to automatically learn features from data and perform classification or regression by building models such as support vector machines and random forests, emerging a detection paradigm that combines feature engineering and classifiers.

[0011] Bai et al. combined color and texture features of cherry fruits with support vector machines to achieve fruit detection in simple backgrounds; Reis et al. fused color histogram, LBP texture and HOG shape features, and then used random forest classifier for cherry fruit detection, but its cross-species generalization ability was obviously insufficient. In addition, Zhang et al. built an AdaBoost cascade classifier based on Haar-like features, and Tian et al. combined SURF key point features with support vector machine classifier, which also achieved certain effect.

[0012] Overall, compared with traditional image processing methods, this kind of method improves the accuracy and robustness of cherry fruit detection to some extent. However, due to the dependence on artificial feature design and the lack of adaptive ability to complex scenes, this kind of method still lacks effective detection means for cherry fruits in scenes with light changes, fruit occlusion / overlap, etc., and the detection performance is often poor.

[0013] (3) Cherry fruit detection method based on deep learning

[0014] This kind of method aims to extract spatial (such as color, edge, contour) and contextual features of images using deep convolutional neural networks (such as ResNet, VGG, etc.), Transformer networks and other models, so as to accurately locate cherry fruits.

[0015] Gai et al. added a small target detection layer to the backbone network of YOLOv7 and used the Soft-NMS method to remove redundant detection boxes, thus proposing a night cherry detection method based on improved YOLOv7; Miao et al. replaced the backbone network of the YOLOv7 model with MobileNetV3 to reduce the number of network parameters, and added a global attention module in the feature fusion subnetwork, which is suitable for detecting cherries at different maturity stages. In order to achieve more efficient target feature extraction, Ji et al. introduced YOLOv8n as the backbone network for cherry fruit detection, replaced the bottleneck block of C2f in YOLOv8n with a Fasternet module based on partial convolution, and added a CBAM attention module between the SPPF module and the C2f module, thus proposing a cherry classification and detection method based on improved YOLOv8n.

[0016] The above methods have achieved good detection accuracy on specific data sets and to some extent have focused on small targets and night lighting conditions. However, in actual applications, the image data set obtained in the orchard usually has more complex dynamic scene changes, such as leaf shading, fruit shading and overlapping, diverse lighting and weather conditions (such as rain, fog, wind), etc. At the same time, the size of cherry fruits may vary due to different shooting distances, fruit growth stages, and image resolutions, and existing technologies ignore the adaptability of network models to cherry fruit detection under multiple scales. For this reason, Song et al. proposed a sweet cherry detection method based on YOLOv8-DCPF, which uses a dynamic weight adjustment module to replace the C2f module of the YOLOv8 network and introduces a convolution block attention module before the target detection head to improve the network's ability to represent multi-scale features, but the method still has room for improvement in small target recognition accuracy; Wu et al. introduced a context-integrated semantic segmentation module based on the YOLOv8n-seg framework to enhance the feature extraction capability of small targets and overlapping fruits, and used cross-entropy loss, Dice loss, and Hausdorff distance loss to improve the boundary feature extraction efficiency, thus proposing a real-time cherry overlapping and defect segmentation method based on improved YOLOv8n-seg. However, the deployment of the above methods in edge computing environments or embedded devices still faces great challenges, and the balance between the lightweight and performance of the network model still needs to be further optimized. Therefore, existing deep learning-based cherry fruit detection technologies still cannot fully meet the needs of practical applications.

[0017] In summary, it is still a highly challenging task to achieve accurate automatic cherry fruit detection. Currently, there is no cherry fruit detection method that can effectively resist complex background and dynamic lighting interference, has high accuracy, good robustness, strong scale adaptability, and low computational complexity. SUMMARY

[0018] The present application is to solve the technical problems existing in the prior art, and provides a YOLOv8 cherry fruit detection method capable of effectively inhibiting complex background and dynamic light interference, high accuracy, good robustness, strong scale adaptation ability and low computational complexity.

[0019] The technical solution of the present application is: a YOLOv8 cherry fruit detection method based on gated spatial-temporal feature distillation and fusion, which is performed according to the following steps:

[0020] Step 1. Input a multi-scale cherry fruit image training set , an artificially labeled label set , and a cherry target detection image I to be processed;

[0021] Step 2. Establish and initialize a cherry fruit detection network , wherein the cherry fruit detection network comprises: 1 feature extraction subnetwork based on gated spatial-temporal feature distillation , 1 multi-dimensional feature interweaving fusion subnetwork , and 1 target detection head subnetwork ;

[0022] Step 3. Train the cherry fruit detection network using the image training set and the artificially labeled label set , to obtain a trained deep convolutional neural network ;

[0023] Step 4. Use the trained deep convolutional neural network to detect cherry fruit targets in the image I, and output the detection results.

[0024] Preferably, the step 2 is specifically as follows:

[0025] Step 2.1: Establish and initialize a feature extraction subnetwork based on gated spatial-temporal feature distillation , wherein the feature extraction subnetwork based on gated spatial-temporal feature distillation is sequentially provided with a convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , and a convolution module , multi-path gating interaction and dynamic deformation convolution module and fast spatial pyramid pooling SPPF module ;

[0026] Step 2.2: Establish and initialize a multi-dimensional feature interweaving and fusion subnetwork , the multi-dimensional feature interweaving and fusion subnetwork is composed of a multi-path gating interaction and dynamic deformation convolution module and a convolution module ;

[0027] Step 2.3: Establish and initialize a target detection head subnetwork , the target detection head subnetwork is composed of three YOLOv8 detection head modules ;

[0028] Step 2.4: Establish a feature extraction subnetwork based on gated spatial-frequency feature distillation , the multi-dimensional feature interweaving and fusion subnetwork and the target detection head subnetwork , the output of the fast spatial pyramid pooling SPPF module is connected to the output of the multi-path gating interaction and dynamic deformation convolution module after upsampling operation, and then the connected feature map is taken as the input of the multi-path gating interaction and dynamic deformation convolution module ; the output of the multi-path gating interaction and dynamic deformation convolution module is connected to the output of the multi-path gating interaction and dynamic deformation convolution module after upsampling operation, and then the connected feature map is taken as the input of the multi-path gating interaction and dynamic deformation convolution module ; the output of the multi-path gating interaction and dynamic deformation convolution module is taken as the input of the detection head module ;

[0029] The output of the multi-path gating interaction and dynamic deformation convolution module is connected to the output of the multi-path gating interaction and dynamic deformation convolution module after processing by the convolution module , and then the connected feature map is taken as the input of the multi-path gating interaction and dynamic deformation convolution module ; the output of the multi-path gating interaction and dynamic deformation convolution module is taken as the input of the detection head module ;

[0030] the output of the multi-path gating interaction and dynamic deformation convolution module is connected with the output of the fast spatial pyramid pooling (SPPF) module after processing by the convolution module , and then the connected feature map is taken as the input of the multi-path gating interaction and dynamic deformation convolution module ; the output of the multi-path gating interaction and dynamic deformation convolution module is taken as the input of the detection head module .

[0031] Preferably, the multi-path gating interaction and dynamic deformation convolution module each is provided with a bottleneck block composed of a plurality of 1×1 convolution kernels, a cross-channel feature aggregation block composed of a plurality of 1×1 convolution kernels, a projection block composed of a plurality of 1×1 convolution kernels, an efficient feature extraction module, a Split operation layer, a connection operation layer, and a gated frequency feature distillation block; the feature map output by the bottleneck block is taken as the input of the cross-channel feature aggregation block, the efficient feature extraction module, and the Split operation layer respectively, and the output of the Split operation layer is further processed by the gated frequency feature distillation block; the output of the cross-channel feature aggregation block, the efficient feature extraction module, the Split operation layer, and the gated frequency feature distillation block is connected and then processed by the projection block.

[0032] Preferably, the efficient feature extraction module is sequentially provided with a bottleneck block composed of a plurality of 1×1 convolution kernels, a spatial separable convolution block, and a point-wise convolution block composed of a plurality of 1×1 convolution kernels.

[0033] Preferably, the gated frequency feature distillation block is sequentially provided with a convolution block composed of a plurality of 3×3 convolution kernels, a bottleneck block composed of a plurality of 1×1 convolution kernels, a convolution block composed of a plurality of 3×3 convolution kernels, a GELU activation function layer, a gating operation layer, a projection block composed of a plurality of 1×1 convolution kernels, and a residual connection operation layer.

[0034] Preferably, the convolution module is sequentially provided with a plurality of convolution kernels with a size of 3×3 and a stride of 2, a BatchNorm2d batch normalization layer, and a SiLU activation function layer.

[0035] Preferably, step 3 is specifically as follows:

[0036] Step 3.1: taking each image and its corresponding label from the image training set ​ ;

[0037] Step 3.2 utilizes a feature extraction subnetwork based on gated spatial-temporal feature distillation and a multi-dimensional feature interleaving fusion subnetwork to obtain a multi-path gated interaction and dynamic deformation convolution module to output a small-scale feature map , the multi-path gated interaction and dynamic deformation convolution module to output a medium-scale feature map , and the multi-path gated interaction and dynamic deformation convolution module to output a large-scale feature map ;

[0038] Step 3.3 utilizes a target detection head subnetwork to calculate the fruit target detection results of the image ;

[0039] Step 3.3.1 utilizes a detection head module to process the large-scale feature map to obtain the detection results of larger targets , and adopts a binary cross-entropy loss function to evaluate the classification loss value and the confidence loss value, adopts a CIoU loss function to evaluate the positioning loss value, thereby obtaining the total loss value of the large-scale feature map ;

[0040] Step 3.3.2 utilizes a detection head module to process the medium-scale feature map to obtain the detection results of medium targets , and adopts a binary cross-entropy loss function to evaluate the classification loss value and the confidence loss value, adopts a CIoU loss function to evaluate the positioning loss value, thereby obtaining the total loss value of the medium-scale feature map ;

[0041] Step 3.3.3 utilizes a detection head module to process the small-scale feature map to obtain the detection results of smaller targets , and adopts a binary cross-entropy loss function to evaluate the classification loss value and the confidence loss value, adopts a CIoU loss function to evaluate the positioning loss value, thereby obtaining the total loss value of the small-scale feature map ;

[0042] Step 3.3.4: adopts a non-maximum suppression algorithm to process and to filter out redundant boxes with high overlap, thereby obtaining the final detection results and the total loss value thereof ;

[0043] Step 3.4 updating parameters of the deep convolutional neural network according to the total loss value by using a back-propagation algorithm based on a stochastic gradient descent method to obtain a trained deep convolutional neural network , wherein the number of iterations of the back-propagation algorithm is , and the number of iterations is preset.

[0044] Preferably, the step 4 is specifically as follows:

[0045] Step 4.1: processing the cherry target detection image I to be processed by using the trained deep convolutional neural network to obtain a small-scale feature map , a medium-scale feature map and a large-scale feature map , respectively.

[0046] Step 4.2: processing the large-scale feature map by using the trained detection head module to obtain a detection result of a large target , and evaluating a classification loss value and a confidence loss value by using a binary cross-entropy loss function and evaluating a positioning loss value by using a CIoU loss function, thereby obtaining a total loss value of .

[0047] Step 4.3: processing the medium-scale feature map by using the trained detection head module to obtain a detection result of a medium target , and evaluating a classification loss value and a confidence loss value by using a binary cross-entropy loss function and evaluating a positioning loss value by using a CIoU loss function, thereby obtaining a total loss value of .

[0048] Step 4.4: processing the small-scale feature map by using the trained detection head module to obtain a detection result of a small target , and evaluating a classification loss value and a confidence loss value by using a binary cross-entropy loss function and evaluating a positioning loss value by using a CIoU loss function, thereby obtaining a total loss value of .

[0049] Step 4.5: processing and by using a non-maximum suppression algorithm to filter out redundant boxes with high overlap, thereby obtaining a final detection result.

[0050] ​Step 4.6 outputs the final detection result.

[0051] Compared with the prior art, the present application has the following advantages:

[0052] First, the image data set obtained by the orchard usually has dynamic scene changes such as tree leaf occlusion, fruit occlusion and overlap, various light and weather conditions (such as rain, fog, wind), etc., which causes the prior art to learn a large number of redundant or invalid features, such as background noise or repeated features, due to the inability to effectively suppress interference information, thereby affecting the detection ability of the network, and even causing missed detection or false detection of cherry fruits. To solve this problem, the present application designs a multi-path gated interaction and dynamic deformation convolution module with a three-branch structure, which realizes multi-dimensional feature extraction through the cooperation of multiple components by using a cross-channel feature aggregation block, an efficient feature extraction module and a gated frequency feature distillation block. The efficient feature extraction module focuses on local fine-grained features and strengthens local details and fruit dominant channels through spatial separable convolution and point-by-point convolution. The cross-channel feature aggregation block fuses multi-channel spatial position information and establishes spatial correlation between different feature maps by combining the Split operation. The gated frequency feature distillation block dynamically allocates feature weights through multi-layer superimposed convolution and GELU activation function layer, filters out key features and suppresses redundant noise, and then integrates local, spatial and global features by the projection block to separate the fruit features with rich information and the redundant background features, and obtain a compact representation of cherry fruits in multiple dimensions. On this basis, the present application realizes the interweaving of multi-dimensional compact representations and the strengthening of key features through the cooperative work of the feature extraction sub-network based on gated space-frequency feature distillation and the multi-dimensional feature interweaving and fusion sub-network, which helps to improve the performance of cherry fruit target detection under occlusion and overlap.

[0053] Second, due to the differences in growth stage (young fruit period-mature period), shooting distance and image resolution, the size variation range of cherry fruits is large, which makes the prior art prone to miss small targets or misposition large targets. Therefore, on the one hand, the multi-path gated interaction and dynamic deformation convolution module with a three-branch structure can represent multi-scale features of the whole life cycle of fruits: the efficient feature extraction module extracts the edge profile of small fruits through shallow convolution to avoid information loss caused by a large receptive field; the cross-channel feature aggregation block integrates middle-level features by using spatial correlation to adapt to the context information of medium-sized fruits; the gated frequency feature distillation block and the projection block capture important features of large fruits through local-global feature. On the other hand, the strategy of efficient feature extraction module and multi-branch cooperation can preserve the step-by-step feature extraction ability of spatial dimension and optimize the fusion efficiency of multi-scale features through channel shuffling to ensure that the features of fruits of different sizes are accurately expressed, thereby enhancing the multi-scale expression ability of the network model for cherry fruit targets.

[0054] Third, the orchard scene is dynamic and variable, for example, leaf swaying, fruit overlapping, rain and fog blocking, and light mutation, which often leads to insufficient detection robustness of the prior art in complex orchard scenes, and the core problem lies in the weak filtering ability of the feature extraction process of the network model to interference information. The present application uses a gating frequency feature distillation strategy, first performs preliminary transformation of the features through a 3x3 convolution block and a 1x1 convolution block, then further filters through a GELU activation function and a gating operation, and then dynamically allocates weights according to the feature effectiveness, dynamically amplifies the features strongly related to the fruit and suppresses the low correlation features, thereby reconstructing a feature representation with stronger representation ability, which helps to enhance the attention of the network model to the important features of the fruit and improve the robustness of the fruit target detection under various light and weather conditions.

[0055] Fourth, when the existing model is deployed on an orchard edge device, the real-time performance is often insufficient due to large parameter quantity and high computational complexity. Therefore, the present application uses spatial separable convolution and point-wise convolution in the feature extraction sub-network based on the gating space-frequency feature distillation and the multi-dimensional feature interweaving fusion sub-network, and the combination of the two reduces the computational load of the corresponding network branch by about 30% compared with the C2f module of the traditional YOLOv8. At the same time, in the second 3x3 convolution block of the gating frequency feature distillation block, only 25% of the feature channels are executed for deep convolution to extract detailed features, and a feature effectiveness score matrix is constructed in combination with a gating linear unit to filter 95% of the invalid features through a residual connection. This design scheme can focus on the key channel features of cherry fruits while ensuring accuracy, greatly reducing invalid calculations, and thus reducing the computational cost of the network model.

[0056] In summary, the present application can effectively suppress complex background and dynamic light interference, and has the characteristics of high accuracy, good robustness, strong scale adaptability, and low computational complexity. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a flowchart of an embodiment of the present application.

[0058] Figure 2 is a general structure diagram of a cherry fruit detection network of an embodiment of the present application.

[0059] Figure 3 is a network structure diagram of a multi-path gating interaction and dynamic deformation convolution module in an embodiment of the present application.

[0060] Figure 4 is a network structure diagram of an efficient feature extraction module in an embodiment of the present application.

[0061] Figure 5 is a network structure diagram of a gating frequency feature distillation block in an embodiment of the present application.

[0062] Figure 6 is the confusion matrix obtained by the embodiment of the present application on the test data set.

[0063] Figure 7 is a detection result chart of cherry fruits by the embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be described clearly and completely below with reference to the drawings in the embodiment of the present application.

[0065] The present application provides a YOLOv8 cherry fruit detection method with gated spatial-temporal feature distillation and fusion, as shown in Figure 1 , the following steps are performed;

[0066] Step 1: input a multi-scale cherry fruit image training set , a manually labeled label set , and a cherry target detection image I to be processed;

[0067] Step 2: establish and initialize a cherry fruit detection network , the cherry fruit detection network comprises: 1 gated spatial-temporal feature distillation-based feature extraction subnetwork , 1 multi-dimensional feature interweaving fusion subnetwork , and 1 target detection head subnetwork ;

[0068] Specifically, the cherry fruit detection network is established and initialized The specific steps are as follows:

[0069] Step 2.1: establish and initialize a gated spatial-temporal feature distillation-based feature extraction subnetwork , as shown in Figure 2 , the gated spatial-temporal feature distillation-based feature extraction subnetwork is sequentially provided with a convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , a convolution module , a multi-path gated interaction and dynamic deformation convolution module , and a fast spatial pyramid pooling SPPF module ;

[0070] The convolution module The system consists of a group of multiple convolutional kernels of size 3×3 with a stride of 2, a BatchNorm2d batch normalization layer, and a SiLU activation function layer. Specifically, in this embodiment, the... It contains 64 convolutional kernels with a size of 3×3 and a stride of 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer; It contains 128 convolutional kernels of size 3×3 with stride 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer; It contains 256 convolutional kernels of size 3×3 with a stride of 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer; It contains 512 convolutional kernels of size 3×3 with stride 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer; It contains 1024 convolutional kernels with a size of 3×3 and a stride of 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer;

[0071] like Figure 3 As shown, the multi-path gated interaction and dynamic deformation convolution module Each module includes a bottleneck block composed of multiple 1×1 convolutional kernels, a cross-channel feature aggregation block composed of multiple 1×1 convolutional kernels, a projection block composed of multiple 1×1 convolutional kernels, an efficient feature extraction module, a split operation layer, a connection operation layer, and a gated frequency feature distillation block. The feature maps output from the bottleneck block serve as inputs to the cross-channel feature aggregation block, the efficient feature extraction module, and the split operation, respectively. The output of the split operation is further processed by the gated frequency feature distillation block. The outputs of the cross-channel feature aggregation block, the efficient feature extraction module, the split operation, and the gated frequency feature distillation block are connected and then processed by the projection block. Specifically, in this embodiment, the... It includes one bottleneck block consisting of 64 1×1 convolutional kernels, one cross-channel feature aggregation block consisting of 128 1×1 convolutional kernels, one projection block consisting of 128 1×1 convolutional kernels, one split operation layer, and one efficient feature extraction module. 1-layer connection operation, 1 gated frequency characteristic distillation block ; comprises 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 group of cross-channel feature aggregation blocks composed of 256 1x1 convolution kernels, 1 group of projection blocks composed of 256 1x1 convolution kernels, 1 layer of Split operation, 1 efficient feature extraction module , 1 layer of connection operation, 1 gated frequency feature distillation block ; comprises 1 group of bottleneck blocks composed of 256 1x1 convolution kernels, 1 group of cross-channel feature aggregation blocks composed of 512 1x1 convolution kernels, 1 group of projection blocks composed of 512 1x1 convolution kernels, 1 layer of Split operation, 1 efficient feature extraction module , 1 layer of connection operation, 1 gated frequency feature distillation block ; comprises 1 group of bottleneck blocks composed of 512 1x1 convolution kernels, 1 group of cross-channel feature aggregation blocks composed of 1024 1x1 convolution kernels, 1 group of projection blocks composed of 1024 1x1 convolution kernels, 1 layer of Split operation, 1 efficient feature extraction module , 1 layer of connection operation, 1 gated frequency feature distillation block ;

[0072] As shown in Figure 4 , the efficient feature extraction module is sequentially provided with 1 group of bottleneck blocks composed of a plurality of 1x1 convolution kernels, 1 spatial separable convolution block, and 1 group of pointwise convolution blocks composed of a plurality of 1x1 convolution kernels; specifically, in the embodiment, the comprises 1 group of bottleneck blocks composed of 64 1x1 convolution kernels, 1 spatial separable convolution block, and 1 group of pointwise convolution blocks composed of 128 1x1 convolution kernels, which are sequentially connected; comprises 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 spatial separable convolution block, and 1 group of pointwise convolution blocks composed of 256 1x1 convolution kernels, which are sequentially connected; comprises 1 group of bottleneck blocks composed of 256 1x1 convolution kernels, 1 spatial separable convolution block, and 1 group of pointwise convolution blocks composed of 512 1x1 convolution kernels, which are sequentially connected; comprises 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 spatial separable convolution block, and 1 group of pointwise convolution blocks composed of 256 1x1 convolution kernels, which are sequentially connected;

[0073] As shown in Figure 5As shown, the gated frequency feature distillation block sequentially includes a convolution block composed of multiple 3×3 convolution kernels, a bottleneck block composed of multiple 1×1 convolution kernels, a convolution block composed of multiple 3×3 convolution kernels, a GELU activation function layer, a gated operation layer, a projection block composed of multiple 1×1 convolution kernels, and a residual connection operation layer; specifically, in this embodiment, the... It includes a convolutional block consisting of 64 3×3 convolutional kernels, a bottleneck block consisting of 128 1×1 convolutional kernels, a convolutional block consisting of 256 3×3 convolutional kernels, a GELU activation function layer, a gating operation, a projection block consisting of 512 1×1 convolutional kernels, and a residual connection operation. It includes a convolutional block consisting of 64 3×3 convolutional kernels, a bottleneck block consisting of 128 1×1 convolutional kernels, a convolutional block consisting of 256 3×3 convolutional kernels, a GELU activation function layer, a gating operation, a projection block consisting of 512 1×1 convolutional kernels, and a residual connection operation. It includes a convolutional block consisting of 64 3×3 convolutional kernels, a bottleneck block consisting of 128 1×1 convolutional kernels, a convolutional block consisting of 256 3×3 convolutional kernels, a GELU activation function layer, a gating operation, a projection block consisting of 512 1×1 convolutional kernels, and a residual connection operation. It includes a convolutional block consisting of 64 3×3 convolutional kernels, a bottleneck block consisting of 128 1×1 convolutional kernels, a convolutional block consisting of 256 3×3 convolutional kernels, a GELU activation function layer, a gating operation, a projection block consisting of 512 1×1 convolutional kernels, and a residual connection operation.

[0074] Step 2.2: Establish and initialize a multi-dimensional feature interweaving and fusion subnetwork ,like Figure 2 As shown, the multi-dimensional feature interweaving and fusion subnetwork Composed of multi-path gated interaction and dynamic deformation convolution module and convolutional modules constitute;

[0075] like Figure 3 As shown, the multi-path gated interaction and dynamic deformation convolution module Each module includes a bottleneck block composed of multiple 1×1 convolutional kernels, a cross-channel feature aggregation block composed of multiple 1×1 convolutional kernels, a projection block composed of multiple 1×1 convolutional kernels, an efficient feature extraction module, a split operation layer, a connection operation layer, and a gated frequency feature distillation block. The feature maps output from the bottleneck block serve as inputs to the cross-channel feature aggregation block, the efficient feature extraction module, and the split operation, respectively. The output of the split operation is further processed by the gated frequency feature distillation block. The outputs of the cross-channel feature aggregation block, the efficient feature extraction module, the split operation, and the gated frequency feature distillation block are connected and then processed by the projection block. Specifically, in this embodiment, the... It includes one bottleneck block consisting of 256 1×1 convolutional kernels, one cross-channel feature aggregation block consisting of 512 1×1 convolutional kernels, one projection block consisting of 512 1×1 convolutional kernels, one split operation layer, and one efficient feature extraction module. 1-layer connection operation, 1 gated frequency characteristic distillation block ; It includes one bottleneck block consisting of 128 1×1 convolutional kernels, one cross-channel feature aggregation block consisting of 256 1×1 convolutional kernels, one projection block consisting of 256 1×1 convolutional kernels, one split operation layer, and one efficient feature extraction module. 1-layer connection operation, 1 gated frequency characteristic distillation block ; It includes one bottleneck block consisting of 256 1×1 convolutional kernels, one cross-channel feature aggregation block consisting of 512 1×1 convolutional kernels, one projection block consisting of 512 1×1 convolutional kernels, one split operation layer, and one efficient feature extraction module. 1-layer connection operation, 1 gated frequency characteristic distillation block ; It includes one bottleneck block consisting of 512 1×1 convolutional kernels, one cross-channel feature aggregation block consisting of 1024 1×1 convolutional kernels, one projection block consisting of 1024 1×1 convolutional kernels, one split operation layer, and one efficient feature extraction module. 1-layer connection operation, 1 gated frequency characteristic distillation block ;

[0076] like Figure 4 As shown, the efficient feature extraction module sequentially includes a bottleneck block composed of multiple 1×1 convolutional kernels, a spatially separable convolutional block, and a pointwise convolutional block composed of multiple 1×1 convolutional kernels; specifically, in this embodiment, the... comprises 1 group of bottleneck blocks composed of 64 1x1 convolution kernels, 1 spatial separable convolution block, 1 group of pointwise convolution blocks composed of 128 1x1 convolution kernels connected in sequence; comprises 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 spatial separable convolution block, 1 group of pointwise convolution blocks composed of 256 1x1 convolution kernels connected in sequence; comprises 1 group of bottleneck blocks composed of 256 1x1 convolution kernels, 1 spatial separable convolution block, 1 group of pointwise convolution blocks composed of 512 1x1 convolution kernels connected in sequence; comprises 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 spatial separable convolution block, 1 group of pointwise convolution blocks composed of 256 1x1 convolution kernels connected in sequence;

[0077] As shown in Figure 5 , the gating frequency feature distillation block is sequentially provided with 1 group of convolution blocks composed of a plurality of 3x3 convolution kernels, 1 group of bottleneck blocks composed of a plurality of 1x1 convolution kernels, 1 group of convolution blocks composed of a plurality of 3x3 convolution kernels, 1 layer of GELU activation function layer, 1 layer of gating operation layer, 1 group of projection blocks composed of a plurality of 1x1 convolution kernels, and 1 layer of residual connection operation layer; specifically, in the embodiment, the comprises 1 group of convolution blocks composed of 64 3x3 convolution kernels, 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 group of convolution blocks composed of 256 3x3 convolution kernels, 1 layer of GELU activation function layer, 1 layer of gating operation, 1 group of projection blocks composed of 512 1x1 convolution kernels, and 1 layer of residual connection operation; comprises 1 group of convolution blocks composed of 64 3x3 convolution kernels, 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 group of convolution blocks composed of 256 3x3 convolution kernels, 1 layer of GELU activation function layer, 1 layer of gating operation, 1 group of projection blocks composed of 512 1x1 convolution kernels, and 1 layer of residual connection operation; comprises 1 group of convolution blocks composed of 64 3x3 convolution kernels, 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 group of convolution blocks composed of 256 3x3 convolution kernels, 1 layer of GELU activation function layer, 1 layer of gating operation, 1 group of projection blocks composed of 512 1x1 convolution kernels, and 1 layer of residual connection operation; comprises 1 group of convolution blocks composed of 64 3x3 convolution kernels, 1 group of bottleneck blocks composed of 128 1x1 convolution kernels, 1 group of convolution blocks composed of 256 3x3 convolution kernels, 1 layer of GELU activation function layer, 1 layer of gating operation, 1 group of projection blocks composed of 512 1x1 convolution kernels, and 1 layer of residual connection operation;

[0078] The convolution module The system consists of a group of multiple convolutional kernels of size 3×3 with a stride of 2, a BatchNorm2d batch normalization layer, and a SiLU activation function layer. Specifically, in this embodiment, the... It contains 256 convolutional kernels of size 3×3 with a stride of 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer; It contains 512 convolutional kernels of size 3×3 with stride 2, 1 BatchNorm2d batch normalization layer, and 1 SiLU activation function layer;

[0079] Step 2.3: Establish and initialize the target detection head sub-network ,like Figure 2 As shown, the target detection head sub-network Composed of 3 YOLOv8 detection head modules constitute;

[0080] Step 2.4: Establish a feature extraction subnetwork based on gated space-frequency feature distillation Multi-dimensional feature interweaving and fusion sub-network With target detection head network The interaction relationship between feature maps, such as Figure 2 As shown, the fast spatial pyramid pooling SPPF module The output, after upsampling, interacts with multipath gating and the dynamic deformation convolution module. The outputs are concatenated, and the concatenated feature maps are then used as multi-path gated interaction and dynamic deformation convolution modules. The input; the multi-path gated interaction and dynamic deformation convolution module The output, after upsampling, interacts with multipath gating and the dynamic deformation convolution module. The outputs are concatenated, and the concatenated feature maps are then used as multi-path gated interaction and dynamic deformation convolution modules. The input; the multi-path gated interaction and dynamic deformation convolution module The output is used as the detection head module. Input;

[0081] The multi-path gating interaction and dynamic deformation convolution module The output is passed through the convolution module. After processing, it interacts with multi-path gating and dynamic deformation convolution modules. The outputs are concatenated, and the concatenated feature maps are then used as multi-path gated interaction and dynamic deformation convolution modules. The input; then the multi-path gating interaction and dynamic deformation convolution module are combined. The output is used as the detection head module. Input;

[0082] The multi-path gating interaction and dynamic deformation convolution module The output is passed through the convolution module. After processing, it is used with the fast spatial pyramid pooling SPPF module. The outputs are concatenated, and the concatenated feature maps are then used as multi-path gated interaction and dynamic deformation convolution modules. The input of the multi-path gated interaction and dynamic deformation convolution module The output is used as the detection head module. Input.

[0083] Step 3. Using the image training set and manually labeled sets cherry fruit detection network Training is performed to obtain a trained deep convolutional neural network. ;

[0084] Specifically, the cherry fruit testing network The specific steps for training are as follows:

[0085] Step 3.1 From the image training set Extract each image and their corresponding tags ;

[0086] Step 3.2 Utilize a feature extraction subnetwork based on gated space-frequency feature distillation Interwoven and fused subnetworks with multi-dimensional features This yields a multi-path gated interaction and dynamic deformation convolution module. Output small-scale feature maps Multi-path gated interaction and dynamic deformation convolution module Output mesoscale feature map And multi-path gated interaction and dynamic deformation convolution module Output large-scale feature map ;

[0087] Specifically, feature map The specific calculation steps are as follows:

[0088] Step 3.2.1: Utilize a feature extraction subnetwork based on gated space-frequency feature distillation Convolutional module right The feature map is obtained through processing. ;

[0089] Step 3.2.2: process the feature map using the convolution module of ; ;

[0090] Step 3.2.3: process the feature map using the multi-path gated interaction and dynamic deformation convolution module of ; ;

[0091] Step 3.2.4: process the feature map using the convolution module of ; ;

[0092] Step 3.2.5: process the feature map using the multi-path gated interaction and dynamic deformation convolution module of ; ;

[0093] Step 3.2.6: process the feature map using the convolution module of ; ;

[0094] Step 3.2.7: process the feature map using the multi-path gated interaction and dynamic deformation convolution module of ; ;

[0095] Step 3.2.8: process the feature map using the convolution module of ; ;

[0096] Step 3.2.9: process the feature map using the multi-path gated interaction and dynamic deformation convolution module of ; ;

[0097] Step 3.2.10: process the feature map using the fast spatial pyramid pooling SPPF module of ; ; ​​​​​​​​​​​​​​​​​​

[0098] Step 3.2.11: performing an up-sampling operation on the feature map , and connecting the up-sampled feature map with the feature map , to obtain a feature map ;

[0099] Step 3.2.12: processing the feature map using the multi-dimension feature interweaving fusion subnetwork , to obtain a feature map ;

[0100] Step 3.2.13: performing an up-sampling operation on the feature map , and connecting the up-sampled feature map with the feature map , to obtain a feature map ;

[0101] Step 3.2.14: processing the feature map using the multi-path gate interaction and dynamic deformation convolution module , to obtain a small-scale feature map ;

[0102] Step 3.2.15: processing the feature map using the convolution module , to obtain a feature map ;

[0103] Step 3.2.16: connecting the feature map with the feature map , to obtain a feature map ;

[0104] Step 3.2.17: processing the feature map using the multi-path gate interaction and dynamic deformation convolution module , to obtain a medium-scale feature map ;

[0105] Step 3.2.18: processing the feature map using the convolution module , to obtain a feature map ;

[0106] Step 3.2.19: connecting the feature map with the feature map , to obtain a feature map ;

[0107] Step 3.2.20: processing the feature map​​​​​ Multi-path gated interaction and dynamic deformation convolution module right Processing is performed to obtain large-scale feature maps. ;

[0108] Step 3.3 Utilizing the target detection head sub-network Computational Image Fruit target detection results;

[0109] Step 3.3.1 Using the detection head module For large-scale feature maps The process is performed to obtain detection results for larger targets. The binary cross-entropy loss function is used to evaluate the classification loss and confidence loss, and the CIoU loss function is used to evaluate the localization loss, thus obtaining... The total loss value;

[0110] Step 3.3.2 Using the detection head module For mesoscale feature maps The data was processed to obtain the detection results for medium-sized targets. The binary cross-entropy loss function is used to evaluate the classification loss and confidence loss, and the CIoU loss function is used to evaluate the localization loss, thus obtaining... The total loss value;

[0111] Step 3.3.3 Using the detection head module For small-scale feature maps The process is performed to obtain detection results for smaller targets. The binary cross-entropy loss function is used to evaluate the classification loss and confidence loss, and the CIoU loss function is used to evaluate the localization loss, thus obtaining... The total loss value;

[0112] Step 3.3.4: Use the nonmaximum suppression algorithm to... and The process involves filtering out redundant bounding boxes with high overlap to obtain the final detection result and its total loss value. ;

[0113] Step 3.4 Based on the total loss value Iterative updates are performed using a backpropagation algorithm based on stochastic gradient descent. The parameters are used to obtain the trained deep convolutional neural network. The number of iterations of the backpropagation algorithm is... The This represents the preset number of iterations. In this embodiment, let... =300;

[0114] Step 4. Detecting the cherry fruit target in the image I by using the trained deep convolutional neural network, and outputting the detection result;

[0115] Specifically, the specific steps of detecting the cherry fruit target in the image I are as follows:

[0116] Step 4.1: using the trained deep convolutional neural network to process the cherry target detection image I to be processed, to obtain small-scale feature maps , medium-scale feature maps , and large-scale feature maps , respectively.

[0117] Specifically, the specific calculation steps of the feature map are as follows:

[0118] Step 4.1.1: using the feature extraction subnetwork based on the gated space-frequency feature distillation to process the image I by using the convolution module to obtain the feature map .

[0119] Step 4.1.2: using the convolution module of to process to obtain the feature map .

[0120] Step 4.1.3: using the multi-path gated interaction and dynamic deformation convolution module of to process to obtain the feature map .

[0121] Step 4.1.4: using the convolution module of to process to obtain the feature map .

[0122] Step 4.1.5: using the multi-path gated interaction and dynamic deformation convolution module of to process to obtain the feature map .

[0123] Step 4.1.6: using the convolution module of to process to obtain the feature map .​

[0124] Step 4.1.7: processing the feature map using the multi-path gated interaction and dynamic deformation convolution module of , to obtain a feature map

[0125] Step 4.1.8: processing the feature map using the convolution module of , to obtain a feature map

[0126] Step 4.1.9: processing the feature map using the multi-path gated interaction and dynamic deformation convolution module of , to obtain a feature map

[0127] Step 4.1.10: processing the feature map using the fast spatial pyramid pooling (SPPF) module of , to obtain a feature map

[0128] Step 4.1.11: performing an up-sampling operation on the feature map , and performing a connection operation on the up-sampled feature map and the feature map , to obtain a feature map

[0129] Step 4.1.12: processing the feature map using the multi-path gated interaction and dynamic deformation convolution module of the multi-dimensional feature interweaving fusion sub-network , to obtain a feature map

[0130] Step 4.1.13: performing an up-sampling operation on the feature map , and performing a connection operation on the up-sampled feature map and the feature map , to obtain a feature map

[0131] Step 4.1.14: processing the feature map using the multi-path gated interaction and dynamic deformation convolution module of , to obtain a small-scale feature map

[0132] Step 4.1.15: processing the small-scale feature map using the convolution module of ​​​​​​​​​​​​​​​​​​​​​​The feature map is obtained through processing. ;

[0133] Step 4.1.16: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] and Perform a connection operation to obtain a feature map. ;

[0134] Step 4.1.17: Utilize Multi-path gated interaction and dynamic deformation convolution module right Processing is performed to obtain mesoscale feature maps. ;

[0135] Step 4.1.18: Utilize Convolutional module right The feature map is obtained through processing. ;

[0136] Step 4.1.19: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] and Perform a connection operation to obtain a feature map. ;

[0137] Step 4.1.20: Utilize Multi-path gated interaction and dynamic deformation convolution module right Processing is performed to obtain large-scale feature maps. ;

[0138] Step 4.2 Utilize the trained detection head module For large-scale feature maps The process is performed to obtain detection results for larger targets. The binary cross-entropy loss function is used to evaluate the classification loss and confidence loss, and the CIoU loss function is used to evaluate the localization loss, thus obtaining... The total loss value;

[0139] Step 4.3 Utilize the trained detection head module For mesoscale feature maps The data was processed to obtain the detection results for medium-sized targets. The binary cross-entropy loss function is used to evaluate the classification loss and confidence loss, and the CIoU loss function is used to evaluate the localization loss, thus obtaining... The total loss value;

[0140] Step 4.4 Utilize the trained detection head module For small-scale feature maps The process is performed to obtain detection results for smaller targets. and the confidence loss value is evaluated by using a binary cross-entropy loss function, and the positioning loss value is evaluated by using a CIoU loss function, so as to obtain the total loss value of ;

[0141] Step 4.5 adopts a non-maximum suppression algorithm to process and , and filters out redundant boxes with high overlap, so as to obtain the final detection result;

[0142] Step 4.6 outputs the final detection result.

[0143] To verify the effectiveness of the present application, a multi-scale cherry target detection data set is constructed by using the collected cherry whole life cycle monitoring images. The data set contains 2870 cherry images in total, and 80% of the images are randomly selected as the training set, and the remaining 20% of the images are selected as the test set. The cherry fruit detection experiment is carried out on the data set by using the present application, and the precision, recall, F1 value, average precision mean mAP 50 , average precision mean mAP 50~95 , parameter quantity, detection speed and other indicators are used to objectively evaluate the detection result, and the evaluation result is compared with the detection result of YOLOv5, YOLOv6, YOLOv8 and YOLOv10.

[0144] Figure 6 Fig. 4 shows the confusion matrix obtained by the embodiment of the present application on the test data set; Figure 7 Fig. 5 shows the detection result of cherry fruits by the embodiment of the present application. As can be seen from Figure 6 and Figure 7 , the present application has achieved high detection accuracy.

[0145] Table 1 shows the comparison of cherry fruit detection performance of the present application and YOLOv5, YOLOv6, YOLOv8 and YOLOv10.

[0146] Table 1 Comparison of detection results of different cherry fruit detection methods

[0147]

[0148] As can be seen from Table 1:

[0149] (1) In terms of detection accuracy

[0150] Firstly, the cherry fruit detection accuracy of the YOLOv5 method is relatively low, and the main reasons are as follows: 1) YOLOv5 generates a lot of redundant information in the feature extraction process, and the interference of these redundant features leads to the decrease of the detection accuracy; 2) the static receptive field mechanism of YOLOv5 has obvious limitations in processing multi-scale objects, which affects the detection performance of small targets; 3) the network structure of YOLOv5 is relatively fixed, and it lacks a dynamic weighting mechanism for different features, which makes it prone to false detection or missed detection when detecting small and color similar targets such as cherries.

[0151] Secondly, although YOLOv6 has improved the detection accuracy to some extent by introducing certain feature fusion and multi-scale strategy, it still has the problem of insufficient detection accuracy when facing small targets and complex backgrounds, especially at a low IoU threshold, its robustness is poor.

[0152] Thirdly, YOLOv8 and YOLOv10 optimize the feature extraction subnetwork and target detection head to some extent, so that they often perform better than YOLOv5 and YOLOv6 when dealing with small targets or complex scenes. However, YOLOv8 and YOLOv10 are still limited by the design of static receptive field, and their flexibility in target detection at different scales is still insufficient.

[0153] Finally, the method of the present application achieves the best evaluation results on all five objective indicators. Compared with YOLOv5, YOLOv6, YOLOv8 and YOLOv10, the present application uses a three-branch network structure to realize the simultaneous extraction of local fine-grained features, spatial correlation features and global context features, and introduces an efficient feature extraction module and a gated frequency feature distillation block to realize the suppression of redundant noise, the compact expression of key features and the dynamic weight distribution of multiple branches, so as to suppress the interference of light changes, target occlusion and the like, thereby enhancing the feature representation ability of the neural network for cherry fruits of different scales and shapes in complex backgrounds, reducing the missed detection and false detection of small targets, and improving the precision and robustness of cherry fruit detection. Experimental data show that, in terms of precision, recall, F1 value, mAP 50 , mAP 50~95 , the precision of the present application is 12.75 percentage points higher than that of YOLOv5, YOLOv6, YOLOv8 and YOLOv10 on average, the recall of the present application is 21.15 percentage points higher than that of YOLOv5, YOLOv6, YOLOv8 and YOLOv10 on average, the F1 value of the present application is 17.78 percentage points higher than that of YOLOv5, YOLOv6, YOLOv8 and YOLOv10 on average, and the mAP 50The mAP of the present application is improved by 17.37% on average compared with YOLOv5, YOLOv6, YOLOv8 and YOLOv10 50~95 The mAP of the present application is improved by 16.55% on average compared with YOLOv5, YOLOv6, YOLOv8 and YOLOv10.

[0154] (2) Parameter amount control

[0155] The neural network model of the present application only needs 2.37M parameters, which is reduced by 5.2% compared with the parameter amount of YOLOv5, and is significantly lower than the model parameter amount of YOLOv6 and YOLOv8. Moreover, the memory occupation of the network model of the present application can be reduced to 1.8MB (2.3MB for YOLOv8) in the deployment of embedded devices.

[0156] (3) Detection speed

[0157] The real-time processing efficiency of the method of the present application is 2.6f / s, which is the same as YOLOv5, and the mAP 50 of the present application is improved by 15.8%; although the YOLOv10 model with lower parameter amount is improved by 0.5f / s in detection speed compared with the present application, the mAP 50~95 of YOLOv10 is reduced by 18.09% compared with the present application; although YOLOv6 increases the parameter amount to improve the detection speed to 3.6f / s, the F1 value is reduced by 20% compared with the present application, and the unit parameter efficiency is only 41.3% of the present application; the detection speed of YOLOv8 is reduced to 2.5f / s under the condition of 3.0M parameters, which exposes the calculation redundancy defect of the traditional architecture.

[0158] In summary, in the cherry fruit detection process, the present application effectively overcomes the interference caused by dynamic scenes such as leaf shielding, fruit shielding and overlapping, diverse light and weather conditions, growth stage, shooting distance and image resolution difference, realizes the cooperative optimization of detection accuracy, environmental robustness (miss detection rate <3.7%), cross-species generalization ability and calculation efficiency, achieves the optimal balance in the precision-speed-parameter amount triangle game, and can provide an effective visual perception solution for cherry automatic harvesting equipment.

Claims

1. A YOLOv8 cherry fruit detection method of gated spatial-frequency feature distillation and fusion, characterized in that The following steps are performed: Step 1. Input a training set of images of multi-scale cherry fruits , a set of manually annotated labels , and a cherry target detection image to be processed ; Step 2. Establish and initialize cherry fruit detection network , the cherry fruit detection network comprises: 1 feature extraction subnetwork based on gated space-frequency feature distillation , 1 multi-dimensional feature interweaving fusion subnetwork , 1 target detection head subnetwork ; The feature extraction subnetwork based on the gated space-frequency feature distillation Convolution modules are sequentially arranged Convolution modules Multi-path gated interaction and dynamic deformation convolution modules Convolution modules Multi-path gated interaction and dynamic deformation convolution modules Convolution modules Multi-path gated interaction and dynamic deformation convolution modules Convolution modules Multi-path gated interaction and dynamic deformation convolution modules And a fast spatial pyramid pooling (SPPF) module ​ The multi-dimensional feature interweaving and fusing sub-network By multi-path gating interaction and dynamic deformation convolution module 、 、 、 And convolution module 、 ; The multi-path gating interaction and dynamic deformation convolution module 、 、 、 、 、 、 、 Each is provided with a bottleneck block composed of a plurality of 1x1 convolution kernels, a cross-channel feature aggregation block composed of a plurality of 1x1 convolution kernels, a projection block composed of a plurality of 1x1 convolution kernels, an efficient feature extraction module, a Split operation layer, a connection operation layer, and a gated frequency feature distillation block; the feature maps output by the bottleneck block are respectively taken as the inputs of the cross-channel feature aggregation block, the efficient feature extraction module, and the Split operation; the output of the Split operation is further processed by the gated frequency feature distillation block; the outputs of the cross-channel feature aggregation block, the efficient feature extraction module, the Split operation, and the gated frequency feature distillation block are connected and then processed by the projection block. The high-efficiency feature extraction module is sequentially provided with a group of bottleneck blocks composed of a plurality of 1x1 convolution kernels, a spatial separable convolution block, and a group of point-by-point convolution blocks composed of a plurality of 1x1 convolution kernels; The gating frequency feature distillation block is sequentially provided with a group of convolution blocks composed of a plurality of 3x3 convolution kernels, a group of bottleneck blocks composed of a plurality of 1x1 convolution kernels, a group of convolution blocks composed of a plurality of 3x3 convolution kernels, a GELU activation function layer, a gating operation layer, a projection block composed of a plurality of 1x1 convolution kernels, and a residual connection operation layer; Step 3. Train with image training set and human-annotated label set , a cherry fruit detection network is trained to obtain a trained deep convolutional neural network ; Step 4. Using the trained deep convolutional neural network Detecting cherry fruit targets in images and outputting the detection results.

2. The gated spatial-channels distillation and fusion YOLOv8 cherry fruit detection method according to claim 1, characterized in that The step 2 is specifically as follows: Step 2.1: Establish and initialize the feature extraction subnetwork based on gated space-frequency feature distillation ; Step 2.2: Establish and initialize multi-dimension feature interweave fusion sub-network ; Step 2.3: Establish and initialize target detection head sub-network , the target detection head sub-network is composed of 3 detection head modules of YOLOv8 , , ; Step 2.4: Establishing a feature extraction subnetwork based on gated space-frequency feature distillation , a multi-dimensional feature interweaving fusion subnetwork , and a feature map interaction relationship between the target detection head subnetwork The output of the fast spatial pyramid pooling (SPPF) module is connected to the output of the multi-path gated interaction and dynamic deformation convolution module after an upsampling operation, and then the connected feature map is taken as the input of the multi-path gated interaction and dynamic deformation convolution module ; the output of the multi-path gated interaction and dynamic deformation convolution module is connected to the output of the multi-path gated interaction and dynamic deformation convolution module after an upsampling operation, and then the connected feature map is taken as the input of the multi-path gated interaction and dynamic deformation convolution module ; and the output of the multi-path gated interaction and dynamic deformation convolution module is taken as the input of the detection head module . The multi-path gating interaction and dynamic deformation convolution module The output is passed through the convolution module. After processing, it interacts with multi-path gating and dynamic deformation convolution modules. The outputs are concatenated, and the concatenated feature maps are then used as multi-path gated interaction and dynamic deformation convolution modules. The input; then the multi-path gating interaction and dynamic deformation convolution module are combined. The output is used as the detection head module. Input; The output of the multi-path gating interaction and dynamic deformation convolution module is connected with the output of the fast spatial pyramid pooling (SPPF) module after processing by the convolution module , and then the connected feature map is taken as the input of the multi-path gating interaction and dynamic deformation convolution module ; the output of the multi-path gating interaction and dynamic deformation convolution module is taken as the input of the detection head module .

3. The gated spatial-channels distillation and fusion YOLOv8 cherry fruit detection method according to claim 2, characterized in that: The convolution module 、 、 、 、 、 、 A group of multiple convolution kernels with a size of 3*3 and a step of 2, a BatchNorm2d batch normalization layer, and a SiLU activation function layer are sequentially arranged.

4. The gated spatial-channels distillation and fusion YOLOv8 cherry fruit detection method according to claim 3, characterized in that The step 3 is specifically as follows: Step 3.

1. Extracting each image from the image training set and its corresponding label from the image training set ; Step 3.2 utilizes a feature extraction subnetwork based on gated space-frequency feature distillation and a multi-dimensional feature interweaving fusion subnetwork , to obtain a multi-path gated interaction and dynamic deformation convolution module an output small-scale feature map , a multi-path gated interaction and dynamic deformation convolution module an output medium-scale feature map and a multi-path gated interaction and dynamic deformation convolution module an output large-scale feature map ; Step 3.3 utilizes the target detection head subnetwork computing the image of the fruit target detection result; Step 3.3.1 utilizes the detection head module to the large-scale feature map is processed to obtain the detection result of the larger target , and a binary cross-entropy loss function is used to evaluate the classification loss value and the confidence loss value, and a CIoU loss function is used to evaluate the positioning loss value, so as to obtain the total loss value of . Step 3.3.2 utilizes the detection head module The mid-scale feature map is centered The processing obtains the detection result of the medium target And the binary cross-entropy loss function is used to evaluate the classification loss value and the confidence loss value, and the CIoU loss function is used to evaluate the positioning loss value, so as to obtain the total loss value of ​ Step 3.3.3 utilizes the detection head module to the small-scale feature map is processed to obtain a detection result of the smaller target , and a binary cross-entropy loss function is used to evaluate the classification loss value and the confidence loss value, and a CIoU loss function is used to evaluate the positioning loss value, so as to obtain a total loss value of . Step 3.3.4: using non-maximum suppression algorithm to process 、 and , filter out redundant boxes with high overlap, and further obtain the final detection result and its total loss value ; Step 3.4 updating parameters of the deep convolutional neural network according to the total loss value by using a back-propagation algorithm based on a stochastic gradient descent method to obtain a trained deep convolutional neural network , wherein the back-propagation algorithm is iterated for times , and the times represent a preset iteration number.

5. The gated spatial-channels distillation and fusion YOLOv8 cherry fruit detection method according to claim 4, characterized in that The step 4 is specifically as follows: Step 4.1: Utilizing the trained deep convolutional neural network detecting a cherry target image to be processed processing, respectively, to obtain a small-scale feature map a medium-scale feature map and a large-scale feature map ; Step 4.2 using the detection head module that has completed training processing, to obtain the detection result of the larger target and using a binary cross-entropy loss function to evaluate the classification loss value and the confidence loss value, and using a CIoU loss function to evaluate the positioning loss value, to obtain the total loss value of the detection result ​​ Step 4.3 Utilizing the detection head module with completed training The mid-scale feature map is centered The processing is performed to obtain the detection result of the medium target And the binary cross-entropy loss function is used to evaluate the classification loss value and the confidence loss value, and the CIoU loss function is used to evaluate the positioning loss value, so as to obtain the total loss value of . Step 4.4 using the detection head module that has completed training to the small-scale feature map processing to obtain the detection result of the smaller target and using a binary cross-entropy loss function to evaluate the classification loss value and the confidence loss value, using a CIoU loss function to evaluate the positioning loss value, thereby obtaining the total loss value of the detection result ​ Step 4.

5. Process the results of step 4.4 using a non-maximum suppression algorithm to filter out redundant boxes with high overlap, and obtain the final detection results. , and ​ Step 4.6 outputs the final detection result.

Images