Improved YOLOv8n-based cloudy orchard multi-class sheltered pear detection method

Abstract

The invention belongs to the technical field of agricultural robot visual perception and intelligent decision crossing, and particularly relates to an improved YOLOv8n-based cloudy day orchard multi-class sheltered pear detection method, which comprises the following steps of: obtaining a to-be-detected image containing pear fruits in a cloudy day environment; inputting the to-be-detected image into a trained improved YOLOv8n detection model, and receiving and outputting a detection result generated by the improved YOLOv8n detection model; the detection result comprises a bounding box position of the pear fruit and a corresponding shielding state category label; the sheltering state category labels comprise no sheltering, sheltering among fruits and sheltering of branches and leaves; the problems that an existing pome detection method is difficult to stably work in a cloudy low-light environment and cannot finely distinguish the fruit shielding state are solved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of agricultural robot visual perception and intelligent decision-making, specifically involving a method for detecting pears with multiple types of shading in orchards on cloudy days based on an improved YOLOv8n. Background Technology

[0002] Automated pear harvesting is a key technology for addressing agricultural labor shortages. However, the vision systems of harvesting robots face severe challenges in orchard environments with low light, such as on cloudy days. Insufficient light leads to decreased overall image contrast, distorted color information, and blurred fruit outlines and surface textures. This is especially true for pear varieties with yellowish-green peels, whose color characteristics are highly similar to those of background leaves under diffused light on cloudy days, further increasing the difficulty of distinguishing the target from the background. These factors collectively weaken the recognition capabilities of traditional vision algorithms, resulting in a significant increase in missed detection and false detection rates in actual operations.

[0003] Object detection technology has evolved from traditional methods relying on manual features to automatic feature extraction based on deep learning. Among these, single-stage detection models such as YOLO are widely used in agricultural inspection due to their balance between speed and accuracy. While current research has made progress in fruit detection under normal lighting conditions, robust detection methods specifically designed for low-light environments, such as cloudy days, remain a weak point. Furthermore, lightweight model design is crucial to meet the deployment requirements of embedded robotic platforms.

[0004] More importantly, existing research largely focuses on detecting the presence of fruit or making only coarse-grained judgments of occlusion / unocclusion. However, in actual automated harvesting operations, the successful grasping and efficient path planning of robotic arms heavily rely on more refined fruit state information. For example, robots need to adopt drastically different approach strategies, grasping plans, and obstacle avoidance maneuvers for unoccluded, partially occluded, severely occluded, and overlapping fruits. The lack of fine-grained perception of the type and degree of fruit occlusion has become a key bottleneck restricting the advancement of harvesting robots from the "seeing" stage to the "grasping" and "grasping well" stage.

[0005] Therefore, it is necessary to invent a method for detecting pears with multiple types of shading in orchards on cloudy days based on an improved YOLOv8n. Summary of the Invention

[0006] To address the problem that existing pear fruit detection methods are difficult to operate stably in cloudy, low-light conditions and cannot accurately distinguish the shading status of the fruit, this invention provides a multi-category shading pear fruit detection method in cloudy orchards based on an improved YOLOv8n.

[0007] This invention is achieved using the following technical solution:

[0008] A method for detecting pears with multiple types of shading in orchards on cloudy days, based on an improved YOLOv8n, includes the following steps:

[0009] Acquire images of pear fruits to be detected under cloudy conditions;

[0010] The image to be detected is input into the trained improved YOLOv8n detection model, and the detection results generated by the improved YOLOv8n detection model are received and output. The detection results include the bounding box position of the pear fruit and its corresponding occlusion state category label. The occlusion state category label includes: no occlusion, occlusion between fruits, and occlusion by branches and leaves.

[0011] The improved YOLOv8n detection model is constructed and trained through the following steps:

[0012] The YOLOv8n model was used as the baseline model.

[0013] Replace the standard downsampling operations in the backbone and neck networks of the baseline model with the ADown module;

[0014] The detection head of the baseline model is reconstructed using lightweight design.

[0015] Replace the bounding box regression loss function of the baseline model with the WIoU loss function based on the distance attention mechanism;

[0016] The improved model was trained using a dataset of labeled pear images taken on a cloudy day, resulting in the improved YOLOv8n detection model.

[0017] Furthermore, the execution process of the ADown module includes:

[0018] The input feature map is subjected to average pooling to obtain the first feature map;

[0019] The first feature map is divided into a first part and a second part in the channel dimension;

[0020] Perform a depthwise separable convolution on the first part to obtain the first branch output;

[0021] The second part is then subjected to max pooling and depthwise separable convolution in sequence to obtain the output of the second branch;

[0022] The output of the first branch and the output of the second branch are concatenated along the channel dimension to obtain the downsampled output.

[0023] Furthermore, the lightweight reconstruction of the detection head of the baseline model specifically involves:

[0024] At the input end of the detection head, two sets of consecutive grouped convolutional layers are configured for both the classification branch and the regression branch; wherein the classification branch and the regression branch share the convolutional kernel weights of these two sets of grouped convolutional layers.

[0025] Furthermore, the WIoU loss function based on the distance attention mechanism is WIoUv1, and its loss value is... L WIoUv1 The calculation formula is:

[0026] ;

[0027] in, L IoU The formula for calculating the intersection-union ratio (IU) loss between the bounding box and the ground truth bounding box is as follows:

[0028] ;

[0029] In the formula, This represents the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box.

[0030] The R WIoU The dynamic non-monotonic focusing coefficient is calculated using the following formula:

[0031] ;

[0032] In the formula, and This indicates the coordinates of the center point of the predicted bounding box. and This represents the coordinates of the center point of the actual bounding box. This represents the square of the diagonal length of the smallest bounding rectangle between the predicted bounding box and the true bounding box.

[0033] Furthermore, the labeled pear image dataset under cloudy conditions is constructed through the following steps:

[0034] Images of original pear orchard scenes under cloudy conditions were collected, covering typical complex orchard scenes under cloudy conditions, including single fruit, multiple fruit, overlapping fruit, and branches and leaves obscuring the fruit.

[0035] To enhance sample diversity and improve the utilization rate of the target area, the original pear orchard scene image was cropped to obtain image samples of different sizes and scene compositions, but all with pears as the main target.

[0036] To improve the robustness and generalization ability of the improved YOLOv8n detection model, data augmentation operations are applied to the cropped image samples, and the samples are divided into training, validation, and test sets. The data augmentation operations include at least one of random addition of salt-and-pepper noise, image sharpening, affine transformation, and brightness adjustment.

[0037] The augmented image samples are labeled, and each pear is labeled as unoccluded, occluded between fruits, or occluded by branches and leaves, based on the actual situation of the pear fruit being occluded. Unoccluded means that the fruit is not occluded, occluded between fruits means that the fruit is occluded by other fruits, and occluded by branches and leaves means that the fruit is occluded by branches or leaves. Thus, the labeled pear fruit image dataset under the cloudy environment is obtained, which consists of a training set, a validation set, and a test set.

[0038] Furthermore, the method also includes an application deployment step:

[0039] The trained and improved YOLOv8n detection model was deployed to the embedded vision processing unit of the pear fruit picking robot.

[0040] The embedded vision processing unit acquires and detects images of pears under cloudy conditions in real time, and outputs fruit location information containing labels for different occlusion states. This provides pear perception information, including location and occlusion category, for the robotic arm path planning of the pear picking robot.

[0041] This invention provides a method for detecting multiple types of shaded pears in orchards on cloudy days based on an improved YOLOv8n, which has the following advantages compared to existing technologies:

[0042] 1. Improved detection accuracy under low light conditions: Compared with the original YOLOv8n detection model, the improved YOLOv8n detection model of this invention improves accuracy by 1.7%, recall, mAP@0.5 and mAP@0.95 by 2.6%, 1.9% and 1.2% respectively, indicating that the model of this invention still has stable and reliable recognition capabilities under low light conditions.

[0043] 2. Excellent and balanced detection performance for multiple types of occluded fruits: For the three types of pears commonly encountered in actual harvesting—no occlusion (NO), fruit-to-fruit occlusion (FCC), and branch-to-leaf occlusion (OBL)—the model achieved average precision (AP) of 94.9%, 93.1%, and 93.0%, respectively. The detection performance of each type remained high and balanced, providing a perceptual basis for accurately locating fruits under different occlusion conditions.

[0044] 3. Possesses excellent lightweight characteristics and deployment potential: The model has only 4.2M parameters and the computational cost (FLOPs) is controlled at 5.0G. While achieving a significant improvement in detection accuracy, it maintains a lightweight structure, which is conducive to efficient deployment in resource-constrained pear orchard harvesting machinery embedded platforms, laying a technical foundation for a real-time visual perception and grasping planning integrated system.

[0045] In summary, this method not only significantly improves the overall detection performance of pears under cloudy conditions, but also enables precise and stable identification of various types of shaded fruits, balancing accuracy and efficiency. This provides reliable technical support for subsequent intelligent harvesting path planning based on multi-category shading information. Attached Figure Description

[0046] Figure 1 This is an overall flowchart of the method for detecting pears under multiple types of shading in orchards on cloudy days, as described in this invention.

[0047] Figure 2 This is a schematic diagram of the structure of the improved YOLOv8n detection model of the present invention.

[0048] Figure 3 This is a schematic diagram of the ADown module in this invention.

[0049] Figure 4 This is a schematic diagram of the lightweight detection head structure based on weight sharing in this invention.

[0050] Figure 5 This is a schematic diagram of the distance attention mechanism in the WIoU loss function of the present invention.

[0051] Figure 6 This is a comparison chart of the detection accuracy between the improved YOLOv8n model and the original YOLOv8n model of this invention.

[0052] Figure 7 Here is a comparison example of the feature attention heatmap of the improved YOLOv8n detection model in this invention under different occlusion conditions: (a) Original input image; (b) Corresponding model feature attention heatmap.

[0053] Figure 8 Examples of comparison images of the detection results of the improved YOLOv8n detection model of this invention and the original YOLOv8n model in several typical scenarios are as follows: (a) Detection result of the original YOLOv8n model in a sparse fruit scenario; (b) Detection result of the improved YOLOv8n detection model of this invention in a sparse fruit scenario; (c) Detection result of the original YOLOv8n model in a dense fruit scenario; (d) Detection result of the improved YOLOv8n detection model of this invention in a dense fruit scenario; (e) Detection result of the original YOLOv8n model in a cloudy low-light scenario; (f) Detection result of the improved YOLOv8n detection model of this invention in a cloudy low-light scenario. Detailed Implementation

[0054] The present invention will be further explained and described below with reference to the accompanying drawings, embodiments, and comparative examples. Obviously, the described embodiments are only a part of the embodiments, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0055] Example

[0056] A method for detecting pears in orchards under multiple types of shading on cloudy days, based on an improved YOLOv8n method, is attached. Figure 1 As shown, it includes the following steps:

[0057] I. Construction of a dataset of labeled pear images under cloudy conditions

[0058] To achieve accurate detection of pears under cloudy conditions, it is first necessary to construct an image dataset suitable for this scenario. The specific steps are as follows:

[0059] Image Acquisition: This embodiment uses mature red fragrant pears as the research object, and image acquisition was conducted under cloudy conditions. Smartphones were used as shooting devices, including but not limited to Vivo Z3 and OPPO Reno5 smartphones. The shooting distance was controlled at 30-50 cm. Images of the pear orchard, including typical complex scenes such as single fruit, multiple fruits, overlapping fruits, and foliage obstruction, were acquired from multiple shooting angles, including but not limited to front, side, upward, and downward views. A total of 734 original images were acquired, with an image resolution of 3648×4624 pixels.

[0060] Image preprocessing and cropping: To improve the utilization of the target area and enhance data diversity, the original high-resolution image is cropped to generate sample images of different sizes suitable for model input. The sample images are mainly based on pear fruits.

[0061] Data Augmentation and Dataset Splitting: To improve the robustness of the subsequently constructed model in complex cloudy environments, data augmentation operations were applied to the cropped image samples. These operations included at least one of the following: random addition of salt-and-pepper noise, image sharpening, affine transformation (rotation, translation, scaling), and brightness adjustment. After augmentation, the total number of samples was expanded to 2936 images, which were randomly divided into a training set (2055 images), a validation set (293 images), and a test set (588 images) in a 7:1:2 ratio.

[0062] Data Labeling and Category Definition: In natural environments, pears are often obscured by leaves or branches, and the fruits themselves often block each other, posing a significant challenge to the harvesting robot's robotic arm. To improve harvesting efficiency, the harvesting robot can adopt different harvesting strategies when encountering pears in different scenarios. For example, for unobstructed targets, path planning is relatively simple, and conventional path planning and grasping tasks can be performed directly. When the target is partially obscured, path planning needs to consider how to bypass the obstruction or adjust the grasping angle. In environments with densely packed fruits, considering the occlusion and overlap of multiple fruits, a multi-objective path planning algorithm can be used to determine the optimal path. Therefore, in this invention, based on the fruit's location, pears are defined into three categories:

[0063] Unobstructed: The fruit is fully visible and not obstructed by branches, leaves or other fruits; marked as NO.

[0064] Inter-fruit occlusion: A fruit is partially or completely obscured by other fruits, marked as FCC;

[0065] Obstruction by branches or leaves: The fruit is partially or completely obscured by branches or leaves, marked as OBL.

[0066] In cases where branches, leaves, and fruits simultaneously create shading, the situation is prioritized for classification as FCC to reduce the risk of harvesting damage.

[0067] The LabelImg tool was used to perform the above fine annotation on the data augmented image samples to form a labeled pear image dataset under cloudy conditions, consisting of a training set, a validation set, and a test set.

[0068] Statistical Analysis of Dataset Characteristics: To clarify the size distribution of pear fruit targets in this dataset and better understand the performance focus of subsequent models, target size statistics were performed on the labeled dataset. Targets with bounding box areas smaller than 32×32 pixels were defined as small targets, those larger than 96×96 pixels as large targets, and those in between as medium targets. Statistical results show that in a cloudy orchard scene, due to the moderate shooting distance (30-50 cm), large pear fruit targets dominate in all occlusion categories (NO, FCC, OBL) (each category accounting for over 55%), while small targets account for less than 5% in all cases. This distribution characteristic aligns with the actual visual perception of a harvesting robot operating at close range, and also implies that the model optimization of this invention will focus on the stable detection of large and medium-sized fruits under different occlusion conditions.

[0069] II. Construction of an Improved YOLOv8n Detection Model for Cloudy Environments

[0070] As attached Figure 2As shown, this invention uses the YOLOv8n model as the baseline model and makes systematic improvements to the complex scene of cloudy low light and multiple occlusions from three aspects: feature extraction, network lightweighting and loss optimization, to build the final improved YOLOv8n detection model.

[0071] Feature extraction enhancement based on the ADown (Asymmetric Downsampling) module: To alleviate the problem of blurred image details and information loss caused by traditional downsampling in cloudy weather, the standard downsampling operation in the backbone and neck networks of the original YOLOv8n model is replaced with the ADown module.

[0072] As attached Figure 3 As shown, the execution process of the ADown module includes:

[0073] The first feature map is obtained by performing average pooling (AvgPool) on the input feature map of size h×w×c to reduce the size to (h-1)×(w-1)×c.

[0074] The first feature map is divided into a first part and a second part in the channel dimension, and the size of each part is (h-1)×(w-1)×c / 2;

[0075] Perform a depthwise separable convolution on the first part to further reduce the size to h / 2×w / 2×c / 2, and obtain the first branch output;

[0076] Max pooling and depthwise separable convolution are performed sequentially on the second part, and the size is further reduced to h / 2×w / 2×c / 2 to obtain the output of the second branch.

[0077] The outputs of the first branch and the second branch are concatenated along the channel dimension to form a unified feature map of size h / 2×w / 2×c, which is used as the downsampled output.

[0078] The ADown module employs a dual-branch design: one branch performs downsampling using average pooling and depthwise separable convolution, while the other branch performs downsampling using max pooling and depthwise separable convolution. Finally, features are concatenated along the channel dimension. This approach reduces the spatial resolution of the feature map and the computational load while more effectively preserving the edge and detail information of the image, significantly enhancing the model's basic feature extraction capabilities under low-light conditions.

[0079] Lightweight Reconstruction of the Detection Head Based on Shared Weights and Group Convolution: To reduce the computational complexity of the model and adapt it to embedded deployment, the decoupled detection head of the original YOLOv8n model is reconstructed. (See attached image) Figure 4 As shown, the improvement of the present invention lies in:

[0080] Firstly, group convolution is introduced: the standard 3×3 convolution in the decoupled detection head is replaced with group convolution. By grouping the input feature channels and performing independent convolution operations, the number of parameters and computational cost are significantly reduced while maintaining feature extraction capabilities.

[0081] Secondly, shared convolutional weights: At the input of the classification (Cls) and regression (Reg) branches, two identical groups of convolutional layers are shared. These two branches share convolutional kernel weights, which avoids redundant calculation of similar feature maps, thus improving computational efficiency and enabling a lightweight design of the detection head.

[0082] Optimization of Bounding Box Regression Loss Function Based on WIoU (Wise Intersection Over Union): The bounding box loss function is a crucial component of the object detection loss function; a well-defined bounding box loss function can significantly improve the performance of the object detection model. The CIoU (Complete Intersection Over Union) loss function used in YOLOv8n applies the same optimization weights to difficult targets such as overlap and occlusion as to simple targets, resulting in limited accuracy in detecting pears in complex scenes. Therefore, this invention replaces the original CIoU loss function with the WIoUv1 loss function, whose loss value... L WIoUv1 The calculation formula is:

[0083] ;

[0084] in, L IoU The formula for calculating the intersection-union ratio (IU) loss between the bounding box and the ground truth bounding box is as follows:

[0085] ;

[0086] In the formula, This represents the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box.

[0087] The R WIoU The dynamic non-monotonic focusing coefficient is calculated using the following formula:

[0088] ;

[0089] In the formula, and This indicates the coordinates of the center point of the predicted bounding box. and This represents the coordinates of the center point of the actual bounding box. This represents the square of the diagonal length of the smallest bounding rectangle between the predicted bounding box and the true bounding box.

[0090] As attached Figure 5 As shown, the core of WIoUv1 lies in constructing a dynamic attention mechanism based on normalized distance. This mechanism utilizes a dynamic weight coefficient that is monotonically positively correlated with the normalized distance (i.e., the... R WIoU This is achieved by: the coefficient monotonically increasing as the normalized distance between the center points of the predicted bounding box and the ground truth bounding box increases. This allows the loss function to adaptively balance the influence of samples of different quality—applying higher loss weights to lower-quality samples that are farther away, while relatively reducing attention to higher-quality samples that are closer. Through this design, WIoUv1 prevents model training from being dominated by extreme samples of extremely high or extremely low quality, instead focusing more on optimizing a larger number of samples of average quality. This effectively accelerates the model's convergence process and significantly improves the localization accuracy for pear fruits with difficult occlusion and overlap.

[0091] III. Model Training

[0092] Training Environment and Parameter Settings: The improved YOLOv8n detection model constructed in Step 2 was trained using the training and validation sets obtained in Step 1. Model training was performed on a workstation equipped with an NVIDIA GeForce RTX 4080 graphics card, using PyTorch 2.0.1 as the deep learning framework. During training, the input image size was fixed at 640×640, the batch size was set to 32, the SGD optimizer was selected, the initial learning rate was 0.01, and cosine annealing was used for dynamic adjustment with a weight decay coefficient of 0.0005. The total number of training epochs was 200. Finally, the improved YOLOv8n detection model was obtained after training.

[0093] IV. Model Deployment and Application

[0094] Model Deployment: The trained model weight files (.pt format) can be exported and deployed in the embedded vision processing unit of the harvesting robot using inference optimization tools such as TensorRT and ONNXRuntime. The core hardware of this unit can utilize embedded computing platforms such as the NVIDIA Jetson series. The model's high accuracy and lightweight characteristics ensure that it provides the robotic arm with real-time and accurate perception information, including fruit location and its fine occlusion category, even in cloudy conditions, thus supporting subsequent adaptive path planning and grasping decisions.

[0095] In practical applications, based on the aforementioned trained and deployed model, the detection method of this invention is executed according to the following core steps:

[0096] First, the embedded vision processing unit acquires images of pears to be detected in real time under cloudy conditions.

[0097] Subsequently, the image to be detected is input into the trained improved YOLOv8n detection model.

[0098] Finally, the detection results generated by the model are received and output. The detection results include the bounding box position information of each pear fruit and its corresponding occlusion state category label (no occlusion, fruit-to-fruit occlusion, or branch-to-leaf occlusion), providing a perception basis for the precise operation of the harvesting robot.

[0099] V. Model Evaluation Experiment and Analysis

[0100] The core evaluation metrics include: precision (P), recall (R), average precision (AP), mean average precision (mAP), number of model parameters, and floating-point operations (FLOPs).

[0101] Experimental environment and network settings: The system ran on Windows 10, equipped with 32 GB of RAM and an NVIDIA GeForce RTX 4080 graphics card, and an Intel(R) Core(TM) i7-13700F @2.10GHz processor. The deep learning framework used was PyTorch 2.0.1, with CUDA 11.8 and CUDNN 8.8.0. Network training parameters were set as follows: image input size was 640×640, batch size was 32, and the maximum number of iterations was 200. The optimizer was SGD, the learning rate was dynamically adjusted using cosine annealing, the initial learning rate was set to 0.01, the momentum factor was 0.937, and the weight decay coefficient was 0.0005.

[0102] (a) Experiments and results of the improved model of the present invention

[0103] The improved model of this invention was validated using 588 pear images in the test set. Table 1 shows the detection results of the improved model for different types of pear fruits. As can be seen from Table 1, the AP values ​​of the improved model for NO, FCC, and OBL pear fruits reached 94.9%, 93.1%, and 93.0%, respectively. The mean precision, recall, and average precision for different types of pear fruit targets were 90.8%, 87.8%, and 93.7%, respectively, indicating that the model performed well in detecting different types of pear fruit targets.

[0104] Table 1. Detection results of the improved model of this invention

[0105] unit:%

[0106]

[0107] To evaluate the performance improvement and statistical significance of the improved model in target detection, the 95% confidence interval of the average AP was calculated using the Bootstrapping method based on the AP value of each image in the test set of 588 images. A paired t-test was then used to compare the performance difference between the improved model and the baseline model. The results showed that the 95% confidence interval of the improved model was [0.9020, 0.9214], indicating a high confidence level. Compared with the baseline model, the paired t-test result was t = 2.9656, p = 0.0031, significantly lower than the significance level of 0.01, indicating that the improvement in the AP index of the improved model in target detection is statistically significant. The above analysis shows that the improvement in detection performance of the improved model is not only practically significant but also effectively supported by rigorous statistical tests, enhancing the credibility and persuasiveness of the experimental conclusions.

[0108] (b) Ablation experiments and comparative verification

[0109] To evaluate the effectiveness and feasibility of the improved model of this invention in detecting pear fruit without occlusion, occlusion between fruits, and occlusion by branches and leaves, ablation experiments were conducted based on the YOLOv8n model. In the experiments, the hardware environment and parameter settings used for training all models were kept consistent, and the results were objectively evaluated using test set data. Table 2 shows the results of five ablation experiments. First, the introduction of the ADown module (YOLOv8n-A) significantly improved the model's feature extraction capability under low light and blurred fruit edges. Compared to the basic YOLOv8n model, precision, recall, mAP@0.5, and mAP@0.95 were improved by 0.6%, 2.1%, 1.5%, and 1.3%, respectively. This improvement effectively alleviated the shortcomings of traditional downsampling methods in capturing details, enhanced the model's ability to identify pear fruit outlines in low-light environments, and thus reduced missed detections. Secondly, the improved lightweight detection head (YOLOv8n-H) not only reduces the model's computational complexity by introducing group convolution and sharing mechanisms, but also enhances the ability to identify overlapping and occluded fruit regions, resulting in a 0.5% improvement in accuracy. The improved detection head optimizes feature separation for complex targets, enhancing the model's ability to distinguish adjacent fruits and effectively reducing false detections. Combining ADown with the improved detection head (YOLOv8n-AH) achieves complementary advantages, significantly improving the overall model performance, increasing mAP@0.5 to 93.4%, while FLOPs and model size are greatly reduced, validating the dual goals of lightweight design and performance improvement. Finally, the WIoU loss function was introduced into the improved model of this invention to further accelerate the model convergence speed and improve the localization accuracy. Especially in samples under cloudy and low-light conditions, WIoU reduces the influence of low-quality bounding boxes, which significantly enhances the robustness of the model in complex natural environments. Compared with the basic YOLOv8n model, the precision, recall, mAP@0.5 and mAP@0.95 were improved by 1.7%, 2.6%, 1.9% and 1.2%, respectively, while the model size and FLOPs were reduced by 32.3% and 38.3%, respectively.

[0110] In summary, the ablation experiments demonstrate that the ADown module, lightweight detection head, and WIoU loss function introduced in this invention address the key challenges in detecting pears on cloudy days from three perspectives: feature extraction, structural design, and loss optimization, respectively. Working synergistically, these three components significantly enhance the model's robustness and accuracy in complex agricultural scenarios such as low light and multiple occlusions, while ensuring lightweight design and real-time performance, achieving an effective balance between accuracy and efficiency.

[0111] Table 2 Ablation Test Results

[0112]

[0113] The curves showing the changes in precision, recall, mAP@0.5, and mAP@0.95 of the improved model and the original YOLOv8n model over 200 training epochs are attached. Figure 6 As shown in the figure, compared with the original YOLOv8n model, the improved method of this invention has improvements in precision, recall, mAP@0.5, and mAP@0.95.

[0114] (c) Comparison of experiments with different detection heads

[0115] This invention implements a lightweight design for the YOLOv8n baseline model: by reducing the number of parameters through convolution operations on a shared detection head, the detection head is thus lightweighted, as shown in the attached figure. Figure 4 As shown, a combined structure of shared 1×1 convolutional layers and shared 3×3 convolutional layers was designed for the YOLOv8n detection head. Three schemes were designed in this experiment: (1) sharing two 3×3 convolutional layers; (2) sharing two 3×3 grouped convolutional layers; (3) sharing one 1×1 convolutional layer and one 3×3 convolutional layer. The experimental results of the three schemes are compared in Table 3. As can be seen from Table 3, compared with schemes (1) and (3), scheme (2) maintains a high mAP value of 93.7% while reducing the model size and improving the detection speed. Therefore, scheme (2) - that is, the design of sharing two 3×3 grouped convolutional layers - was finally adopted as the lightweight architecture of the YOLOv8 detection head.

[0116] Table 3. Experimental results of different detection head designs

[0117]

[0118] (d) Ablation experiment on the effectiveness of data augmentation strategies

[0119] To quantitatively evaluate the overall contribution of data augmentation strategies, this invention designed an ablation experiment: under the exact same model architecture (the improved model in this paper) and hyperparameter settings, the model performance was compared with and without any data augmentation strategies. The experimental results are shown in Table 4. Analysis shows that the data augmentation strategy significantly improved all detection metrics, fully demonstrating its effectiveness. After using data augmentation, the model's mAP@0.5 increased from 76.2% to 93.7%, an absolute increase of 17.5%, indicating a qualitative leap in the overall average detection accuracy of the model. mAP@0.95 also increased from 51.5% to 69.9%, an improvement of 18.4%, demonstrating that data augmentation effectively improved the accuracy of the model's bounding box prediction. After adopting the data augmentation strategy, the pear fruit detection performance was significantly improved in different scenarios. The improvement was most significant in the inter-fruit occlusion (FCC) scenario, where mAP@0.5 jumped dramatically from 69.6% to 93.1%, an increase of 23.5%, far exceeding other categories. This demonstrates that data augmentation strategies (such as affine transformation and image sharpening) significantly enhance the model's ability to distinguish densely occluded fruit, effectively addressing the challenge of detecting objects that are stuck together. The improvement in the branch-and-leaf occlusion (OBL) scene is also significant, with mAP@0.5 jumping from 77.9% to 93%, an improvement of 15.1%. This proves that data augmentation strategies (such as adding noise and adjusting brightness) effectively improve the model's robustness to changes in lighting and branch-and-leaf occlusion. Even in the relatively simple unoccluded (NO) scene, mAP@0.5 increased from 81.2% to 94.9%, an improvement of 13.7%, indicating that data augmentation is also effective in improving the detection stability of basic scenes. Precision (P) and recall (R) for all categories also showed significant improvements, indicating that the model significantly reduced missed detections after data augmentation and found more true targets. This ablation experiment quantitatively confirms the effectiveness of the data augmentation strategy employed. It effectively expands the distribution of training data, significantly enhances the model's generalization ability and detection accuracy in different scenarios, avoids overfitting, and makes the model more suitable for real orchard environments.

[0120] Table 4. Results of ablation experiments using data augmentation strategies

[0121]

[0122] (e) Comparison of experiments with different target detection models

[0123] The detection results of our improved model are compared with those of mainstream object detection models such as Faster R-CNN, RT-DETR, YOLOv3, YOLOv5n, YOLOv7, YOLOv8n, YOLOv9s, YOLOv10n, and YOLOv11n, as shown in Table 5. The comparison of experimental results across all models shows that RT-DETR has the highest precision and recall among all models. However, the RT-DETR model has excessively large FLOPs and parameters, making it unsuitable for real-time detection by harvesting robots. Our improved model's mAP@0.5 is 14.7%, 0.5%, 0.8%, 2.7%, 12.2%, 2.5%, 11.5%, 2.5%, and 1.8% higher than Faster R-CNN, RT-DETR, YOLOv3, YOLOv5n, YOLOv7, YOLOv8n, YOLOv9s, YOLOv10n, and YOLOv11n, respectively. The improved method of this invention also demonstrates advantages in terms of precision, recall, FLOPs, and parameter count. Therefore, through a comprehensive comparison of all indicators, the improved method presented in this paper is more suitable for pear target detection tasks of different categories under complex overcast conditions.

[0124] Table 5 Detection results of different detection models

[0125]

[0126] VI. Visualization Results

[0127] This study uses the Grad-CAM (Gradient-weighted Class Activation Mapping) method to visualize the features of an improved pear fruit detection model. This method generates a heatmap through gradient weighting, where a redder color indicates a higher contribution of the corresponding region to the model's judgment, reflecting the network's attention to different parts of the image. Figure 7 The results of the improved model of this invention are visualized using heatmaps in a complex orchard environment under cloudy conditions. It can be seen that the improved model can effectively focus on the actual area of ​​the pear fruit, especially under conditions of low light and foliage obstruction, maintaining high response accuracy and regional concentration, indicating that the model has a good ability to capture and distinguish key features.

[0128] To compare model performance from a qualitative perspective, Figure 8 The results of the improved model of this invention and the original YOLOv8n model in several typical scenarios are shown. Red circles mark missed detection areas, and yellow circles mark false positive areas. Analysis shows that in the sparse fruit scenario (… Figure 8 (a) and Figure 8In (b), when the fruit occlusion area is greater than 80%, the original model shows significant missed detections, while the improved model can detect the fruit, indicating an enhanced feature extraction capability. In dense fruit scenes ( Figure 8 (c) and Figure 8 In (d), the original model exhibited duplicate detections, while the improved model did not, indicating its superior ability to distinguish dense targets. In overcast, low-light scenarios (…), Figure 8 (e) and Figure 8 In (f), the original model exhibited both false negatives and false positives. The improved model demonstrated more complete detection results in these cases, initially indicating its adaptability to low-light environments. In summary, in the typical scenarios presented, the improved model in this paper demonstrated more stable detection performance and scene adaptability compared to the original model.

[0129] In summary, this invention has constructed and implemented a lightweight, high-precision pear fruit detection method specifically designed for complex environments such as cloudy days. It effectively solves the problems of low fruit recognition accuracy and high false negative rate under low light conditions on cloudy days, and provides key technical support for intelligent decision-making in automated harvesting.

[0130] In the specific implementation process, it is important to note that after model training and evaluation are completed, the performance metrics on the test set (such as mean average precision, mAP) are compared with a preset deployment performance threshold. If the evaluation result meets or exceeds the threshold, it indicates that the model performance has met the application requirements, and the trained model weight file can be exported and deployed to the target hardware platform (such as the embedded system of a pear-picking robot). If the evaluation result does not meet the threshold, iterative optimization and adjustment of the model structure or training parameters (such as network depth, loss function weights, data augmentation strategies, and learning rate) are required, followed by retraining and evaluation until the model performance meets the final deployment and application standards. This process ensures the reliability of the model in real-world applications.

[0131] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting pears with multiple types of shading in orchards on cloudy days based on an improved YOLOv8n, characterized by: Includes the following steps: Acquire images of pear fruits to be detected under cloudy conditions; The image to be detected is input into the trained improved YOLOv8n detection model, and the detection results generated by the improved YOLOv8n detection model are received and output. The detection results include the bounding box position of the pear fruit and its corresponding occlusion status category label; The occlusion status category labels include: no occlusion, occlusion between fruits, and occlusion by branches and leaves; The improved YOLOv8n detection model is constructed and trained through the following steps: The YOLOv8n model was used as the baseline model. Replace the standard downsampling operations in the backbone and neck networks of the baseline model with the ADown module; The detection head of the baseline model is reconstructed using lightweight design. Replace the bounding box regression loss function of the baseline model with the WIoU loss function based on the distance attention mechanism; The improved model was trained using a dataset of labeled pear images taken on a cloudy day, resulting in the improved YOLOv8n detection model.

2. The method for detecting pears with multiple types of shading in orchards on cloudy days based on the improved YOLOv8n according to claim 1, characterized in that: The execution process of the ADown module includes: The input feature map is subjected to average pooling to obtain the first feature map; The first feature map is divided into a first part and a second part in the channel dimension; Perform a depthwise separable convolution on the first part to obtain the first branch output; The second part is then subjected to max pooling and depthwise separable convolution in sequence to obtain the output of the second branch; The output of the first branch and the output of the second branch are concatenated along the channel dimension to obtain the downsampled output.

3. The method for detecting multiple types of shaded pears in orchards on cloudy days based on the improved YOLOv8n according to claim 1, characterized in that: The lightweight reconstruction of the detection head of the baseline model specifically involves: At the input end of the detection head, two sets of consecutive grouped convolutional layers are configured for both the classification branch and the regression branch; wherein the classification branch and the regression branch share the convolutional kernel weights of these two sets of grouped convolutional layers.

4. The method for detecting pears with multiple types of shading in orchards on cloudy days based on the improved YOLOv8n according to claim 1, characterized in that: The WIoU loss function based on the distance attention mechanism is WIoUv1, and its loss value is... L WIoUv1 The calculation formula is: ； in, L IoU The formula for calculating the intersection-union ratio (IU) loss between the bounding box and the ground truth bounding box is as follows: ； In the formula, This represents the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box. The R WIoU The dynamic non-monotonic focusing coefficient is calculated using the following formula: ； In the formula, and This indicates the coordinates of the center point of the predicted bounding box. and This represents the coordinates of the center point of the actual bounding box. This represents the square of the diagonal length of the smallest bounding rectangle between the predicted bounding box and the true bounding box.

5. The method for detecting pears with multiple types of shading in orchards on cloudy days based on the improved YOLOv8n according to claim 1, characterized in that: The labeled pear image dataset under cloudy conditions was constructed through the following steps: Images of the original pear orchard scene were captured under cloudy conditions; The original pear orchard scene image was cropped to obtain image samples with pears as the main target. Apply data augmentation operations to the cropped image samples, the data augmentation operations including at least one of random addition of salt and pepper noise, image sharpening, affine transformation and brightness adjustment; The augmented image samples were labeled, and each pear was labeled as unobstructed, occluded between fruits, or occluded by branches and leaves, based on the actual situation of the pear fruits being occluded. This yielded the labeled pear image dataset under the cloudy weather conditions.

6. The method for detecting multiple types of shaded pears in orchards on cloudy days based on the improved YOLOv8n according to any one of claims 1 to 5, characterized in that: The method also includes an application deployment step: The trained and improved YOLOv8n detection model was deployed to the embedded vision processing unit of the pear fruit picking robot. The embedded vision processing unit acquires and detects images of pears under cloudy conditions in real time, and outputs fruit location information containing labels for different occlusion states. This provides pear perception information, including location and occlusion category, for the robotic arm path planning of the pear picking robot.

Images