Method and system for classifying maturity of chestnut shells of Chinese chestnuts and storage medium

Abstract

The invention relates to the technical field of Chinese chestnut screening, in particular to a Chinese chestnut shell maturity classification method and system and a storage medium. The classification method comprises the steps that a Chinese chestnut image data set in a natural scene is collected, a chestnut shell position frame and maturity grades are marked, and the maturity grades are divided into an immature stage, a half-mature stage of transition from turquoise to yellowish-brown, a yellowish-brown mature stage and an over-mature stage of cracking and deterioration according to chestnut shell color change characteristics; and forcibly marking the target of which the shielding area exceeds 30% as an immature stage, and not marking the long-range target of which the maturity cannot be identified. According to the method, a deep coupling mechanism of the target characteristics and the model architecture is established, a noise transmission path is blocked from a data source, and the tolerance of the model to illumination variation and angle distortion is remarkably enhanced in cooperation with the combined effect of pre-enhancement and online random enhancement.

Description

Technical Field

[0001] The present invention relates to the technical field of chestnut screening, and in particular to a method, system and storage medium for classifying the maturity of chestnut buds. Background Art

[0002] Intelligent detection of chestnut bud maturity in natural scenes faces an essential technical contradiction: although existing general-purpose target detection frameworks (such as the YOLO series) can achieve basic positioning, their core design concepts are deeply disconnected from the intrinsic characteristics of chestnut buds. On the one hand, the unique spherical geometry of the chestnut bud and its continuous color change from green to yellow-brown during the ripening process (especially the color difference ambiguity between the semi-ripe and ripe stages) cause the rectangular box regression loss function (such as CIoU) to produce systematic positioning offsets on clustered and occluded targets, thereby triggering a chain reaction of maturity misjudgments; on the other hand, the standard compression strategy of lightweight models sacrifices channel depth, weakening the ability to capture microscopic color change features. Although the introduction of complex attention mechanisms can improve accuracy, it undermines the real-time constraints of edge computing platforms. More importantly, the insufficient resolution of distant targets and the truncation of occluded target features cause traditional labeling rules to generate noise transmission, further amplifying the generalization error of the model. Summary of the Invention

[0003] The present disclosure proposes a chestnut bud maturity classification method, system and storage medium, aiming to overcome at least one defect in the prior art.

[0004] To achieve the above objectives, the technical solutions disclosed in the present invention are as follows:

[0005] According to one aspect of the present disclosure, a method for classifying chestnut buds by maturity is provided, wherein the steps of the classification method include:

[0006] A dataset of chestnut images in natural scenes was collected, and the chestnut pod location frames and maturity levels were annotated. The maturity levels were divided into unripe, semi-ripe (green to yellow-brown transition), yellow-brown mature, and overripe (cracking and deterioration) based on the color variation characteristics of the chestnut pod. Targets with an obscured area exceeding 30% were forcibly labeled as unripe, and distant targets whose maturity could not be identified were not labeled.

[0007] A combination of pre-enhancement and random enhancement during training is used. Pre-enhancement includes left-right flipping, random 90° rotation, brightness adjustment, and Gaussian noise injection of the original image, increasing the sample size by 6 times. Random enhancement uses Mosaic splicing and HSV color space transformation.

[0008] Using YOLOv8s as the baseline model, the original C2f module is replaced with the C2f_PConv module in the backbone network to reduce the number of parameters. The C2f_PConv module processes only 1 / 4 of the feature channels through a partial convolution strategy. The SA attention module is embedded in the neck network to fuse channel attention and spatial attention. The MPDIoU loss function is used in the head network instead of CIoU.

[0009] Conduct five parallel experiments on the SA attention module: ECA, CBAM, SE, CA, and SA. Conduct five comparative training on the MPDIoU loss function: IoU, GIoU, CIoU, SIoU, and MPDIoU. Select the combination with the highest mAP@0.5 improvement.

[0010] The trained model is quantized and converted to FP16 precision using TensorRT and deployed on the Jetson Orin NX edge computing platform of the chestnut picking robot to achieve real-time inference at more than 25 frames per second.

[0011] Furthermore, the strategy for marking chestnut pod location boxes and maturity levels includes:

[0012] Separate and label the overlapping chestnut pod clusters into individual fruits, and independently label each chestnut pod with its bounding box and maturity grade.

[0013] When the area blocked by branches and leaves is ≥30%, it is marked as immature regardless of the actual maturity state; when the blocking ratio is <30%, the maturity is marked according to the visible features;

[0014] Targets within the field of view of the picking robot and within a distance of 0.5-3 meters from the lens are fully labeled, and distant targets outside this range are only labeled with their location information. A Kappa consistency test is used, and three labelers perform cross-validation on a random 10% sample to ensure that the labeling consistency coefficient is ≥0.85.

[0015] Furthermore, the C2f_PConv module includes:

[0016] The input feature map is divided into 4 equal parts, and only one of them is subjected to the conventional convolution operation;

[0017] The convolution output and the three unprocessed feature maps are spliced ​​along the channel axis;

[0018] After splicing, channel fusion is performed through 1×1 convolution;

[0019] The Bottleneck structure is formed by cascading two PConv_Blocks. Its mathematical expression is:

[0020] Among them F out is the output feature map, Fin is the input feature map, and PConv is the partial convolution kernel.

[0021] Furthermore, the SA attention module implementation process includes:

[0022] Divide the input feature map into 32 groups;

[0023] Each group is split into two parallel branches:

[0024] Channel attention branch: Generate channel weights through global average pooling → fully connected layer → Sigmoid;

[0025] Spatial attention branch: Use group normalization → depthwise separable convolution → sigmoid to generate spatial weights;

[0026] The weighted feature map is rearranged through channels to achieve information interaction between groups, and the final output is expressed as:

[0027] in, is element-wise multiplication, Feature splicing.

[0028] Furthermore, the MPDIoU loss function optimization process includes:

[0029] Calculate the Euclidean distance between the upper left corner of the predicted box and the real box:

[0030]

[0031] Calculate the Euclidean distance of the lower right corner:

[0032]

[0033] Construct a distance-sensitive loss term:

[0034]

[0035] Combined with IoU to form a complete loss function:

[0036]

[0037] Furthermore, the classification method also includes deploying an inference optimization method:

[0038] Compress the number of SA attention layer channels through channel pruning;

[0039] Use mixed precision quantization strategy to convert weights;

[0040] Build a pipeline architecture using a neural network inference engine;

[0041] Enables HDR image synthesis when the ambient light intensity falls below a threshold.

[0042] Furthermore, the classification method also includes a maturity early warning mechanism:

[0043] Generate picking instructions for multiple consecutive frames of over-ripe objects;

[0044] The spatial density of mature chestnut buds was counted to generate a heat map;

[0045] Generate location and timestamp identification codes for detected targets.

[0046] According to another aspect of the present disclosure, a chestnut tuft maturity classification system is provided, for executing the chestnut tuft maturity classification method described above, the classification system comprising:

[0047] Image acquisition module, used to collect chestnut images in visible light and short-wave infrared bands;

[0048] a data processing module for executing the data processing steps of the classification method;

[0049] Edge computing platform for running lightweight object detection models;

[0050] The control execution module drives the picking robot arm based on the detection results;

[0051] Communication module, used to transmit maturity distribution data to the cloud platform.

[0052] Furthermore, the classification system further includes:

[0053] An environmental adaptation unit, used to switch imaging modes according to light intensity;

[0054] Safety obstacle avoidance unit, which calculates the safety distance based on 3D point cloud data;

[0055] Energy management unit, used to dynamically adjust the processor operating frequency.

[0056] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer-readable program is stored. When the computer-readable program is executed by a processor, the method for classifying the maturity of chestnut buds as described above is implemented.

[0057] The beneficial effects of the present invention are:

[0058] This invention achieves three breakthroughs by establishing a deep coupling mechanism between target characteristics and model architecture:

[0059] 1. Through the designed occluded target labeling rules (>30% occlusion is forcibly marked as immature) and the distant target culling strategy, the noise transmission path is blocked at the data source. Combined with the combined effect of pre-enhancement and online random enhancement, the model's tolerance to lighting variation and angular distortion is significantly enhanced.

[0060] 2. The standard convolution structure is replaced by the C2f_PConv module, and the organic balance between parameter compression and occlusion feature retention is achieved through the partial channel convolution strategy. Assisted by the SA attention module, the color-sensitive areas are dynamically enhanced to accurately capture the critical color gradient characteristics from the semi-ripe stage to the mature stage.

[0061] 3. The MPDIoU loss function is used to optimize the distance between the diagonal points of the sphere's circumscribed rectangle, fundamentally correcting the geometric adaptation deviation of the general detection framework. Combined with the experimentally driven optimization strategy of the attention mechanism and loss function, a two-way closed loop is formed to improve positioning accuracy and classification accuracy.

[0062] 4. The TensorRT-based FP16 quantization deployment solution compresses computing density to within the critical threshold of the edge platform, enabling the first unification of high-precision color change recognition and millisecond-level inference speed in chestnut picking scenarios, establishing a universal technical paradigm for the automated harvesting of global agricultural targets.

[0063] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Flowchart of a method for classifying chestnut bud maturity in one embodiment of the present invention;

[0065] Figure 2 This is a visualization diagram of chestnut maturity marking in one embodiment of the present invention;

[0066] Figure 3 Schematic diagram of the PSM-YOLO model structure in one embodiment of the present invention;

[0067] Figure 4 Schematic diagram of the C2f_PConv structure in one embodiment of the present invention;

[0068] Figure 5 Schematic diagram of the SA attention mechanism structure in one embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of comparative analysis of model performance in one embodiment of the present invention;

[0070] Figure 7This is a schematic diagram of the visualization of model inference results in one embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0072] The term "comprise" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products, or apparatuses. In addition, the use of "and / or" in the specification and claims to indicate at least one of the connected objects, such as A and / or B, means that A alone, B alone, and both A and B are included.

[0073] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0074] The present invention provides the following preferred embodiments:

[0075] Example 1

[0076] In order to accurately identify the maturity level of chestnut buds in natural scenes and meet the real-time processing requirements of embedded devices, this embodiment provides a chestnut bud maturity classification method. Figure 1 As shown, the classification method process is as follows:

[0077] S100: Collect chestnut image datasets in natural scenes, annotate the chestnut head position frame and maturity level. The maturity level is divided into unripe stage, semi-ripe stage with a transition from green to yellow-brown, yellow-brown mature stage, and over-ripe stage with cracking and deterioration based on the color change characteristics of the chestnut head. Targets with an occlusion area exceeding 30% are forcibly labeled as unripe. Long-range targets whose maturity cannot be identified are not labeled.

[0078] S200: It combines pre-enhancement with random enhancement during training. Pre-enhancement includes left-right flipping, random 90° rotation, brightness adjustment, and Gaussian noise injection of the original image, and amplifies the sample size by 6 times. Random enhancement uses Mosaic splicing and HSV color space transformation.

[0079] S300: Using YOLOv8s as the baseline model, the C2f_PConv module is used to replace the original C2f module in the backbone network to reduce the number of parameters. The C2f_PConv module only processes 1 / 4 of the feature channels through a partial convolution strategy. The SA attention module is embedded in the neck network to fuse channel attention and spatial attention. The MPDIoU loss function is used instead of CIoU in the head network.

[0080] S400: Conduct five parallel experiments on the SA attention module: ECA, CBAM, SE, CA, and SA. Conduct five comparative training on the MPDIoU loss function: IoU, GIoU, CIoU, SIoU, and MPDIoU. Select the combination with the highest mAP@0.5 improvement.

[0081] S500: The trained model is quantized and converted to FP16 precision using TensorRT and deployed on the Jetson Orin NX edge computing platform of the chestnut picking robot to achieve real-time inference at more than 25 frames per second.

[0082] Specifically, we first collected chestnut image datasets using an industrial camera under natural lighting conditions (500-1200 lux illumination range). Figure 2 The annotation specifications shown in the figure perform structured annotation on the captured images: Based on the continuous color variation of chestnut thorns, maturity is divided into unripe (green thorn coverage >80%), semi-ripe (green to yellow-brown transition), mature (yellow-brown coverage >85%), and overripe (dark brown husk with shrunken husk). Objects with more than 30% obstruction by branches and leaves are forcibly labeled as unripe. Furthermore, blurred objects in the field of view beyond 3 meters are not labeled to ensure the validity of the training data.

[0083] Furthermore, a two-stage strategy is employed in the data augmentation phase to improve the model's generalization capabilities. In the pre-enhancement phase, the original image is horizontally flipped, randomly rotated 90°, brightness adjusted, and Gaussian noise injected, generating a six-fold increase in sample size. During training, dynamic enhancement uses Mosaic multi-image stitching technology to simulate the clustering of chestnut leaves. Combined with HSV color space transformation, the algorithm focuses on optimizing the gradient from cyan (H∈[80°,120°]) to tan (H∈[20°,60°]). This transformation applies a ±15° perturbation to the hue channel and ±20% and ±30% nonlinear adjustments to the saturation and lightness channels, respectively.

[0084] Furthermore, if Figure 3 The model architecture shown is triple-optimized using YOLOv8s as the baseline:

[0085] like Figure 4 As shown in Figure 1, the C2f module in the backbone network is replaced by the C2f_PConv structure, which processes only 1 / 4 of the input channels through the partial convolution strategy, significantly reducing computational redundancy.

[0086] like Figure 5 As shown in the figure, the neck network is embedded in the SA attention module, which integrates the dual-path feature optimization mechanism of channel attention and spatial attention to enhance the feature response of the critical area of ​​color change.

[0087] The head network uses the MPDIoU loss function instead of the standard CIoU, and its regression loss term is defined as:

[0088] Where d1 and d2 represent the Euclidean distances between the top-left and bottom-right corners of the predicted and ground-truth boxes, respectively, and w and h are the width and height of the minimum bounding rectangle of the two boxes. This design optimizes corner point location accuracy based on the spherical geometry of the Li Peng tree.

[0089] Furthermore, to verify the effectiveness of the core components, five parallel experiments were conducted on the attention module: ECA, CBAM, SE, CA, and SA, and five comparative trainings were conducted on the loss function: IoU, GIoU, CIoU, SIoU, and MPDIoU. Figure 6 The performance analysis results shown in the figure finally select the combination of SA attention module and MPDIoU loss function.

[0090] The benefit of this embodiment is that the model deployment phase uses the TensorRT engine to perform FP16 mixed precision quantization, builds an asynchronous processing pipeline on the Jetson Orin NX edge computing platform, and realizes parallel processing of image acquisition, model inference, and result output. It should be noted that this deployment solution stabilizes the single-frame processing delay within 38 milliseconds, meeting the real-time operation requirements of more than 25 frames per second. Figure 7 The on-site recognition results shown verify the adaptability of the grading model in complex scenarios, especially its significant advantages in determining the maturity of overlapping chestnut pods.

[0091] Example 2

[0092] This paper provides a high-precision, lightweight method for chestnut tuft detection (including precise tuft location and maturity classification) in natural scenes using computer vision technology, achieving the goal of lightweight deployment in automated chestnut harvesting equipment. The method includes the following four steps:

[0093] Step 1: Analyze the phenotypic characteristics of chestnut pods and chestnut maturity classification, design image acquisition methods, analyze chestnut image features, and perform data annotation and data enhancement methods, as well as data set division.

[0094] Step 2: Based on the chestnut image features and model design goals, an innovative model design is carried out. Taking the lightweight model YOLOv8s as the basic model, this paper proposes the chestnut and chestnut bud detection model PSM-YOLO. The C2f module in the backbone is improved to the C2f_PConv module, the SA (shuffle attention) mechanism is embedded in the neck part, and the bounding box regression loss function MPDIOU suitable for circular object detection is used instead of CIOU.

[0095] Step 3: Design a model training platform, select model evaluation indicators, design data enhancement strategies and hyperparameters during training, conduct comparative experiments on different attention mechanisms, conduct comparative experiments on different bounding box regression loss functions, conduct ablation experiments on each improved module, and finally determine the model architecture that is most suitable for chestnut bud detection.

[0096] Step 4: Verify the superiority of the model of the present invention over the classic target detection model in chestnut clump target detection performance, and apply it to lightweight deployment on the chestnut picking robot in a natural chestnut orchard environment to achieve high-precision detection.

[0097] Specifically, the phenotypic characteristics of the chestnut pod were analyzed. Chestnut seeds are enclosed in a hard pod and often hang on the tree's branches. The pod is spherical and covered with sharp thorns. When the seeds mature, the pod splits open in a cross-shaped pattern from the abdomen. The pod's color matches the leaves, gradually transitioning from turquoise to yellowish-brown during maturity. The pods can grow singly or in clusters, often obscured by branches and leaves.

[0098] Furthermore, chestnut maturity is graded. Based on the formation of the chestnut fruit's nutrients, taste, and storage conditions, chestnut maturity is divided into four levels: unripe, semi-ripe, mature, and overripe. The maturity levels are primarily based on the phenotypic characteristics of the chestnut pod. Chestnuts are harvested at maturity, when they have the best taste and quality. The maturity and morphological characteristics of chestnuts are shown in Table 1.

[0099] Table 1 Maturity and phenotypic characteristics of chestnut

[0100]

[0101] Furthermore, chestnut images were collected. These chestnut images were collected from Dahongpao chestnuts grown in the hilly Dabie Mountains. All samples were collected from chestnut plantations in natural settings in the autumn of 2024. Mobile phones were used to capture images at different stages of maturity, in varying weather conditions, and at different times of day. After data cleaning, a total of 1,090 valid images were obtained. Typical characteristics of chestnut images in natural settings include complex lighting and backgrounds, overlapping clusters of chestnut pods, obstructions from branches and leaves, and numerous small objects in long-range shots.

[0102] Furthermore, chestnut image feature analysis was performed. Chestnut puff target detection and maturity classification, as well as data preprocessing and labeling strategies, all require designing strategies based on the image features of chestnut samples. The following data characteristics are analyzed: ① The complex background of the natural scene in the chestnut orchard poses challenges to target detection due to varying weather conditions and shooting angles. Using a harvesting robot for automated harvesting, multi-angle detection is possible. ② Detecting single and multiple flat chestnuts is relatively simple, while detecting clustered or obscured objects is more difficult. ③ Detecting the maturity of chestnuts from a distance is difficult, as harvesting robots focus more on close-up targets. How can we label the maturity of distant targets? ④ Chestnut maturity classification is challenging, especially since the difference between semi-ripe and ripe chestnuts is unclear, leading to classification errors.

[0103] Furthermore, a data annotation strategy was developed. Based on image feature analysis, a data annotation method for this dataset was designed. Data annotation included the location and label annotation of chestnut targets. Labelme image annotation software was used to annotate sample images. The annotation strategy was as follows: ① Single chestnuts were annotated with location boxes and maturity levels, and clusters of multiple fruits were annotated individually. ② For single fruits obscured by branches and leaves or by fruit, if the obscured area was too large to identify the maturity level, they were labeled as immature. ③ For small targets at a distance, if the maturity level could not be identified, they were not annotated. For automated picking robots, the closest targets were always the focus.

[0104] Furthermore, we implemented a pre-data augmentation strategy. Since the number of chestnut samples in this paper is relatively small, we employed pre-data augmentation during dataset creation and random data augmentation during model training to augment the data. Pre-data augmentation involves flipping each chapter image left and right, performing random 90-degree positive and negative rotations, performing random translations and scaling, adding noise, and varying the brightness. This pre-data augmentation increased the sample size sixfold, yielding a total of 6,540 labeled chestnut sample images.

[0105] Furthermore, the dataset was divided into three parts: training set, validation set, and test set, with the ratio of 70%, 15%, and 15%, respectively, forming the dataset of this paper.

[0106] Furthermore, the model design goal is to develop a high-precision lightweight target detection model for chestnut picking robot vision system, which is suitable for edge deployment and can perform chestnut pod target positioning and maturity detection.

[0107] Model design is carried out based on the chestnut image characteristics (Step1(4)) and model design goals (Step2(1)).

[0108] The lightweight model YOLOv8s is selected as the basic model, and the chestnut target detection model PSM-YOLO (YOLOv8s+C2f_PConv+SA+MPDIOU) in this paper is proposed.

[0109] By improving the C2f module in the backbone to the C2f_PConv module, the model parameters are reduced and the model is lightweight; the SA (shuffle attention) attention mechanism is embedded in the Neck part to fuse channel and spatial attention, thereby improving the positioning accuracy and classification accuracy of the model; and the bounding box regression loss function MPDIOU suitable for circular target detection is used instead of CIOU to improve the model's positioning accuracy for the target and further enhance the detection effect. The improved model structure is shown in the figure below. Figure 3 Figure captions: ① The model structure includes the backbone network (backbone), neck network (neck), and head network (head); ② The light green module is an innovative design module of the present invention, using C2f_Pconv to replace the C2f module in the raw module; ③ The neck module SPPCSPC module replaces the SPPF module of YOLOv8s; ④ The SA attention module is embedded in the head network.

[0110] Further, Figure 4 The C2f_PConv module structure in the

[15] is designed to achieve a lightweight model. The partial convolution module PConv_Block is a lightweight convolution design that leverages redundant information in feature maps to selectively convolve some channels of the feature map, while leaving the other channels unchanged. Finally, the convolved and unconvolved feature maps are concatenated as the output. PConv improves efficiency by reducing redundancy and memory accesses while maintaining feature expressiveness. The C2f_PConv module is implemented by replacing the conventional convolution in the C2f module with PConv.

[0111] Figure 4The bottleneck module in (3) is the Bottleneck module. The core component of the Bottleneck module is PConv_Block. PConv is combined with Pointwise Conv. PConv accepts the feature map X∈R h×w×c , 1 / 4 of it is subjected to conventional convolution to extract features, and its output is concat-operated with the other 3 / 4 input part. Next, the inter-channel fusion of features is performed through Conv 1×1 pointwise conv, and finally an efficient convolution module PConv_Block is constructed through BatchNorm and SiLU operations.

[0112] Figure 4 In (2), the bottleneck module is constructed by cascading two PConv_Blocks and performing residual connection blocks. The bottleneck module is the core module for achieving lightweight model.

[0113] Figure 4 (1) is the C2f_PConv module, which reduces redundant computations, optimizes gradient propagation, and performs cross-stage feature fusion to improve feature representation by splitting, partially processing, and concatenating feature channels.

[0114] Furthermore, the Shuffle Attention (SA) attention mechanism realizes parallel processing by grouping feature maps and then aggregating them, and integrates channel attention and spatial attention. The structure of the Shuffle Attention module is as follows: Figure 5 shown.

[0115] Furthermore, the principle of SA attention mechanism. Shuffle Attention takes the input feature map X∈R h×w×c The algorithm is divided into g groups, and each group is split into two parallel sub-features branches by channel splitting. The channel attention branch is weighted to the feature map of the corresponding channel by global average pooling, and the weights and biases are adjusted by learning. Finally, after the Sigmoid activation function is passed, the obtained channel attention is weighted to the feature map of the corresponding channel. The spatial attention branch is calculated by group normalization (GroupNorm). After learning adjustment and Sigmoid function activation, the obtained spatial attention is weighted to the spatial dimension of the feature map. Finally, the weighted feature maps of the two branches are concatenated. After aggregating the features of each group, the feature maps are rearranged by the Channel Shuffle operation to realize information exchange between different feature channels.

[0116] By integrating Shuffle Attention into the head network of our model, we can achieve accurate positioning of chestnut targets and high-precision recognition of maturity levels while keeping the model lightweight.

[0117] Furthermore, in chestnut target detection, based on the spherical shape of the chestnut puff, in order to better evaluate the regression loss, this paper uses MPDIoU (Minimum Point Distance based IoU) to construct the bounding box regression loss function instead of the CIOU loss function of YOLOv8s.

[0118] The MPDIoU loss function is suitable for chestnut object detection. Chestnut hulls are spiny and nearly spherical, making them prone to clustering and occlusion. MPDIoU directly optimizes the distance between two pairs of diagonal points, which is more suitable for the chestnut hull's shape and provides more stable and accurate detection of clustered and occluded objects.

[0119] Furthermore, in order to verify the effect of the improved model in this paper, the model PSM-YOLO in this paper is compared with the classic target detection models Faster-RCNN, YOLOv5s, YOLOv8s, YOLOv8n, and YOLOv10s, and the average precision mean, number of parameters, and floating-point operations are selected as evaluation indicators.

[0120] The experimental results are shown in Table 2. As can be seen from the table, the classic visual detection model, Faster-RCNN, has a large number of parameters and floating-point operations, resulting in the worst detection performance. YOLOv8n, the nano version of YOLOv8, has the smallest number of parameters and floating-point operations, but its detection accuracy is lower than that of YOLOv5s. The PSM-YOLO model has a parameter and floating-point operation count close to that of YOLOv10s, and its detection accuracy (mAP@0.5) has increased by 1.4 percentage points, and its mAP@0.5:0.95 has increased by 0.6 percentage points. The PSM-YOLO model has 36% and 28% fewer parameters and floating-point operations than YOLOv8s, respectively, but its detection accuracy (mAP@0.5:0.95) has increased by 4.6 percentage points, and its mAP@0.5:0.95 has increased by 3.5 percentage points. The comparative results show that this improved model meets the high-precision and lightweight requirements for chestnut bud detection.

[0121] Table 2 Comparison of different models with PSM-YOLO

[0122]

[0123] Model comparison tests show that the PSM-YOLO model of the present invention has the best indicators in chestnut bud target detection, achieving high-precision and lightweight chestnut bud target positioning and maturity detection, and can be deployed on the edge in chestnut picking robots and automation systems.

[0124] The benefit of this embodiment is that the model of the present invention adopts the YOLOv8s model pre-trained on the large-scale dataset COCO, and is trained and fine-tuned on the chestnut bud detection dataset in this article through transfer learning.

[0125] This model improves upon the classic YOLOv8s model, innovating its design based on chestnut image characteristics and model design objectives. This results in the proposed chestnut target detection model, PSM-YOLO (YOLOv8s+C2f_PConv+SA+MPDIOU). This model reduces model parameters and achieves model lightweighting by upgrading the C2f module in the backbone to a C2f_PConv module. The SA (shuffle attention) mechanism is embedded in the neck region, fusing channel and spatial attention to improve the model's positioning and classification accuracy. Finally, the MPDIOU loss function, suitable for circular target detection, is adopted instead of the CIOU loss function, improving the model's positioning accuracy and further enhancing detection results.

[0126] The model of the present invention adopts transfer learning and data enhancement strategies for a small sample data set of chestnut and Ligusticum mollissima target detection. The data enhancement strategy adopts data enhancement in the data preprocessing stage and random enhancement strategy in training.

[0127] The present invention analyzes the phenotypic characteristics (Step 1(1)) and image characteristics (Step 1(4)) of the chestnut tuft in detail. Each model designed is developed based on the characteristics. The SA attention mechanism is used to solve the problem that the background of the natural scene of the chestnut orchard is complex, the maturity classification of chestnuts is difficult, and the difference between semi-ripe and ripe chestnuts is not obvious. Since the chestnut tuft is a thorny, nearly spherical target that is easily superimposed and occluded, MPDIOU is selected to directly optimize the distance between two diagonal points, which is more suitable for the appearance characteristics of the chestnut tuft and more stable and accurate for superimposed and occluded targets.

[0128] The chestnut bud target detection model of the present invention is designed to be suitable for edge-side deployment on a chestnut picking robot. A lightweight design is adopted to ensure the real-time performance of the model detection, while improving the model detection accuracy as much as possible while meeting the real-time performance.

[0129] The model experiments and model design were carried out alternately. Through model design, experimental verification, redesign, and further experimental verification, a high-precision, lightweight model optimal for chestnut bud detection was ultimately constructed. Five different attention modules were compared in this model, and five different bounding box regression loss functions were compared in this model. Ablation experiments were also conducted on each of the innovatively designed modules.

[0130] Although the present invention has been described above in detail with reference to its preferred embodiments, it is to be understood that the present invention is not limited to the embodiments described above. Instead, various modifications and changes may be made by those skilled in the art without departing from the spirit of the invention, and these modifications and changes should fall within the scope defined by the appended claims and their equivalents.

Claims

1. A method for classifying chestnut bud maturity, characterized in that: The steps of the classification method include: A dataset of chestnut images in natural scenes was collected, and the chestnut pod location frames and maturity levels were annotated. The maturity levels were divided into unripe, semi-ripe (green to yellow-brown transition), yellow-brown mature, and overripe (cracking and deterioration) based on the color variation characteristics of the chestnut pod. Targets with an obscured area exceeding 30% were forcibly labeled as unripe, and distant targets whose maturity could not be identified were not labeled. A combination of pre-enhancement and random enhancement during training is used. Pre-enhancement includes left-right flipping, random 90° rotation, brightness adjustment, and Gaussian noise injection of the original image, increasing the sample size by 6 times. Random enhancement uses Mosaic splicing and HSV color space transformation. Using YOLOv8s as the baseline model, the original C2f module is replaced with the C2f_PConv module in the backbone network to reduce the number of parameters. The C2f_PConv module processes only 1 / 4 of the feature channels through a partial convolution strategy. The SA attention module is embedded in the neck network to fuse channel attention and spatial attention. The MPDIoU loss function is used in the head network instead of CIoU. Conduct five parallel experiments on the SA attention module: ECA, CBAM, SE, CA, and SA. Conduct five comparative training on the MPDIoU loss function: IoU, GIoU, CIoU, SIoU, and MPDIoU. Select the combination with the highest mAP@0.5 improvement. The trained model is quantized and converted to FP16 precision using TensorRT and deployed on the Jetson Orin NX edge computing platform of the chestnut picking robot to achieve real-time inference at more than 25 frames per second.

2. The method for classifying chestnut bud maturity as claimed in claim 1, wherein: The strategy for marking chestnut bud position boxes and maturity levels includes: Separate and label the overlapping chestnut pod clusters into individual fruits, and independently label each chestnut pod with its bounding box and maturity grade. When the area blocked by branches and leaves is ≥30%, it is marked as immature regardless of the actual maturity state; when the blocking ratio is <30%, the maturity is marked according to the visible features; Targets within the field of view of the picking robot and within a distance of 0.5-3 meters from the lens are fully labeled, and distant targets outside this range are only labeled with their location information. A Kappa consistency test is used, and three labelers perform cross-validation on a random 10% sample to ensure that the labeling consistency coefficient is ≥0.

85.

3. The method for classifying chestnut bud maturity according to claim 1, wherein: The C2f_PConv module includes: The input feature map is divided into 4 equal parts, and only one of them is subjected to the conventional convolution operation; The convolution output and the three unprocessed feature maps are spliced ​​along the channel axis; After splicing, channel fusion is performed through 1×1 convolution; The Bottleneck structure is formed by cascading two PConv_Blocks. Its mathematical expression is: Among them F out is the output feature map, F in is the input feature map, and PConv is the partial convolution kernel.

4. The method for classifying chestnut bud maturity according to claim 1, wherein: The SA attention module implementation process includes: Divide the input feature map into 32 groups; Each group is split into two parallel branches: Channel attention branch: Generate channel weights through global average pooling → fully connected layer → Sigmoid; Spatial attention branch: Use group normalization → depthwise separable convolution → sigmoid to generate spatial weights; The weighted feature map is rearranged through channels to achieve information interaction between groups, and the final output is expressed as: in, is element-wise multiplication, and ⊕ is feature concatenation.

5. The method for classifying chestnut bud maturity according to claim 1, wherein: The MPDIoU loss function optimization process includes: Calculate the Euclidean distance between the upper left corner of the predicted box and the real box: Calculate the Euclidean distance of the lower right corner: Construct a distance-sensitive loss term: Combined with IoU to form a complete loss function:

6. The method for classifying chestnut bud maturity according to claim 1, wherein: The classification method also includes deploying an inference optimization method: Compress the number of SA attention layer channels through channel pruning; Use mixed precision quantization strategy to convert weights; Build a pipeline architecture using a neural network inference engine; Enables HDR image synthesis when the ambient light intensity falls below a threshold.

7. The method for classifying chestnut bud maturity according to claim 1, wherein: The classification method also includes a maturity early warning mechanism: Generate picking instructions for multiple consecutive frames of over-ripe objects; The spatial density of mature chestnut buds was counted to generate a heat map; Generate location and timestamp identification codes for detected targets.

8. A chestnut bud maturity classification system for executing the chestnut bud maturity classification method according to any one of claims 1 to 7, characterized in that: The classification system includes: Image acquisition module, used to collect chestnut images in visible light and short-wave infrared bands; a data processing module for executing the data processing steps of the classification method; Edge computing platform for running lightweight object detection models; The control execution module drives the picking robot arm based on the detection results; Communication module, used to transmit maturity distribution data to the cloud platform.

9. The chestnut bud maturity classification system according to claim 8, characterized in that: The classification system also includes: An environmental adaptation unit, used to switch imaging modes according to light intensity; Safety obstacle avoidance unit, which calculates the safety distance based on 3D point cloud data; Energy management unit, used to dynamically adjust the processor operating frequency.

10. A computer-readable storage medium having a computer-readable program stored thereon, characterized in that: When the readable program is executed by a processor, the chestnut bud maturity classification method as described in any one of claims 1 to 7 is implemented.

Images