Fruit classification method based on improved YOLOv8n network
By improving the YOLOv8n network and combining data augmentation and feature reconstruction, the problem of insufficient detection accuracy of fruit recognition technology in complex environments was solved, achieving lightweight and efficient fruit classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-04-03
Smart Images
Figure CN121789207A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of interdisciplinary technology of agricultural intelligent equipment and computer vision, specifically a fruit classification method based on an improved YOLOv8n network. Background Technology
[0002] With the development of intelligent agriculture, automated harvesting and intelligent sorting technologies have become core requirements for the upgrading of the fruit industry, and accurate fruit detection and classification are key prerequisites for realizing these technologies. Existing fruit recognition technologies are mainly divided into two categories: traditional image processing methods and deep learning-driven object detection methods. Traditional image processing methods rely on manually designed features such as color, texture, and shape. They are not robust to changes in lighting conditions, complex backgrounds with intertwined branches and leaves, and scenes where fruit obscures the view, making it difficult to adapt to dynamic and ever-changing harvesting environments.
[0003] Deep learning object detection methods are divided into two-stage and single-stage algorithms: Two-stage algorithms have high detection accuracy, but high computational complexity and slow detection speed, which cannot meet the real-time recognition requirements of picking equipment; Single-stage algorithms balance speed and accuracy through an end-to-end structure, but existing single-stage models still have problems with insufficient detection accuracy and poor robustness when dealing with small target fruits, fruit occlusion, multi-scale changes and complex background interference in picking scenarios, and some models have insufficient lightweight design, making it difficult to adapt to the hardware computing power limitations of "multiple fruits on one machine" end effectors.
[0004] Therefore, there is an urgent need for a fruit classification and identification technology that combines lightweight design, high detection accuracy, and strong robustness to solve the core visual recognition problem in the "multiple fruits in one machine" harvesting process.
[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] The purpose of this invention is to provide a fruit classification method based on an improved YOLOv8n network to solve the problems mentioned in the background art.
[0007] To achieve the above objectives, the present invention provides the following technical solution: A fruit classification method based on an improved YOLOv8n network, comprising the following steps: Step 1: Collect RGB images containing various types of fruits, preprocess the RGB images, and divide the preprocessed RGB images into training and validation sets in an 8:2 ratio; Step 2: Construct an improved YOLOv8n model, which includes a backbone network, a neck network, and a detection head. Based on the basic YOLOv8n model, an SPD-Conv module is inserted after each C2f module, the PAN structure of the neck network is replaced with a Slim-Neck based on GSConv and VoV-GSCSP, and the Detect-SEAM module is integrated before the classification and regression branches of the detection head. Step 3: Input the training set RGB images into the improved YOLOv8n model, obtain the corresponding multi-scale enhanced feature maps in the Detect-SEAM module, process the multi-scale enhanced feature maps of the training set RGB images in the detection head to determine their fruit category, bounding box center point coordinates and bounding box size, and validate them with the validation set RGB images to generate a classification prediction model; Step 4: Input the RGB image to be predicted into the classification prediction model to obtain the fruit category and its confidence score, bounding box center point coordinates and bounding box size corresponding to the RGB image to be predicted.
[0008] Furthermore, the preprocessing steps for RGB images specifically include: scaling, bounding box annotation, and data augmentation. Scaling refers to resizing the RGB images to 640×640 pixels. Bounding box annotation involves using the Labellmg tool to annotate the bounding boxes of the fruits in each RGB image. After annotation, 10% of the images are randomly selected for verification until the average annotation error is no higher than 2.4%. Data augmentation involves sequentially performing the following steps on the annotated images: Random rotation, scaling and translation, horizontal and vertical flipping, and lighting parameter adjustment.
[0009] Furthermore, the specific operation steps of the SPD-Conv module are as follows: The input image is spatially partitioned by dividing it into two non-overlapping regions along both the height and width directions, resulting in four sub-regions, each with a size of [size missing]. ,in, Indicates the height of the image, Indicates the width of the image. This represents the number of channels in the image, and ; All sub-regions are stitched together along the channel dimension to generate an intermediate feature map, the size of which is [size missing]. ; The kernel size is set to 1×1, the stride to 1, and the number of output channels are set. A convolution operation is performed on the intermediate feature maps to output a backbone feature map. The size of the backbone feature map is... ,in, Indicates the number of output channels; The feature maps of three depth levels in the backbone network are selected for output. The three depth level feature maps include shallow feature maps, mid-level feature maps, and deep feature maps. The shallow feature maps represent the feature maps extracted before processing by the SPD-Conv module, the mid-level feature maps represent the backbone feature maps, and the deep feature maps represent the feature maps that are processed by the SPD-Conv module and then further processed in the backbone network.
[0010] Furthermore, the principle of replacing the PAN structure with a Slim-Neck based on GSConv and VoV-GSCSP is as follows: The original PAN structure in the neck network of the basic YOLOv8n model was completely replaced with a Slim-Neck structure based on GSConv and VoV-GSCSP. The specific processing procedure of GSConv for each input feature map is as follows: the convolution kernel size is set to 3×3, the convolution stride is set to 2, the input feature map is downsampled, the downsampled feature map is input into the depthwise separable convolutional layer, and channel-wise convolution and point-wise convolution are performed respectively. Channel-wise convolution means that each channel is convolutionally operated independently, and point-wise convolution means that channels are fused through a convolution kernel of size 1×1. Then, channel shuffling is performed, which means that the feature information of different channels is rearranged and combined. VoV-GSCSP processes each input feature map after GSCV processing as follows: It adopts a dual-branch design. One branch uses a 1×1 convolution kernel with a stride of 1 for standard convolution. The other branch first passes through a Ghost convolution, then performs channel shuffling on the Ghost convolution output, and then passes it through convolution kernels of sizes 1×1, 3×3, and 1×1 in sequence to generate a lightweight bottleneck structure. The outputs of the two branches are concatenated in the channel dimension to form the concatenated input feature map, which includes a shallow feature map, a middle feature map, and a deep feature map. The input feature map includes shallow feature map, medium feature map and deep feature map.
[0011] Furthermore, the specific principle of integrating the Detect-SEAM module is as follows: The Detect-SEAM module is embedded as a plugin before the classification and regression branches, and the shallow, mid-level, and deep feature maps after concatenation are processed in parallel by the Detect-SEAM module. The specific process of processing the concatenated shallow feature map using the Detect-SEAM module is as follows: Each channel in the concatenated shallow feature map is independently spatially convolved using depthwise separable convolution, followed by pointwise convolution to fuse the channels and generate an intermediate feature map. For the intermediate feature map, global average pooling is performed to generate channel description vectors, with the channel description vectors having a size of [value missing]. , This represents the number of channels in the channel description vector. This channel description vector is input into a two-layer fully connected network, where the first fully connected layer compresses the number of channels to... , This indicates the reduction ratio, and the number of channels is restored to the original value through the second fully connected layer. The output of the two fully connected layers is converted using the sigmoid function. The dimensional vector is normalized to generate the attention weight vector for each channel. Each channel in the concatenated shallow feature map is multiplied by the corresponding attention weight vector to output the shallow enhanced feature map. Similarly, the concatenated middle and deep feature maps are processed in the same way to generate the middle and deep enhanced feature maps, respectively. The shallow, middle, and deep enhanced feature maps are then combined to generate a multi-scale enhanced feature map.
[0012] Furthermore, the principle underlying the generation of the classification prediction model is as follows: In the detection head, for the multi-scale enhanced feature maps of the RGB images in the training set, the bounding box coordinates, bounding box size, and fruit category are predicted. The AdamW optimizer is used, and the learning rate is set to [value missing]. The training run consisted of 200 rounds with a batch size of 16. The bounding box loss function was Clou Loss, and the class loss function was binary cross-entropy. After each training round, the MAP was calculated using the validation set RGB images. When the MAP stopped improving after ten consecutive training rounds, training was stopped, and the model with the highest MAP corresponding to the validation set RGB images was saved as the classification prediction model.
[0013] Compared with the prior art, the beneficial effects of the present invention are: This invention employs data augmentation strategies in the data preprocessing stage, combined with a bounding box annotation verification and optimization process, effectively expanding the diversity and quality of training data and improving the model's generalization ability from the source. Furthermore, based on the basic YOLOv8n model, an SPD-Conv module is inserted after each C2f module to spatially partition and reconstruct the input feature map, achieving downsampling without information loss. This effectively enhances the model's perception and feature preservation capabilities for small-scale fruit targets, significantly reducing the problem of feature information attenuation for small targets in deep networks. Finally, the original PAN structure of the neck network is replaced with a GSCO-based structure. The Slim-Neck structure of YOLOv8n and VoV-GSCSP significantly reduces the number of parameters and computational cost while maintaining multi-scale feature fusion performance through lightweight operations such as depthwise separable convolution and Ghost convolution. This allows the model to more efficiently fuse features from shallow, mid, and deep layers, improving its ability to distinguish occluded and overlapping fruits. A Detect-SEAM module is integrated before the detection head, dynamically strengthening key feature channels for fruit classification and localization through channel attention, suppressing irrelevant background interference, and maintaining high class confidence and localization accuracy even in complex backgrounds. The improved YOLOv8n model has only a slight increase in the number of parameters and computational complexity compared to the original model, but significantly improves both real-time detection and detection accuracy. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the method flow of an embodiment of the present invention. Detailed Implementation
[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.
[0016] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0017] Example: Please see Figure 1 The present invention provides a technical solution: A fruit classification method based on an improved YOLOv8n network, comprising the following steps: Step 1: Collect RGB images containing various types of fruits, preprocess the RGB images, and divide the preprocessed RGB images into training and validation sets in an 8:2 ratio; In this embodiment, the preprocessing steps for RGB images specifically include: scaling, bounding box annotation, and data augmentation. Scaling refers to scaling the RGB image to 640×640 pixels; bounding box annotation involves using the Labellmg tool to annotate the bounding boxes of the fruits in each RGB image, and after annotation, randomly selecting 10% of the images for verification until the average annotation error is no higher than 2.4%; data augmentation involves sequentially performing the following steps on the annotated images: Random rotation, scaling and translation, horizontal and vertical flipping, and lighting parameter adjustment.
[0018] Images of citrus fruits from Susong County in Anqing, apples and pears from Dangshan County in Suzhou, and pomegranates from Huaiyuan County in Bengbu were collected outdoors under natural light. The images covered different time periods, weather conditions, and shooting angles to ensure sample diversity. Time periods included morning, noon, and evening; weather conditions included sunny and cloudy; and shooting angles included frontal, side, and overhead shots. The fruit images were scaled to 640×640 pixels and uniformly converted to the RGB color space. The Labellmg tool was used to label the fruit categories, assigning 0, 1, 2, and 3 to pomegranates, apples, pears, and citrus fruits, respectively. Ten percent of the images were randomly selected for verification to ensure an average labeling error of ≤2.4%. Scaling the image to 640×640 maintains resolution while being relatively lightweight, making it suitable for real-time detection tasks. Bounding box representation aims to label the location and category of each fruit in the image. The specific steps are as follows: Collect RGB images containing multiple types of fruit; manually annotate using Labellmg; draw a bounding box for each visible fruit in the image, ensuring the bounding box closely matches the fruit's edge to avoid including too much background or missing fruits; determine the fruit category, assigning a unique code to each fruit category; and label each bounding box with the corresponding fruit category. After annotation, randomly select 10% of the images for verification. Verification includes checking whether the bounding boxes accurately cover the fruits, whether any fruits are missing or unlabeled, and whether the category labels are correct. Repeat the verification process until the average annotation error is no higher than 2.4%. The principle for calculating the average annotation error is as follows: calculate the intersection-union ratio (CIRR) of each bounding box. The CIRR represents the ratio of the intersection of the annotation box and the ground truth box to the union of the annotation box and the ground truth box. The ground truth box represents the average value of the annotations by experts. Subtract the CIRR from 1 to obtain the annotation error of each annotation box. Take the average of the annotation errors of all annotation boxes to obtain the average annotation error of all annotation boxes.
[0019] Step 2: Construct an improved YOLOv8n model, which includes a backbone network, a neck network, and a detection head. Based on the basic YOLOv8n model, an SPD-Conv module is inserted after each C2f module, the PAN structure of the neck network is replaced with a Slim-Neck based on GSConv and VoV-GSCSP, and the Detect-SEAM module is integrated before the classification and regression branches of the detection head. In this embodiment, the specific operation steps of the SPD-Conv module are as follows: The input image is spatially partitioned by dividing it into two non-overlapping regions along both the height and width directions, resulting in four sub-regions, each with a size of [size missing]. ,in, Indicates the height of the image, Indicates the width of the image. This represents the number of channels in the image, and ; Along the vertical direction, it is divided into an upper part and a lower part, each part being [height missing]. It is divided into a left half and a right half along the width direction, and the width of each half is [missing information]. The final result is four heights. Width is The sub-region.
[0020] All sub-regions are stitched together along the channel dimension to generate an intermediate feature map, the size of which is [size missing]. ; By stitching the four sub-regions along the channel dimension, the spatial size is halved, achieving downsampling. The number of channels is increased to four times the original number, preserving all the original pixel information and only rearranging them, thus avoiding information loss caused by interpolation or pooling in traditional downsampling.
[0021] The kernel size is set to 1×1, the stride to 1, and the number of output channels are set. A convolution operation is performed on the intermediate feature maps to output a backbone feature map. The size of the backbone feature map is... ,in, Indicates the number of output channels; The feature maps of three depth levels in the backbone network are selected for output. The three depth level feature maps include shallow feature maps, mid-level feature maps, and deep feature maps. The shallow feature maps represent the feature maps extracted before processing by the SPD-Conv module, the mid-level feature maps represent the backbone feature maps, and the deep feature maps represent the feature maps that are processed by the SPD-Conv module and then further processed in the backbone network.
[0022] Shallow feature maps come from the feature extraction layer before the SPD-Conv module in the backbone network. They do not undergo spatial downsampling and channel transformation operations of the SPD-Conv module, and retain the most original output state before entering the SPD-Conv module. They come from the earlier part of the network, have fewer downsampling times, and retain the fine details and high spatial resolution of the image. They contain basic visual information such as color, gradient and texture, and are used to detect small fruits and accurately locate bounding boxes. The mid-layer feature map is the direct output of the SPD-Conv module. It is the product of the input feature map after a series of processes including spatial partitioning, channel concatenation, and 1×1 convolution. It has undergone downsampling without information loss and is the result of a structured transformation and compression of the shallow feature map. The spatial resolution is halved compared to the shallow feature map, but the channel information is richer, extracting features while retaining certain spatial details. Deep feature maps are obtained by further processing mid-level feature maps through convolutional layers and C2f modules in the backbone network. These mid-level feature maps are then fed into the next stage of the backbone network, including subsequent convolutional layers and C2f modules, where further feature extraction, nonlinear transformations, and downsampling are performed to obtain even deeper feature maps, hence the term deep feature maps. They have lower spatial resolution, smaller feature map size, and contain higher-level semantic features, including the overall shape, structure, and category information of the fruit. They are more sensitive to target type and are suitable for classification and localization. shallow feature map, middle feature map and deep feature map Figure 1 The input is fed into the neck network for multi-scale feature fusion, enhancing the model's ability to perceive fruits at different scales.
[0023] The principle of replacing the PAN structure with a Slim-Neck based on GSConv and VoV-GSCSP is as follows: The original PAN structure in the neck network of the basic YOLOv8n model was completely replaced with a Slim-Neck structure based on GSConv and VoV-GSCSP. The specific processing procedure of GSConv for each input feature map is as follows: the convolution kernel size is set to 3×3, the convolution stride is set to 2, the input feature map is downsampled, the downsampled feature map is input into the depthwise separable convolutional layer, and channel-wise convolution and point-wise convolution are performed respectively. Channel-wise convolution means that each channel is convolutionally operated independently, and point-wise convolution means that channels are fused through a convolution kernel of size 1×1. Then, channel shuffling is performed, which means that the feature information of different channels is rearranged and combined. The GSConv module is a key component of the Slim-Neck architecture, used to achieve effective feature extraction and downsampling with lightweight design. It first uses a standard 3×3 convolutional kernel with a stride of 2 to descend the input feature map, resulting in a feature map with halved spatial size but unchanged channel number, preserving the original channel count and achieving initial spatial information compression. The downsampled feature map is then subjected to depthwise separable convolutions, specifically channel-wise and pointwise convolutions. Channel-wise convolution means that each channel of the downsampled feature map is independently convolved with a 3×3 kernel, without cross-channel fusion. Spatial features of each channel are extracted; pointwise convolution uses a 1×1 kernel to fuse the output of each channel across channels, integrating information from different channels and enhancing feature representation; then channel shuffling is performed, rearranging the feature maps output by pointwise convolution along the channel dimension. Through grouping and recombination, information interaction between different channel groups is promoted, enhancing the information flow of feature maps between channels, avoiding information isolation, and improving the model's expressive power; finally, the output feature map is half the size of the input feature map while the number of channels remains the same, preserving sufficient spatial information while possessing good channel representation ability; Input a feature map from a certain depth level of the backbone network into GSConv, and perform standard convolution downsampling, depthwise separable convolution decomposition, and channel shuffling operations in sequence. Output a feature map with half the spatial size and the same number of channels as the input feature map. Perform the above processing on shallow feature maps, mid-level feature maps, and deep feature maps respectively.
[0024] VoV-GSCSP processes each input feature map after GSCV processing as follows: It adopts a dual-branch design. One branch uses a 1×1 convolution kernel with a stride of 1 for standard convolution. The other branch first passes through a Ghost convolution, then performs channel shuffling on the Ghost convolution output, and then passes it through convolution kernels of sizes 1×1, 3×3, and 1×1 in sequence to generate a lightweight bottleneck structure. The outputs of the two branches are concatenated in the channel dimension to form the concatenated input feature map, which includes a shallow feature map, a middle feature map, and a deep feature map. The GSCSP inputs a feature map processed by GSCV. Two parallel branches process the GSCV-processed feature map: a standard convolution branch, which first compresses channels using a 1×1 kernel, then extracts features using a 3×3 kernel, and finally restores channels using a 1×1 kernel; and a lightweight Ghost convolution branch, designed to further reduce the number of parameters. This branch first applies a Ghost convolution to the GSCV-processed feature map, then performs channel shuffling on the Ghost convolution output to enhance inter-channel information exchange. Features are further extracted by sequentially passing the outputs through 1×1, 3×3, and 1×1 kernels. The outputs of the two branches are then concatenated along the channel dimension. Through Ghost convolution and channel shuffling, the number of parameters and computational cost are significantly reduced with almost no loss of accuracy.
[0025] The input feature map includes shallow feature map, medium feature map and deep feature map.
[0026] The specific principle of integrating the Detect-SEAM module is as follows: The Detect-SEAM module is embedded as a plugin before the classification and regression branches, and the shallow, mid-level, and deep feature maps after concatenation are processed in parallel by the Detect-SEAM module. The specific process of processing the concatenated shallow feature map using the Detect-SEAM module is as follows: Each channel in the concatenated shallow feature map is independently spatially convolved using depthwise separable convolution, followed by pointwise convolution to fuse the channels and generate an intermediate feature map. For the intermediate feature map, global average pooling is performed to generate channel description vectors, with the channel description vectors having a size of [value missing]. , This represents the number of channels in the channel description vector. This channel description vector is input into a two-layer fully connected network, where the first fully connected layer compresses the number of channels to... , Indicates the reduction ratio; the reduction ratio r is used in the channel attention mechanism to reduce the number of parameters and computational cost of the fully connected layer by compressing the number of channels, while retaining key channel information. A good balance is achieved between information retention and compression, providing strong channel selection capabilities without excessive information loss; the number of channels is restored through a second fully connected layer. The output of the two fully connected layers is converted using the sigmoid function. The dimensional vector is normalized to generate the attention weight vector for each channel. Each channel in the concatenated shallow feature map is multiplied by the corresponding attention weight vector to output the shallow enhanced feature map. Similarly, the concatenated middle and deep feature maps are processed in the same way to generate the middle and deep enhanced feature maps, respectively. The shallow, middle, and deep enhanced feature maps are then combined to generate a multi-scale enhanced feature map.
[0027] The Detect-SEAM module combines channel attention with a lightweight convolutional design to enhance the model's ability to perceive key channel features and suppress irrelevant background information, thereby improving detection accuracy. Taking the concatenated shallow feature map as an example, to extract spatial context information while maintaining channel independence, depthwise separable convolution is first performed on the input feature map, including channel-wise convolution and pointwise convolution. Channel-wise convolution specifically uses a 3×3 convolution kernel independently for each channel to extract spatial features within that channel. Pointwise convolution specifically uses a 1×1 convolution kernel to fuse the information from each channel, generating a transitional feature map. The transitional feature map is then subjected to global average pooling along the spatial dimension to obtain a 1×1×1 convolutional kernel. The channel description vector represents the global feature response intensity of each channel; and fully connected layers are used for channel compression and restoration. The first fully connected layer compresses the number of channels to... The second fully connected layer restores the number of channels to The recovered channel vectors are normalized using the Sigmoid function to obtain the attention weight vector for each channel, with values ranging from [value range missing]. The importance of a channel is indicated by the concatenated shallow feature map. The attention weight vector for each channel is multiplied by the corresponding attention weight vector for each channel to obtain the attention weight vector for each channel. Channels important for fruit detection tasks, such as fruit color, texture, or shape, are strengthened, while background or interfering channels, such as leaf occlusion, lighting, and shadows, are suppressed. Before classification and localization, the model focuses on the features most relevant to the current task, thereby improving discriminative power. Spatial context information is extracted through channel-wise convolution to maintain the independence of channels and avoid premature mixing. Cross-channel information is fused through pointwise convolution.
[0028] Step 3: Input the training set RGB images into the improved YOLOv8n model. Obtain the corresponding multi-scale enhanced feature maps in the Detect-SEAM module. Process the multi-scale enhanced feature maps of the training set RGB images in the detection head to determine the fruit category, bounding box center point coordinates, and bounding box size. Validate the model using the validation set RGB images to generate a classification prediction model. In this embodiment, the principle underlying the generation of the classification prediction model is as follows: In the detection head, for the multi-scale enhanced feature maps of the RGB images in the training set, the bounding box coordinates, bounding box size, and fruit category are predicted. The AdamW optimizer is used, and the learning rate is set to [value missing]. The training run consisted of 200 rounds with a batch size of 16. The bounding box loss function was Clou Loss, and the class loss function was binary cross-entropy. After each training round, the MAP was calculated using the validation set RGB images. When the MAP stopped improving after ten consecutive training rounds, training was stopped, and the model with the highest MAP corresponding to the validation set RGB images was saved as the classification prediction model.
[0029] The RGB images from the training set are input into the improved YOLOv8n model. They are first processed by the backbone network, the neck network, and the Detect-SEAM module, ultimately forming multi-scale enhanced feature maps which are then input into the detection head. These multi-scale enhanced feature maps specifically include: shallow enhanced feature maps (high spatial resolution, rich details, suitable for small target detection); mid-level enhanced feature maps (balancing details and semantic information, suitable for medium-scale targets); and deep enhanced feature maps (strong semantic information, suitable for large targets and classification tasks). For each scale of enhanced feature map, the detection head outputs the fruit category, bounding box center coordinates, and bounding box size corresponding to the bounding box.
[0030] The optimizer used is AdamW, combined with weight decay to prevent overfitting, and the initial learning rate is set to... The learning rate is dynamically adjusted during training using a cosine annealing scheduling strategy. Clou Loss is used as the bounding box loss function to optimize the center point coordinates and width / height predictions of the bounding boxes. Binary cross-entropy loss is used for the class loss function, supporting multi-class classification tasks to determine whether a certain type of fruit exists within the bounding box. Images and their corresponding annotation files are loaded from the training set in batches. Data augmentation strategies are applied in real-time during training. Images are input into the improved YOLOv8n model, passing sequentially through the backbone network, neck network, and Detect-SEAM module, outputting multi-scale feature maps. For each scale feature map, the bounding box coordinates, bounding box size, and fruit class are obtained from the detection head. The predicted values are compared with the true labels, and the loss function is calculated for each. After each training round, the model performance is evaluated using validation set RGB images. The model performance is evaluated using the average accuracy (MAP) across all classes. The specific principle is as follows: for each validation set RGB image… The model outputs multiple prediction boxes, each containing the predicted category, the confidence score corresponding to the category, the coordinates of the bounding box center point, and the bounding box size. For each fruit category, all prediction boxes belonging to that category in the validation set are collected and sorted from highest to lowest confidence score. The ratio P of the number of correctly predicted boxes to the total number of prediction boxes in a category, and the ratio R of the number of correctly predicted boxes to the total number of true boxes are calculated. A PR curve is plotted, and for each category, the area under its PR curve is calculated, which is the AP for that category. The average AP of all categories is taken to obtain the MAP. After each training iteration, a pre-selected classification prediction model and its corresponding MAP are obtained. The MAP values of each pre-selected classification prediction model are compared. When the MAP value no longer increases after ten consecutive training iterations, the pre-selected classification prediction model corresponding to the maximum MAP value is output, which is the final classification prediction model.
[0031] Step 4: Input the RGB image to be predicted into the classification prediction model to obtain the fruit category corresponding to the RGB image to be predicted, its confidence score, the coordinates of the center point of the bounding box, and the size of the bounding box.
[0032] In this embodiment, when the RGB image to be predicted is input into the improved YOLOv8n model, the model will perform the same feature extraction process as during training. First, multi-scale enhanced feature maps are acquired, and then these maps are input into the detection head. Each scale feature map corresponds to a set of prediction outputs, including the fruit category and its confidence score, the coordinates of the bounding box center point, and the bounding box size. The predicted bounding box coordinate offsets are converted into the actual pixel coordinates in the RGB image to be predicted. A confidence threshold of 0.5 is set, and only bounding boxes with a maximum confidence score higher than the confidence threshold are retained. For each retained bounding box, the category corresponding to the highest confidence score is selected as the final category of the fruit within that box. For the four fruits—pomegranate, apple, pear, and citrus—if the confidence combination of an RGB image to be predicted is... This indicates that there is an 85% probability that the fruit is a pomegranate. The probability of it being another fruit is too small to be considered. Furthermore, since 85% is greater than the preset confidence threshold of 0.5, the fruit category of the RGB image to be predicted can be identified as a pomegranate.
[0033] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.
[0034] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.
[0035] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0036] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A fruit classification method based on an improved YOLOv8n network, characterized in that, The specific steps include: Step 1: Collect RGB images containing various types of fruits, preprocess the RGB images, and divide the preprocessed RGB images into training and validation sets in an 8:2 ratio; Step 2: Construct an improved YOLOv8n model, which includes a backbone network, a neck network, and a detection head. Based on the basic YOLOv8n model, an SPD-Conv module is inserted after each C2f module, the PAN structure of the neck network is replaced with a Slim-Neck based on GSConv and VoV-GSCSP, and the Detect-SEAM module is integrated before the classification and regression branches of the detection head. Step 3: Input the training set RGB images into the improved YOLOv8n model, obtain the corresponding multi-scale enhanced feature maps in the Detect-SEAM module, process the multi-scale enhanced feature maps of the training set RGB images in the detection head to determine the fruit category, bounding box center point coordinates and bounding box size, and verify them with the validation set RGB images to generate a classification prediction model; Step 4: Input the RGB image to be predicted into the classification prediction model to obtain the fruit category corresponding to the RGB image to be predicted, its confidence score, the coordinates of the center point of the bounding box, and the size of the bounding box.
2. The fruit classification method based on the improved YOLOv8n network according to claim 1, characterized in that: The preprocessing steps for the RGB images in step 1 specifically include: scaling, bounding box annotation, and data augmentation. Scaling refers to scaling the RGB images to 640×640 pixels. Bounding box annotation involves using the Labellmg tool to annotate the bounding boxes of the fruits in each RGB image. After annotation, 10% of the images are randomly selected for review until the average annotation error is no higher than 2.4%. Data augmentation involves sequentially performing the following steps on the annotated images: Random rotation, scaling and translation, horizontal and vertical flipping, and lighting parameter adjustment.
3. The fruit classification method based on the improved YOLOv8n network according to claim 1, characterized in that: The specific operation steps of the SPD-Conv module in step 2 are as follows: The input image is spatially partitioned by dividing it into two non-overlapping regions along both the height and width directions, resulting in four sub-regions, each with a size of [size missing]. ,in, Indicates the height of the image, Indicates the width of the image. This represents the number of channels in the image, and ; All sub-regions are stitched together along the channel dimension to generate an intermediate feature map, the size of which is [size missing]. ; The kernel size is set to 1×1, the stride to 1, and the number of output channels are set. A convolution operation is performed on the intermediate feature maps to output a backbone feature map. The size of the backbone feature map is... ,in, Indicates the number of output channels; The feature maps of three depth levels in the backbone network are selected for output. The three depth level feature maps include shallow feature maps, mid-level feature maps, and deep feature maps. The shallow feature maps represent the feature maps extracted before processing by the SPD-Conv module, the mid-level feature maps represent the backbone feature maps, and the deep feature maps represent the feature maps that are processed by the SPD-Conv module and then further processed in the backbone network.
4. The fruit classification method based on the improved YOLOv8n network according to claim 3, characterized in that: The principle behind replacing the PAN structure with a Slim-Neck based on GSConv and VoV-GSCSP in step 2 is as follows: The original PAN structure in the neck network of the basic YOLOv8n model was completely replaced with a Slim-Neck structure based on GSConv and VoV-GSCSP. The specific processing procedure of GSConv for each input feature map is as follows: the convolution kernel size is set to 3×3, the convolution stride is set to 2, the input feature map is downsampled, the downsampled feature map is input into the depthwise separable convolutional layer, and channel-wise convolution and point-wise convolution are performed respectively. Channel-wise convolution means that each channel is convolutionally operated independently, and point-wise convolution means that channels are fused through a convolution kernel of size 1×1. Then, channel shuffling is performed, which means that the feature information of different channels is rearranged and combined. VoV-GSCSP processes each input feature map after GSCV processing as follows: It adopts a dual-branch design. One branch uses a 1×1 convolution kernel with a stride of 1 for standard convolution. The other branch first passes through a Ghost convolution, then performs channel shuffling on the Ghost convolution output, and then passes it through convolution kernels of sizes 1×1, 3×3, and 1×1 in sequence to generate a lightweight bottleneck structure. The outputs of the two branches are concatenated in the channel dimension to form the concatenated input feature map, which includes a shallow feature map, a middle feature map, and a deep feature map. The input feature map includes shallow feature map, medium feature map and deep feature map.
5. A fruit classification method based on an improved YOLOv8n network according to claim 4, characterized in that: The specific principle behind integrating the Detect-SEAM module in step 2 is as follows: The Detect-SEAM module is embedded as a plugin before the classification and regression branches, and the shallow, mid-level, and deep feature maps after concatenation are processed in parallel by the Detect-SEAM module. The specific process of processing the concatenated shallow feature map using the Detect-SEAM module is as follows: Each channel in the concatenated shallow feature map is independently spatially convolved using depthwise separable convolution, followed by pointwise convolution to fuse the channels and generate an intermediate feature map. For the intermediate feature map, global average pooling is performed to generate channel description vectors, with the channel description vectors having a size of [value missing]. , This represents the number of channels in the channel description vector. This channel description vector is input into a two-layer fully connected network, where the first fully connected layer compresses the number of channels to... , This indicates the reduction ratio, and the number of channels is restored to the original value through the second fully connected layer. The output of the two fully connected layers is converted using the sigmoid function. The dimensional vector is normalized to generate the attention weight vector for each channel. Each channel in the concatenated shallow feature map is multiplied by the corresponding attention weight vector to output the shallow enhanced feature map. Similarly, the concatenated middle and deep feature maps are processed in the same way to generate the middle and deep enhanced feature maps, respectively. The shallow, middle, and deep enhanced feature maps are then combined to generate a multi-scale enhanced feature map.
6. The fruit classification method based on the improved YOLOv8n network according to claim 1, characterized in that: The principle underlying the generation of the classification prediction model in step 3 is as follows: In the detection head, for the multi-scale enhanced feature maps of the RGB images in the training set, the bounding box coordinates, bounding box size, and fruit category are predicted. The AdamW optimizer is used, and the learning rate is set to [value missing]. The training rounds are 200, the batch size is 16, the bounding box loss function is Clou Loss, and the class loss function is binary cross-entropy. After each training round, the MAP is calculated using the RGB images of the validation set. When the MAP no longer improves after ten consecutive training rounds, training is stopped, and the model with the highest MAP corresponding to the RGB images of the validation set is saved as the classification prediction model.