Tea leaf grading and counting method based on improved YOLOv8n network
The improved YOLOv8n network with SPD-Conv and super token modules enhances tea grading and counting accuracy by improving feature extraction and regression in complex tea garden environments.
Patent Information
- Application Number
- CN202411233621.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-27
AI Technical Summary
Existing tea picking methods rely heavily on manual labor, leading to inefficiency and high costs, and existing machines struggle with accurate tea grading and counting due to poor generalization and robustness in traditional target detection algorithms, especially in complex tea garden environments.
A tea grading and counting method using an improved YOLOv8n network equipped with an SPD-Conv module and a super token visual conversion module, along with an MPDIoU loss function, to enhance feature extraction and regression accuracy.
The method significantly improves tea grading and counting accuracy by enhancing feature extraction and model convergence, addressing the limitations of traditional algorithms in complex tea garden environments.
Smart Images
Figure CN119206710A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tea leaf recognition, and in particular to a tea leaf grading and counting method based on an improved YOLOv8n network. Background Art
[0002] At present, tea picking mainly relies on manual labor, which has problems of low efficiency and high labor costs. Existing tea picking machines are mostly handheld and suitable for small tea gardens. Freshly picked tea leaves are often mixed, making subsequent processing difficult and affecting the production and processing of high-quality tea. This damages the integrity and availability of tea leaves and causes economic losses to a certain extent, thus creating a demand for tea grading.
[0003] If the classification is done by naked eyes, it is undoubtedly inefficient, so there is a need to use machine vision technology for identification. However, due to the dense distribution of tea leaves, traditional target detection algorithms can only extract shallow features from tea images based on color, texture, etc., and their generalization ability and robustness are poor, resulting in poor accuracy in tea grading. In addition, they do not have a counting function and can only be roughly counted manually.
[0004] Therefore, how to provide a tea grading and counting method based on the improved YOLOv8n network to improve the accuracy of tea grading and counting has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a tea grading and counting method based on an improved YOLOv8n network, so as to improve the accuracy of tea grading and counting.
[0006] The present invention is implemented as follows: a tea grading and counting method based on an improved YOLOv8n network comprises the following steps:
[0007] Step S1, collecting a large number of tea images with different lighting conditions, different weather conditions, different shooting angles, and different tea grades, preprocessing each of the tea images and constructing a data set;
[0008] Step S2, creating a tea grading and counting model based on the YOLOv8n network, and setting the loss function of the tea grading and counting model to the MPDIoU function;
[0009] Step S3, training a tea grading and counting model using the data set;
[0010] Step S4: using the trained tea grading and counting model to grade and count tea leaves, and outputting the grading and counting results.
[0011] Furthermore, in step S1, the preprocessing of each tea leaf image is specifically performed as follows:
[0012] Labeling each of the tea images with a tea boundary box, tea grade, tea quantity, coordinates of a center point of the boundary box, and an aspect ratio of the boundary box to the tea image, and randomly performing sample expansion operations such as flipping, contrast adjustment, and rotation on each of the labeled tea images to complete preprocessing of each of the tea images;
[0013] The tea leaves are classified into single bud, one bud and one leaf, or one bud and two leaves.
[0014] Furthermore, in step S2, the YOLOv8n network is provided with an SPD-Conv module and a super token visual conversion module.
[0015] Furthermore, the SPD-Conv module is composed of a space-to-depth layer and a non-stepped convolutional layer;
[0016] The space-to-depth layer is used to downsample the input tea image according to a preset scale factor, and divide it into a sub-image set including a plurality of tea sub-images;
[0017] The non-step convolutional layer is used to extract features from each tea sub-image and combine the tea sub-images back into a tea image.
[0018] Furthermore, the super token visual conversion module is composed of a convolution unit, an STT Block unit, an average pooling layer and a classifier;
[0019] The STT Block unit is provided with a convolutional position embedding subunit, a super token sampling subunit and a convolutional feed-forward network subunit;
[0020] The convolution position embedding subunit is used to learn the correlation between different channels of the tea image; the super token sampling subunit is used to learn the global features of the tea image; and the convolution feedforward network subunit is used to enhance the local features of the tea image.
[0021] Furthermore, in step S2, the formula of the MPD I oU function is:
[0022] L MPDIoU =1-MPDIoU;
[0023]
[0024]
[0025]
[0026] Among them, LMPDIoU represents the loss value of the loss function; MPDIoU represents the loss function based on the minimum point distance; A represents the predicted tea bounding box; B represents the real tea bounding box; w represents the width of the tea image; h represents the height of the tea image; Indicates the coordinate of the upper left corner of A; Indicates the coordinates of the lower right corner of A; Indicates the coordinate of the upper left corner of B; Indicates the coordinates of the lower right corner of B; Represents the square of the Euclidean distance between the upper left corner of the predicted tea leaf bounding box and the true tea leaf bounding box; Represents the square of the Euclidean distance between the predicted tea leaf bounding box and the lower right corner of the true tea leaf bounding box.
[0027] Furthermore, the step S3 is specifically as follows:
[0028] Dividing the data set into a training set and a validation set based on a preset ratio, training the tea grading and counting model through the training set, and continuously optimizing the hyperparameters of the tea grading and counting model including at least a random deactivation rate, a weight decay rate, and a learning rate during the training process until the loss value of the loss function is less than a preset loss threshold;
[0029] The trained tea grading and counting model is verified by the verification set to determine whether the recognition accuracy is greater than a preset accuracy threshold. If so, the training is terminated; if not, the training set is expanded to continue the training.
[0030] Furthermore, the step S4 is specifically as follows:
[0031] The tea image to be identified is obtained, and after preprocessing the tea image to be identified by image enhancement, the trained tea grading and counting model is input to grade and count the tea leaves, and the output includes at least the predicted tea boundary box, the tea grade of each tea boundary box, and the grading and counting results of the number of tea leaves.
[0032] Furthermore, the step S4 further includes:
[0033] The tea grading and counting model is continuously optimized based on the grading and counting results.
[0034] The advantages of the present invention are:
[0035] By collecting a large number of tea images with different lighting conditions, weather conditions, shooting angles, and tea grades, each tea image is preprocessed including annotation and sample expansion operations to construct a data set; then a tea grading and counting model is created based on the YOLOv8n network, and the loss function of the tea grading and counting model is set to MPD I oU function, trains the tea grading and counting model through the data set, and finally uses the trained tea grading and counting model to grade and count the tea, and outputs the grading and counting results; since the YOLOv8n network is equipped with an SPD-Conv module and a super token visual conversion module, the SPD-Conv module consists of a space-to-depth layer and a non-step convolution layer; the space-to-depth layer is used to downsample the input tea image according to a preset scale factor, and divide it into a sub-image set including several tea sub-images, and the non-step convolution layer is used to extract features from each tea sub-image and combine each tea sub-image back into a tea image, that is, the SPD-Conv module replaces the convolution module of the traditional YOLOv8n network to improve the detection performance of low image resolution or small targets; the super token visual conversion module can learn the correlation and global features of different channels of the tea image, and enhance the local features, which greatly improves the tea feature extraction capability; the MPD I oU function replaces the traditional CI oU function, which effectively improves the regression effect of the predicted tea bounding box and accelerates the convergence speed of the tea grading and counting model. That is, the tea grading and counting model created based on the improved YOLOv8n network is used to grade and count tea, which greatly improves the accuracy of tea grading and counting. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.
[0037] Figure 1 It is a flow chart of a tea grading and counting method based on an improved YOLOv8n network of the present invention.
[0038] Figure 2 It is a structural schematic diagram of the tea grading and counting model of the present invention.
[0039] Figure 3 It is a structural schematic diagram of the SPD-Conv module of the present invention.
[0040] Figure 4 It is a structural diagram of the super token visual conversion module of the present invention.
[0041] Figure 5 It is a structural schematic diagram of the convolution position embedding subunit of the present invention. DETAILED DESCRIPTION
[0042] The technical solution in the embodiment of the present application has the following overall idea: tea grading and counting is performed by using a tea grading and counting model created by a YOLOv8n network equipped with an SPD-Conv module and a super token visual conversion module, the SPD-Conv module is composed of a space-to-depth layer and a non-step convolution layer; the space-to-depth layer is used to downsample the input tea image according to a preset scale factor, and divide it into a sub-image set including a number of tea sub-images, the non-step convolution layer is used to extract features from each tea sub-image, and combine each tea sub-image back into a tea image, that is, the SPD-Conv module is used to replace the convolution module of the traditional YOLOv8n network to improve the detection performance of low image resolution or small targets; the super token visual conversion module can learn the correlation and global features of different channels of the tea image, and enhance the local features, which greatly improves the tea feature extraction capability; the MPD I oU function is used to replace the traditional CI oU function as the loss function of the tea grading and counting model, which effectively improves the regression effect of the predicted tea bounding box, accelerates the convergence speed of the tea grading and counting model, and improves the accuracy of tea grading and counting.
[0043] Please refer to Figures 1 to 5 As shown, a preferred embodiment of a tea grading and counting method based on an improved YOLOv8n network of the present invention comprises the following steps:
[0044] Step S1, collecting a large number of tea images with different lighting conditions, different weather conditions, different shooting angles, and different tea grades, preprocessing each of the tea images and constructing a data set;
[0045] In specific implementation, the positioning relationship between the tea-picking robot and the tea tree can be simulated to collect tea images. For example, the distance between the tea-picking robot and the tea tree is set to 55-85 cm, and the camera is located 20-30 cm away from the tea tree buds and about 120 cm from the ground. By collecting tea images with different lighting conditions, different weather conditions, different shooting angles, and different tea grades, the richness and comprehensiveness of the samples are effectively improved to provide a more realistic tea garden environment, which helps to improve the robustness of the tea grading and counting model.
[0046] Step S2: Create a tea grading and counting model based on the YOLOv8n network, and set the loss function of the tea grading and counting model to the MPD I oU function;
[0047] Step S3, training a tea grading and counting model using the data set;
[0048] Step S4: using the trained tea grading and counting model to grade and count tea leaves, and outputting the grading and counting results.
[0049] In step S1, the preprocessing of each tea leaf image is specifically performed as follows:
[0050] Labeling each of the tea images with a tea boundary box, tea grade, tea quantity, coordinates of a center point of the boundary box, and an aspect ratio of the boundary box to the tea image, and randomly performing sample expansion operations such as flipping, contrast adjustment, and rotation on each of the labeled tea images to complete preprocessing of each of the tea images;
[0051] By randomly flipping, adjusting the contrast, and rotating the tea images, the problem of model performance degradation caused by sample imbalance can be solved;
[0052] The tea leaves are classified into single bud, one bud and one leaf, or one bud and two leaves.
[0053] In step S2, the YOLOv8n network is provided with an SPD-Conv module and a super token visual transformation module (Super-Token Vi si on Transformer).
[0054] Since traditional visual recognition models cannot cope with detection requirements under complex and diverse backgrounds, and traditional YOLOv8n networks perform poorly in detection accuracy, the present invention embeds the SPD-Conv module into the backbone of the tea grading and counting model to enhance the deep feature extraction capability of the target, integrates the super token visual conversion module to reduce the attention of the tea grading and counting model to redundant information, thereby improving the perception ability of tea, improves the traditional positioning loss function into the MPDI oU function, accelerates the convergence speed of the tea grading and counting model, optimizes the model performance, and adds classification positioning and counting functions to achieve the purpose of classification counting.
[0055] The SPD-Conv module consists of a space-to-depth layer and a non-strided convolutional layer;
[0056] The space-to-depth layer (SPD) is used to downsample the input tea image according to a preset scale factor, and divide it into a sub-image set including a plurality of tea sub-images;
[0057] The non-step convolutional layer is used to extract features from each tea sub-image and combine the tea sub-images back into a tea image.
[0058] Traditional neural network models show poor generalization ability when extracting low-resolution and small targets. This is because, as the neural network model goes deeper in the network hierarchy, the feature maps are continuously downsampled, which is the result of the calculation of the convolutional layer and the pooling layer. Therefore, in the feature extraction process, fine-grained information is difficult to effectively capture, resulting in reduced detection accuracy and robustness.
[0059] In the YOLOv8n network, the feature extraction module Conv uses a stride convolution layer, which poses a challenge in tasks involving low resolution or small objects, and usually leads to a decrease in detection performance. To overcome this challenge, the present invention introduces a combination of space-to-depth layers and non-step convolution layers, replacing the step convolution layers in the backbone network of the original model to enhance the performance of the model in processing low-resolution images and complex small objects, and can more effectively express image features, especially in tea recognition, thereby improving the robustness of the model.
[0060] The space-to-depth layer in the SPD-Conv module divides the input tea image into sub-atlases in a scale factor-controlled manner. The subsequent non-stepped convolutional layer can effectively reduce the information loss of the sub-atlas and improve the detection accuracy of small object areas.
[0061] When the scale factor is 2, the width and height of the output tea sub-image are halved, indicating that the tea sub-image is obtained by downsampling the input tea image twice, and four tea sub-images are obtained, each of which has the same number of channels as the tea image.
[0062] The super token visual conversion module consists of a convolution unit, an STT Block unit, an average pooling layer and a classifier;
[0063] The STT Block unit is provided with a convolutional position embedding subunit (CPE, Convolutional Position Embedding), a super token sampling subunit (STA, Super Token Attent ion) and a convolutional feed-forward network subunit (Conv FFN);
[0064] The convolution position embedding subunit is used to learn the correlation between different channels of the tea image; the super token sampling subunit is used to learn the global features of the tea image; and the convolution feedforward network subunit is used to enhance the local features of the tea image.
[0065] The attention mechanism is a resource allocation scheme that is used to prioritize more important tasks when computing power is limited, effectively solving the problem of information overload. In the learning of neural network models, the attention mechanism is introduced to focus on the more critical information in the current task among a large number of input information, reduce attention to other information, and even filter out redundant information, thereby solving the problem of information overload and improving the efficiency and accuracy of task processing. In order to enhance the hierarchical detection capability of the YOLOv8n network for tea targets, prioritize key targets in the complex background of tea gardens, and improve scalability and robustness, the super token visual conversion module is introduced.
[0066] The convolution position embedding subunit, the super token sampling subunit, and the convolution feedforward network subunit are connected in sequence, and the formula is:
[0067] X=CPE(X in )+X in ;
[0068] Y = STA (LN (X)) + X;
[0069] Z = ConvFFN(BN(Y)) + Y;
[0070] Among them, X in Represents the input tea image; CPE() represents the convolution position embedding subunit; X represents the input of the super token sampling subunit; STA() represents the super token sampling subunit; LN() represents the logarithmic function; Y represents the input of the convolution feedforward network subunit; ConvFFN() represents the convolution feedforward network subunit; BN() represents the Bernoulli distribution function; Z represents the output of the STT Block unit.
[0071] Depthwise separable convolution plays a vital role in the convolution position embedding subunit, and is particularly suitable for lightweight detection models, including depthwise convolution and pointwise convolution. Unlike traditional convolution, depthwise convolution operates uniquely by independently convolving each channel of the input layer, each convolution kernel is responsible for one channel, the number of feature maps is the same as the number of channels in the input layer, and the convolution operation of each channel is performed in parallel; in the subsequent pointwise convolution stage, the output of the depthwise convolution is convolved with a 1x1 convolution kernel, and the feature map information generated independently by each channel is integrated and interacted to generate a new feature map as output, which aims to enhance the model's ability to learn the correlation between different channels.
[0072] The super token sampling subunit is good at learning global information and adopts a fast sampling method to predict by discerning the sparse relationship between labels and super labels; then, a multi-head self-attention operation is performed in the space of the super token visual transformation module to capture the dependencies between the super token visual transformation modules for modeling; finally, the information from the super token visual transformation module is remapped back to the original label space through an upsampling algorithm to minimize redundant features.
[0073] The convolutional feedforward network subunit is used to enhance local features. It processes features using 3x3 convolution kernels to extract richer local information and has excellent representation learning capabilities. The specific calculation process of the convolutional feedforward network subunit is as follows: first, each input channel is weighted and summed through the convolution kernel to adjust the feature dimension, then, in order to enhance the representativeness of local features, each input channel is independently convolved, and finally, a 1x1 convolution operation is performed again to further adjust the size and combination of features.
[0074] In step S2, the formula of the MPDIoU function is:
[0075] L MPDIoU =1-MPDIoU;
[0076]
[0077]
[0078]
[0079] Among them, L MPDIoU represents the loss value of the loss function; MPDIoU represents the loss function based on the minimum point distance; A represents the predicted tea bounding box; B represents the real tea bounding box; w represents the width of the tea image; h represents the height of the tea image; Indicates the coordinate of the upper left corner of A; Indicates the coordinates of the lower right corner of A; Indicates the coordinate of the upper left corner of B; Indicates the coordinates of the lower right corner of B; Represents the square of the Euclidean distance between the upper left corner of the predicted tea leaf bounding box and the true tea leaf bounding box; Represents the square of the Euclidean distance between the predicted tea leaf bounding box and the lower right corner of the true tea leaf bounding box.
[0080] The traditional CI oU function only considers the intersection area of the bounding box, the distance to the center point and the aspect ratio of the bounding box. The MPD I oU function of the present invention is used for the regression of overlapping and non-overlapping bounding boxes, taking into account the distance from the center point and the deviation of width and height, and using a bounding box similarity measure based on the minimum point distance.
[0081] The step S3 is specifically as follows:
[0082] Dividing the data set into a training set and a validation set based on a preset ratio, training the tea grading and counting model through the training set, and continuously optimizing the hyperparameters of the tea grading and counting model including at least a random deactivation rate, a weight decay rate, and a learning rate during the training process until the loss value of the loss function is less than a preset loss threshold;
[0083] The trained tea grading and counting model is verified by the verification set to determine whether the recognition accuracy is greater than a preset accuracy threshold. If so, the training is terminated; if not, the training set is expanded to continue the training.
[0084] During specific implementation, the performance of the tea grading and counting model can also be evaluated based on precision, recall rate, and average precision.
[0085] The step S4 is specifically as follows:
[0086] The tea image to be identified is obtained, and after preprocessing the tea image to be identified by image enhancement, the trained tea grading and counting model is input to grade and count the tea leaves, and the output includes at least the predicted tea boundary box, the tea grade of each tea boundary box, and the grading and counting results of the number of tea leaves.
[0087] The step S4 further comprises:
[0088] The tea grading and counting model is continuously optimized based on the grading and counting results.
[0089] In summary, the advantages of the present invention are:
[0090] By collecting a large number of tea images with different lighting conditions, weather conditions, shooting angles, and tea grades, each tea image is preprocessed including annotation and sample expansion operations to construct a data set; then a tea grading and counting model is created based on the YOLOv8n network, and the loss function of the tea grading and counting model is set to MPD IoU function, train the tea grading and counting model through the data set, and finally use the trained tea grading and counting model to grade and count the tea, and output the grading and counting results; since the YOLOv8n network is equipped with an SPD-Conv module and a super token visual conversion module, the SPD-Conv module is composed of a space-to-depth layer and a non-step convolution layer; the space-to-depth layer is used to downsample the input tea image according to a preset scale factor, and divide it into a sub-image set including several tea sub-images, and the non-step convolution layer is used to extract features from each tea sub-image and combine each tea sub-image back into a tea image, that is, the convolution module of the traditional YOLOv8n network is replaced by the SPD-Conv module to improve the detection performance of low image resolution or small targets; the super token visual conversion module can learn the correlation and global features of different channels of the tea image, and enhance the local features, which greatly improves the tea feature extraction capability; through MPD I The YOLOv8n function replaces the traditional CIoU function, which effectively improves the regression effect of the predicted tea bounding box and accelerates the convergence speed of the tea grading and counting model. That is, the tea grading and counting model created based on the improved YOLOv8n network is used to grade and count tea, which greatly improves the accuracy of tea grading and counting.
[0091] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.
Claims
1. A tea grading and counting method based on an improved YOLOv8n network, characterized in that: The steps include: Step S1, collecting a large number of tea images with different lighting conditions, different weather conditions, different shooting angles, and different tea grades, preprocessing each of the tea images and constructing a data set; Step S2, creating a tea grading and counting model based on the YOLOv8n network, and setting the loss function of the tea grading and counting model to the MPDIoU function; Step S3, training a tea grading and counting model using the data set; Step S4: using the trained tea grading and counting model to grade and count tea leaves, and outputting the grading and counting results.
2. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 1, characterized in that: In step S1, the preprocessing of each tea leaf image is specifically performed as follows: Labeling each of the tea images with a tea boundary box, tea grade, tea quantity, coordinates of a center point of the boundary box, and an aspect ratio of the boundary box to the tea image, and randomly performing sample expansion operations such as flipping, contrast adjustment, and rotation on each of the labeled tea images to complete preprocessing of each of the tea images; The tea leaves are classified into single bud, one bud and one leaf, or one bud and two leaves.
3. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 1, characterized in that: In step S2, the YOLOv8n network is provided with an SPD-Conv module and a super token visual conversion module.
4. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 3, characterized in that: The SPD-Conv module consists of a space-to-depth layer and a non-strided convolutional layer; The space-to-depth layer is used to downsample the input tea image according to a preset scale factor, and divide it into a sub-image set including a plurality of tea sub-images; The non-step convolutional layer is used to extract features from each tea sub-image and combine the tea sub-images back into a tea image.
5. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 3, characterized in that: The super token visual conversion module consists of a convolution unit, an STT Block unit, an average pooling layer and a classifier; The STT Block unit is provided with a convolutional position embedding subunit, a super token sampling subunit, and a convolutional feedforward network subunit; The convolution position embedding subunit is used to learn the correlation between different channels of the tea image; the super token sampling subunit is used to learn the global features of the tea image; and the convolution feedforward network subunit is used to enhance the local features of the tea image.
6. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 1, characterized in that: In step S2, the formula of the MPDIoU function is: L MPDIoU =1-MPDIoU; Among them, L MPDIoU represents the loss value of the loss function; MPDIoU represents the loss function based on the minimum point distance; A represents the predicted tea bounding box; B represents the real tea bounding box; w represents the width of the tea image; h represents the height of the tea image; Indicates the coordinate of the upper left corner of A; Indicates the coordinate of the lower right corner of A; Indicates the coordinate of the upper left corner of B; Indicates the coordinates of the lower right corner of B; Represents the square of the Euclidean distance between the upper left corner of the predicted tea leaf bounding box and the true tea leaf bounding box; Represents the square of the Euclidean distance between the predicted tea leaf bounding box and the lower right corner of the true tea leaf bounding box.
7. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 1, characterized in that: The step S3 is specifically as follows: Dividing the data set into a training set and a validation set based on a preset ratio, training the tea grading and counting model through the training set, and continuously optimizing the hyperparameters of the tea grading and counting model including at least a random deactivation rate, a weight decay rate, and a learning rate during the training process until the loss value of the loss function is less than a preset loss threshold; The trained tea grading and counting model is verified by the verification set to determine whether the recognition accuracy is greater than a preset accuracy threshold. If so, the training is terminated; if not, the training set is expanded to continue the training.
8. A tea grading and counting method based on an improved YOLOv8n network as claimed in claim 1, characterized in that: The step S4 is specifically as follows: The tea image to be identified is obtained, and after preprocessing the tea image to be identified by image enhancement, the trained tea grading and counting model is input to grade and count the tea leaves, and the output includes at least the predicted tea boundary box, the tea grade of each tea boundary box, and the grading and counting results of the number of tea leaves.
9. The tea grading and counting method based on the improved YOLOv8n network as claimed in claim 1, characterized in that: The step S4 further comprises: The tea grading and counting model is continuously optimized based on the grading and counting results.