Rapid and accurate identification method for tea leaf picking part

Abstract

The invention discloses a rapid and accurate identification method for a tea leaf picking part, and the method comprises the steps: firstly obtaining a tea leaf image, generating a detection frame of a tea leaf tender shoot based on a light-weight improved target detection model, and carrying out the positioning of a tea leaf tender shoot picking key point and the segmentation of a picking region through a light-weight improved multi-task model at the same time. And finally, extracting a skeleton of the picking area, and carrying out key point fusion and skeleton refinement tea leaf tender shoot picking point positioning. According to the scheme of the invention, lightweight improvement is carried out based on a YOLOv8 network model, a target detection model and a multi-task model are constructed, tea leaf key point features and tender shoot stem features are automatically learned, and tea leaf tender shoot picking points are positioned in combination with instance segmentation and key point detection for tea leaf picking which is a mechanical operation requiring precision. Therefore, it is guaranteed that the picking point obtained through calculation is certainly located on the stem of the tea tender shoot, the positioning error is reduced to the maximum extent, and better generalization ability is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision based on deep learning, and specifically relates to a method for quickly and accurately identifying tea picking parts. Background Art

[0002] Current methods for locating tea picking points primarily employ traditional image processing or deep learning methods to identify the specific location of young tea shoots. While these methods can identify tea shoots and determine picking points, they still suffer from several shortcomings: First, traditional image processing methods require artificially designed features, requiring designers to repeatedly adjust parameters to meet detection requirements, and perform poorly in environments with complex backgrounds. Deep learning methods, on the other hand, suffer from large networks and slow detection speeds, making them difficult to meet the real-time requirements of the actual picking process. Furthermore, their positioning design also suffers from flaws, resulting in the inability to accurately locate the picking point on the stem of the tea shoot. Summary of the Invention

[0003] In view of the above problems, the purpose of the present invention is to provide a method for quickly and accurately identifying tea picking positions, aiming to provide accurate position guidance for automatic tea picking, thereby improving the quality of automatically picked tea.

[0004] The specific technical solutions for achieving the purpose of the present invention are as follows:

[0005] A method for quickly and accurately identifying tea picking parts comprises the following steps:

[0006] Step 1: Get tea leaves image;

[0007] Step 2: Build a lightweight and improved target detection model to generate a detection frame for the young tea leaves.

[0008] Step 3: Use a lightweight and improved multi-task model to simultaneously locate the key points for picking tea shoots and segment the picking area;

[0009] Step 4: Extract the skeleton of the picking area and locate the picking points of the young tea shoots by fusing the key points and refining the skeleton.

[0010] Compared with the prior art, the present invention has the following beneficial effects:

[0011] (1) The method of the present invention relies on the target detection model to automatically learn the characteristics of tea leaves’ young shoots, and relies on the multi-task model to automatically learn the characteristics of tea leaves’ key points and young shoot stems. Compared with the traditional tea picking point recognition method that relies on manually designed features, the method of the present invention has better generalization ability.

[0012] (2) The present invention is based on the lightweight improvement and application of the YOLOv8 network model. Compared with other network models, the model has a smaller size and faster speed while ensuring performance, and can meet the real-time requirements of the picking process;

[0013] (3) The present invention combines instance segmentation and key point detection to locate the picking point of the young tea leaves, thereby ensuring that the calculated picking point is located on the stem of the young tea leaves, thereby minimizing the positioning error.

[0014] The present invention will be further described below with reference to specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The figure is a flow chart of the method for quickly and accurately identifying tea picking parts according to the present invention.

[0016] Figure 2 Schematic diagram of a target detection model based on lightweight improvement in an embodiment of the present invention.

[0017] Figure 3 Schematic diagram of target detection annotation in an embodiment of the present invention.

[0018] Figure 4 Schematic diagram of a multi-task model in an embodiment of the present invention.

[0019] Figure 5 Schematic diagram of data annotation in a multi-task model database in an embodiment of the present invention.

[0020] Figure 6 This is a schematic diagram of tea shoot picking point positioning for extracting the picking area skeleton, fusing key points, and refining the skeleton in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Example

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0023] As used in this application and the claims, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural unless the context clearly indicates otherwise. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0024] Unless otherwise specifically stated, the relative arrangement of the parts and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to actual proportional relationships. The techniques, methods and equipment known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods and equipment should be considered as part of the authorization specification. In all examples shown and discussed here, any specific values should be interpreted as being merely exemplary and not as limitations. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following figures, and therefore, once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0025] A method for quickly and accurately identifying tea picking parts comprises the following steps:

[0026] Step 1: Get tea leaves image;

[0027] In this embodiment, the acquired tea leaf image may need to include the tea leaves (including the stems), and the image needs to be corrected for illumination;

[0028] In this embodiment, a camera is used to obtain tea images, that is, the data set is collected at different times of the day, the collection distance is maintained at 10cm-30cm, the collection equipment includes a mobile phone and a camera, and the angle between the shooting equipment and the tea leaves is maintained at 0-30°.

[0029] Specifically, the data collection time in this embodiment is from April 2024 to May 2024, and the data shooting location is Yangzhou Juyuanchun Tea Cooperative. During the shooting process, image acquisition equipment is used to shoot tea shoots of different varieties and different growth states from the front to ensure that the key points and stems of the tea shoots are clearly visible, thereby improving the detection accuracy. A single picture contains one or more tea shoots.

[0030] The illumination correction process includes:

[0031] Convert the tea image from its original format to HSV image format and extract the brightness channel;

[0032] Perform regional brightness detection on the brightness channel, detect its global brightness and central area brightness respectively, and comprehensively judge whether the image needs brightness adjustment: for too dark images, first convert them to LAB color space, then use restricted adaptive histogram equalization method for their brightness channel, and then perform adaptive Gamma correction, where the Gamma value is 0.6 when the l channel mean is less than 30, otherwise it is 0.8; for overexposed images, first convert them to YCrCb color space, separate the brightness component, and then perform adaptive highlight compression. When the color channel mean is greater than the set threshold, for example >220, linear compression is performed on pixels >200, and the scaling factor can be set to 0.8. Otherwise, mild linear compression is used, and the scaling factor can be set to 0.9. Then, detail enhancement processing based on edge-preserving filtering is performed.

[0033] Step 2: Build a lightweight and improved target detection model to generate a detection frame for the young tea leaves.

[0034] The target detection model based on lightweight improvement in this step is a lightweight YOLOv8n-detection model. The model structure constructed in this embodiment is Figure 2 As shown in the figure, it includes modules such as convolution module, Lite-C2f module and SPPF module; among them, P1-P5 are the backbone network, P1 is the convolution module, P2, P3, and P4 are composed of convolution module and Lite-C2f module, P5 is composed of Lite-C2f module and SPPF module, the neck part processes the output of P3, P4, and P5 modules, and finally the detection head outputs the detection result, and the detection box coordinate format is xyxy format;

[0035] The Lite-C2f module is a lightweight version of YOLOv8's C2f module. Specifically, the Bottleneck module in the original C2f module is replaced with an inverted residual module. This inverted residual module first uses point-by-point convolution to double the number of channels in the input feature, then uses 3×3 depth-wise convolution to extract features, and finally uses point-by-point convolution again to reduce the number of channels to the original number. This module significantly reduces the number of parameters and computation while ensuring feature extraction.

[0036] In addition, the lightweight improved target detection model in this solution is trained using images of young tea leaves. Specifically:

[0037] The tea shoot pictures are annotated using the labelme annotation tool to obtain the target detection frame annotation results, and a target detection database for tea shoot pictures is constructed. The database is divided into training set, validation set, and test set. For example, the database is divided according to a ratio of 7:2:1. In this embodiment, the tea target detection dataset constructed has 921 data in the training set, 262 data in the validation set, and 169 data in the test set. The annotated object is the tea shoot body. When annotating, ensure that the detection frame is closely connected to the tea shoot. The target detection annotation is as follows: Figure 3 As shown;

[0038] The constructed target detection model is trained based on the target detection database, and is verified and tested using the validation set and test set to obtain the weight file of the trained target detection model;

[0039] The weight file of the target detection model is loaded into the target detection model to obtain the trained target detection model, and the model is used to obtain the detection frame of the tender shoot part in the collected tea image.

[0040] Step 3: Use a lightweight and improved multi-task model to simultaneously locate the key points for picking tea shoots and segment the picking area:

[0041] Use the labelme annotation tool to annotate the tea shoot images, obtain their key points and instance segmentation labels, and build key point and instance segmentation databases respectively. Divide the training set, validation set, and test set. Like the model in step 2, the division can also be performed in a ratio of 7:2:1.

[0042] The multi-task model is trained based on the instance segmentation database and the key point database, and is verified and tested using the validation set and test set to obtain the weight file of the multi-task model;

[0043] Load the weight file of the multi-task model into the multi-task model to obtain the trained multi-task model;

[0044] Take the image within the detection box obtained by the object detection model, and then scale this part of the image to adapt to the multi-task model input;

[0045] The trained multi-task model is used to detect the extracted images to obtain the tea picking key points and picking areas.

[0046] Among them, the multi-task model structure is as follows Figure 4 As shown,

[0047] Based on the YOLOv8n-pose model, an instance segmentation branch including a neck and an instance segmentation detection head is added to the YOLOv8n-pose model to perform segmentation tasks, thereby constructing a dual-task collaborative network. Based on the original model, the multi-task model adds an instance segmentation branch including a neck and an instance segmentation detection head to perform segmentation tasks, thereby constructing a dual-task collaborative network. During training, the dual-task collaborative network simultaneously loads a key point database and an instance segmentation database to train the model, and applies the DWA loss balancing mechanism to balance the instance segmentation and key point detection tasks. After training, the weights of the multi-task model are obtained, and the multi-task model and its weights together constitute the multi-task model.

[0048] The improved model's segmentation and keypoint tasks share a common backbone network. P1-P5 form the backbone network, with P1 being a convolutional module, P2, P3, and P4 consisting of a convolutional module and a Lite-C2f module, and P5 consisting of a Lite-C2f module and an SPPF module. The outputs of P3, P4, and P5 are then processed by their respective neck components.

[0049] In this embodiment, in the tea key point data set, the training set has 1183 data, the validation set has 338 data, and the test set has 169 data. The annotated objects are the key points of one bud and one leaf of tea and the key points of one bud and two leaves. The former is located at the intersection of the first leaf of the tea shoot and its stem, and the latter is located at the intersection of the second leaf of the tea shoot and its stem. The key point annotations are as follows: Figure 5 As shown, the upper point is the key point of the tea leaves with one bud and one leaf, and the lower point is the key point of the tea leaves with one bud and two leaves.

[0050] In the tea segmentation database, the training set has 1183 data, the validation set has 338 data, and the test set has 169 data. The labeled objects are the young stems of tea leaves. The segmentation annotations are as follows: Figure 5 As shown, the shaded area is the young stem of the marked tea leaves.

[0051] In the implementation, we first use the coordinates of the tea leaves’ young shoots detection frame output by the target detection network in the previous step to extract the part of the image containing the tea leaves’ young shoots from the original image, and then scale the part of the image so that the length and width of the image become 640 pixels.

[0052] Step 4: Combine Figure 6 , extract the picking area skeleton, and locate the tea shoot picking points by fusing key points and refining the skeleton:

[0053] According to the segmentation mask of the picking key points and the picking area obtained in step 3, the picking key points may include one bud and one leaf, or one bud and two leaves key points;

[0054] The obtained segmentation mask is subjected to image refinement operation to obtain the skeleton of the tea shoot stem, and the skeleton is intercepted according to the position of the tea picking key point, and the tea shoot picking point is determined on the skeleton according to a certain ratio.

[0055] Among them, the image thinning operation uses the Zhang-Suen thinning algorithm to obtain the skeleton of the mask, which is represented by a series of continuous pixel points;

[0056] Based on the obtained picking key points, the pixel points between the two picking key points are considered as tender shoot stems;

[0057] The final picking point is confirmed on the stem of the tea leaves according to the set ratio. If the picking grade of the tea leaves is a single bud, the top of the stem of the tea leaves is taken as the single bud picking point. If the picking grade of the tea leaves is one bud and one leaf, the one bud and one leaf picking point is confirmed on the stem of the tea leaves according to a certain ratio, for example, it is set at a ratio of 0.4.

[0058] The solution of the present invention is based on a lightweight improvement of the YOLOv8 network model, constructs a target detection model, automatically learns the features of tea shoots, and relies on a multi-task model to automatically learn the key point features of tea and the stem features of tea shoots. Compared with the traditional tea picking point identification method that relies on manually designed features, the method of the present invention has better generalization ability; in addition, for tea picking, a mechanized operation that requires precision, this solution combines instance segmentation and key point detection to locate the tea shoot picking points, thereby ensuring that the calculated picking points are definitely located on the stems of the tea shoots, minimizing positioning errors.

[0059] In addition, the present solution also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0060] Step 1: Get tea leaves image;

[0061] Step 2: Build a lightweight and improved target detection model to generate a detection frame for the young tea leaves.

[0062] Step 3: Use a lightweight and improved multi-task model to simultaneously locate the key points for picking tea shoots and segment the picking area;

[0063] Step 4: Extract the skeleton of the picking area and locate the picking points of the young tea shoots by fusing the key points and refining the skeleton.

[0064] The present invention also provides a computer storable medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0065] Step 1: Get tea leaves image;

[0066] Step 2: Build a lightweight and improved target detection model to generate a detection frame for the young tea leaves.

[0067] Step 3: Use a lightweight and improved multi-task model to simultaneously locate the key points for picking tea shoots and segment the picking area;

[0068] Step 4: Extract the skeleton of the picking area and locate the picking points of the young tea shoots by fusing the key points and refining the skeleton.

[0069] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for quickly and accurately identifying tea picking parts, characterized in that: The following steps are involved: Step 1: Get tea leaves image; Step 2: Build a lightweight and improved target detection model to generate a detection frame for the young tea leaves. Step 3: Use a lightweight and improved multi-task model to simultaneously locate the key points for picking tea shoots and segment the picking area; Step 4: Extract the skeleton of the picking area and locate the picking points of the young tea shoots by fusing the key points and refining the skeleton.

2. The method for quickly and accurately identifying tea picking parts according to claim 1, characterized in that: The obtained tea leaf image needs to include the young tea leaves, and the image needs to be corrected for illumination; The illumination correction process includes: Convert the tea image from its original format to HSV image format and extract the brightness channel; The brightness channel is subjected to regional brightness detection, and its global brightness and central area brightness are detected respectively to comprehensively judge whether the image needs brightness adjustment: for too dark images, they are first converted to the LAB color space, and then the restricted adaptive histogram equalization method is used on its brightness channel, and then adaptive gamma correction is performed; for overexposed images, they are first converted to the YCrCb color space, and the brightness component is separated, and then adaptive highlight compression is performed. When the color channel mean is greater than the set threshold, the pixels are linearly compressed, otherwise light linear compression is used, and then detail enhancement processing based on edge-preserving filtering is performed.

3. The method for rapid and accurate identification of tea picking parts according to claim 1, characterized in that: The target detection model based on lightweight improvement in step 2 is a lightweight YOLOv8n-detection model, including a convolution module, a Lite-C2f module and an SPPF module; The Lite-C2f module is obtained by lightweighting the C2f module of YOLOv8. That is, the Bottleneck module in the original C2f module is replaced with an inverted residual module. The inverted residual module first uses point-by-point convolution to double the number of channels of the input features, then uses 3×3 depth-wise convolution to extract features, and finally uses point-by-point convolution to reduce the number of channels to the original number.

4. The method for quickly and accurately identifying tea picking parts according to claim 3, characterized in that: The lightweight improved target detection model in step 2 is trained using tea leaves pictures, specifically: Use the labelme annotation tool to annotate images of young tea leaves, obtain the object detection box annotation results, build an object detection database for young tea leaves images, and divide it into training, validation, and test sets; The constructed target detection model is trained based on the target detection database, and is verified and tested using the validation set and test set to obtain the weight file of the trained target detection model; The weight file of the target detection model is loaded into the target detection model to obtain the trained target detection model, and the model is used to obtain the detection frame of the tender shoot part in the collected tea image.

5. The method for rapid and accurate identification of tea picking parts according to claim 1, characterized in that: The positioning of the tea shoot picking points and the segmentation of the picking areas in step 3 are specifically as follows: Use the labelme annotation tool to annotate tea shoot images, obtain their key points and instance segmentation labels, and build key point and instance segmentation databases respectively, dividing them into training sets, validation sets, and test sets; The multi-task model is trained based on the instance segmentation database and the key point database, and is verified and tested using the validation set and test set to obtain the weight file of the multi-task model; Load the weight file of the multi-task model into the multi-task model to obtain the trained multi-task model; Take the image within the detection box obtained by the object detection model, and then scale this part of the image to adapt to the multi-task model input; The trained multi-task model is used to detect the extracted images to obtain the tea picking key points and picking areas.

6. The method for quickly and accurately identifying tea picking parts according to claim 5, characterized in that: The multi-task model is improved based on the YOLOv8n-pose model. A new instance segmentation branch including a neck and instance segmentation detection head is added to the YOLOv8n-pose model to perform the segmentation task, thereby constructing a dual-task collaborative network. During the dual-task collaborative network training, the key point database and the instance segmentation database are loaded simultaneously to train the model, and the DWA loss balancing mechanism is applied to balance the instance segmentation and key point detection tasks. After the training is completed, the weight of the multi-task model is obtained.

7. The method for quickly and accurately identifying tea picking parts according to claim 1, characterized in that: The extraction of the picking area skeleton in step 4 is specifically as follows: According to the segmentation mask of the picking key points and picking area obtained in step 3, the obtained segmentation mask is subjected to image refinement operation to obtain the skeleton of the stem of the young tea shoots, and the skeleton is intercepted according to the position of the tea picking key points, and the picking points of the young tea shoots are determined on the skeleton according to a certain ratio.

8. The method for rapid and accurate identification of tea picking parts according to claim 7, characterized in that: The segmentation mask is refined using the Zhang-Suen refinement algorithm to obtain the skeleton of the mask, which is represented by a series of continuous pixels. Based on the obtained picking key points, the pixel points between the two picking key points are considered to be the young tea shoots and stems; The final picking point is determined on the stem of the tea leaves according to the set ratio.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images