Tea fresh leaf treatment method, storage medium, controller, and sorting apparatus

Abstract

This invention discloses a method for processing fresh tea leaves, a storage medium, a controller, and a sorting device, relating to the field of fresh tea leaf sorting technology. The method includes: inputting an image of the fresh tea leaves to be processed into a deep learning-based fresh tea leaf detection model to obtain detection information for each tea leaf target in the image, wherein the detection information includes the detection box position, overall confidence level, category prediction probability, color feature value, and shape feature value; determining a coarse category for the tea leaf targets based on the category prediction probability, and further refining the category of tea leaf targets belonging to the same coarse category based on the color feature value and / or shape feature value; and performing corresponding processing operations on the tea leaf targets based on the refining category determination results. Thus, refined grading of fresh tea leaves is achieved based on a single deep learning model, which can flexibly adapt to diverse grading standards and effectively overcome the technical bottlenecks of short shelf life and difficult sample collection of fresh leaves.

Description

Technical Field

[0001] This invention relates to the field of tea leaf sorting technology, and in particular to a tea leaf processing method, storage medium, controller and sorting equipment. Background Technology

[0002] With the large-scale development of the tea industry, the harvesting method for fresh tea leaves is gradually shifting from manual to mechanized harvesting. While mechanized harvesting technology has significantly improved harvesting efficiency, it has also led to inconsistent quality of fresh leaves. During mechanized harvesting, buds and leaves of different tenderness levels (such as...) are often mixed in. Figure 1 (As shown), broken leaves, tea stems, and impurities such as sand and insects. Currently, the sorting of fresh tea leaves mainly relies on physical sieving methods such as winnowing and screening. However, these sorting methods can only roughly separate materials based on their size or weight, making it difficult to effectively remove impurities that are similar in shape and weight to qualified fresh leaves. Furthermore, they cannot perform fine classification of buds and leaves of different grades, seriously affecting the uniformity and market value of the final tea product.

[0003] In recent years, deep learning technology has made significant progress in image segmentation, providing a possibility for the accurate segmentation of adhered tea leaves. However, in practical production applications, the widespread adoption of deep learning models still faces the following bottlenecks: different customers or different tea types have different grading standards for fresh leaves (such as single bud, one bud and one leaf, one bud and two leaves, etc.), making it difficult for models trained for specific grading tasks to be directly applied to other scenarios. To meet diverse needs, a large number of samples need to be collected and modeled for each grading standard. However, as fresh materials, tea leaves have high water content and strong respiration, making them extremely prone to wilting and turning red after harvesting, with their appearance and color changing significantly in a very short time. This characteristic means that during the fresh leaf quality preservation period, it is impossible to collect sufficient and representative training samples for each grading standard, thus severely restricting the large-scale application of deep learning technology in the field of tea leaf grading. Summary of the Invention

[0004] The purpose of this invention is to propose a method for processing fresh tea leaves, a storage medium, a controller, and a sorting device to achieve refined grading of fresh tea leaves based on a single deep learning model. This method can flexibly adapt to diverse grading standards and effectively overcome the technical bottlenecks of short shelf life and difficulty in sample collection.

[0005] In a first aspect, embodiments of the present invention propose a method for processing fresh tea leaves, comprising the following steps: inputting an image of fresh tea leaves to be processed into a deep learning-based tea leaf detection model to obtain detection information for each tea leaf target in the image, wherein the detection information includes detection box position, overall confidence level, category prediction probability, color feature value, and shape feature value; determining a coarse category of the tea leaf target based on the category prediction probability, and further refining the category of tea leaf targets belonging to the same coarse category based on the color feature value and / or the shape feature value; and performing corresponding processing operations on the tea leaf targets based on the refining category determination results.

[0006] In some embodiments, the color feature value includes the predicted mean of at least one of hue H, saturation S, and brightness V, and the shape feature value is the predicted area ratio of the tea leaf target within the detection frame.

[0007] In some embodiments, the tea leaf detection model is built on an improved YOLOv5s network, and its output layer is used to output the detection box position, overall confidence, category prediction probability, color feature value, and shape feature value.

[0008] In some embodiments, the training process of the tea leaf detection model includes: acquiring an annotated image, the annotated image containing detection boxes for tea leaf targets and corresponding coarse classification labels; for each detection box, extracting color feature values ​​and shape feature values ​​of the tea leaf targets within the detection box as additional labels; and training the tea leaf detection model using the annotated image, the coarse classification labels, and the additional labels, wherein the loss function of the tea leaf detection model includes bounding box regression loss, target confidence loss, classification loss, color feature loss, and shape feature loss.

[0009] In some embodiments, the color feature loss includes hue loss. saturation loss and brightness loss The shape feature loss is an area loss. The loss function of the tea leaf detection model. Represented as:

[0010] in, For bounding box regression loss, For target confidence loss, For classifying losses, , , , , , , This represents the corresponding loss weight coefficient.

[0011] In some embodiments, the coarse classification includes at least one of buds, leaves, stems, and impurities; the sub-classification determination includes: for tea leaves whose coarse classification is buds, subdividing them into at least one of the following categories based on the color feature value and / or shape feature value of the tea leaves: whole bud, one bud and one leaf, one bud and two leaves, and one bud and multiple leaves.

[0012] In some embodiments, the processing operation includes: generating control instructions based on the subdivision classification result to drive the spray valve to sort the fresh tea leaves into the hopper corresponding to the subdivision classification result.

[0013] Secondly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the tea leaf processing method described in the first aspect embodiment.

[0014] Thirdly, embodiments of the present invention provide a controller, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the tea leaf processing method described in the first aspect embodiment.

[0015] Fourthly, this invention provides a tea leaf sorting device, characterized in that it includes: a feeding device, a conveyor belt, a camera, a spray valve, a hopper, and a main control component; wherein, the feeding device is used to spread and flatten the tea leaves on the conveyor belt for transport; the camera is used to capture images of the tea leaves on the conveyor belt; the main control component is used to receive the tea leaf images, execute the tea leaf processing method described in the first aspect embodiment, output a subdivision classification result, and generate control commands based on the subdivision classification result to drive the spray valve to actuate and sort the tea leaves on the conveyor belt into the hopper corresponding to the subdivision classification result.

[0016] The tea leaf processing method, storage medium, controller, and sorting device of this invention, when sorting tea leaves, first input the image of the tea leaves to be processed into a deep learning-based tea leaf detection model to obtain detection information for each tea leaf target in the image. This detection information includes the detection box position, overall confidence level, category prediction probability, color feature value, and shape feature value. Then, based on the category prediction probability, a coarse category is determined for the tea leaf targets. Based on the color feature value and / or shape feature value, tea leaf targets belonging to the same coarse category are further subdivided into smaller categories. Finally, based on the subdivided category determination results, corresponding processing operations are performed on the tea leaf targets. Therefore, refined grading of tea leaves can be achieved based on a single deep learning model, flexibly adapting to diverse grading standards and effectively overcoming the technical bottlenecks of short shelf life and difficult sample collection for fresh leaves.

[0017] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the classification of fresh tea leaves, as an example of the present invention. Figure 2 This is a flowchart of a method for processing fresh tea leaves according to an embodiment of the present invention; Figure 3 This is a network architecture diagram of a tea leaf detection model according to an embodiment of the present invention; Figure 4 This is a structural block diagram of a controller according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a tea leaf sorting device according to an embodiment of the present invention; Figure 6 This is a thread distribution diagram of the main control component according to an embodiment of the present invention; Figure 7 This is a data flow diagram of each sub-thread in one embodiment of the present invention. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0020] Machine-harvested tea leaves often contain buds and leaves of different grades as well as impurities. Traditional winnowing and sorting methods can only roughly separate leaves based on size and weight, making it difficult to perform precise classification. While deep learning can accurately segment sticky leaves, different customers have different grading standards, and the short shelf life of fresh leaves makes it impossible to collect sufficient samples to model for each standard, which severely restricts its large-scale application.

[0021] To address this issue, this invention proposes a method for processing fresh tea leaves, a storage medium, a controller, and a sorting device. By extracting the color and shape feature values ​​of fresh tea leaves, the grading process is decoupled into two levels: coarse classification based on category probability and fine classification based on fine-grained features. This method, based on a single deep learning model, introduces flexibly adjustable fine-classification criteria. It can effectively adapt to the grading needs of different customers without requiring the re-collection of large numbers of samples for each grading standard, thus overcoming the technical bottlenecks of short fresh leaf shelf life and difficult sample collection.

[0022] The following description, with reference to the accompanying drawings, outlines an embodiment of the tea leaf processing method, storage medium, controller, and sorting equipment of the present invention.

[0023] Figure 2 This is a flowchart of a method for processing fresh tea leaves according to an embodiment of the present invention.

[0024] like Figure 2 As shown, the method for processing fresh tea leaves includes the following steps: S11, input the image of the tea leaves to be processed into the deep learning-based tea leaf detection model to obtain the detection information of each tea leaf target in the image. The detection information includes the detection box position, comprehensive confidence, category prediction probability, color feature value and shape feature value.

[0025] Specifically, images (e.g., RGB format) of fresh tea leaves to be processed (e.g., tea leaves laid flat on a conveyor belt to be sorted) are acquired using an image acquisition device (e.g., a camera), and the images are input into a pre-trained deep learning-based tea leaf detection model. This model can employ a convolutional neural network structure, such as Faster R-CNN, YOLO, Mask R-CNN, etc., and is trained on a tea leaf dataset.

[0026] The model extracts features from the input image and outputs detection information for each tea leaf target through a detection head network, which may include: Detection box position: such as locating the region of each fresh tea leaf target in the image using rectangular box coordinates (x, y, w, h); Overall confidence level: Characterizes the probability that the detection box contains fresh tea leaves, and the value can range from 0 to 1; Category prediction probability: The probability distribution of the model's prediction of the coarse category (such as buds, leaves, tea stems, impurities, etc.) to which the target tea leaves belong; Color feature values: color statistical features extracted within the detection box area, such as the mean and variance of the HSV color space, or the hue histogram, used to characterize the freshness and tenderness of the tea leaves. Shape feature values: Geometric morphological features extracted within the detection box area, such as aspect ratio, perimeter, area, and roundness, are used to characterize the morphological information of fresh tea leaves, such as the number of unfolded leaves and the degree of curling.

[0027] In practical applications, the model can be trained end-to-end on a labeled dataset of fresh tea leaves, enabling the detection head to output both location information and multidimensional feature information simultaneously.

[0028] This step, building upon traditional object detection, introduces color and shape feature values ​​as simultaneous outputs. This allows the model to not only locate and identify fresh tea leaves but also extract fine-grained features for subsequent refined grading. The bounding box position provides spatial localization for subsequent processing; the overall confidence score can be used to filter out low-quality detection results; the category prediction probability supports coarse classification; and color and shape features provide key discriminative criteria for further subdividing the category.

[0029] S12, determine the coarse category of the tea leaf target based on the category prediction probability, and further classify the tea leaf targets belonging to the same coarse category based on color feature value and / or shape feature value.

[0030] The rough classification may include at least one of buds, leaves, stems, and impurities.

[0031] Specifically, the category with the highest probability value can be selected as the coarse classification category for the tea leaf target. For example, when the probability of the "bud" category is the highest, the tea leaf target is coarsely classified as "bud"; when the probability of the "piece" category is the highest, it is coarsely classified as "piece"; when the probability of the "stem" or "impurity" category exceeds the set threshold, it is coarsely classified as "stem and impurity" and can be directly triggered to remove it. For example, when the target reaches the removal position, the removal mechanism (such as a high-pressure air nozzle) is activated to blow it away from the conveyor belt and into the waste collection box, ensuring the purity of the tea leaves entering the subsequent processing stage.

[0032] For tea leaf targets belonging to the same coarse category, further sub-categorization can be determined based on the color and / or shape feature values ​​output by S11. The sub-categorization determination can employ one of the following two methods or a combination thereof: The rule-based threshold approach: Preset feature value ranges for different coarse classification categories. For example, for a target coarsely classified as "bud", the freshness is judged based on color feature values ​​(such as green saturation), and whether leaves have unfolded and the number of unfolded leaves are judged based on shape feature values ​​(such as aspect ratio and outline area), thus subdividing it into different levels such as single bud, one bud and one leaf, and one bud and two leaves; for a target coarsely classified as "piece", single leaf, broken leaf, or old leaf is distinguished based on shape feature values ​​(such as area and aspect ratio); The classifier-based approach combines color and shape feature values ​​into a feature vector, which is then input into a pre-trained fine-grained classifier (such as a support vector machine or a shallow neural network) to output detailed category labels. This classifier can be trained on a small number of samples and can be flexibly adjusted according to different customers' grading standards.

[0033] The results of the subcategorization can include grade labels (such as "premium buds" or "first-grade buds") or continuous quality scores, which can be used for the corresponding processing operations in the subsequent step S13.

[0034] This step decouples the grading process of fresh tea leaves into two levels: "coarse classification" and "fine classification." Coarse classification, based on the category probabilities output by a deep learning model, efficiently distinguishes between broad categories such as buds, leaves, and stems. Fine classification, based on color and shape features, accurately subdivides different grades within the same broad category (e.g., single bud, one bud and one leaf). This two-level grading strategy has the following advantages: First, the fine classification criteria can be flexibly adjusted according to customer needs (e.g., modifying feature thresholds or replacing the fine classifier) ​​without retraining the entire deep learning model. Second, the introduction of color and shape features improves the accuracy of identifying adhered buds and leaves and similarly shaped impurities. Third, a single model can adapt to diverse grading standards, effectively overcoming the technical bottlenecks of short fresh leaf shelf life and difficult sample collection.

[0035] S13, based on the results of the sub-category determination, perform the corresponding processing operations on the target fresh tea leaves.

[0036] Specifically, the processing operation can be implemented by the controller driving the actuator of the sorting device, and may include the following methods: 1) Graded Collection Operation: When the sub-categorization result is different grades of buds and leaves (e.g., single bud, one bud and one leaf, one bud and two leaves), the controller calculates the real-time position of the tea leaf target on the conveyor belt based on the coordinates of the detection frame. When the target is conveyed to the corresponding collection area, the controller activates the high-pressure air nozzle or mechanical lever to accurately blow or push the target into the corresponding collection channel, realizing the automatic classification and collection of tea leaves of different grades.

[0037] 2) Re-inspection or marking operation: For targets with low confidence in subcategories or at the grade boundary, they can be sent to the re-inspection area for secondary identification, or markings can be superimposed on the image for manual review, or their location and grade information can be recorded in the database for subsequent quality traceability.

[0038] 3) Parameter dynamic adjustment operation: The controller can calculate the proportion of fresh tea leaves of each grade according to the real-time sorting results. When the proportion of a certain grade deviates from the preset range, the judgment threshold of the sub-category (such as the color feature threshold) is automatically adjusted to achieve dynamic optimization of the sorting standard.

[0039] In practical applications, the above processing operations can be performed individually or in combination. The controller receives the subdivision classification results output by S12 and the detection frame position output by S11, and combines them with the feedback signal from the conveyor belt speed encoder. Through a precise delay control algorithm, it ensures that the actuator is accurately triggered when the target reaches the designated position.

[0040] This step translates the detailed category determination results into specific physical sorting actions, achieving automated and refined processing of fresh tea leaves. The grading and collection operation ensures that buds and leaves of different grades enter their corresponding subsequent processing steps, improving the uniformity of the raw materials; the impurity removal operation effectively guarantees the purity of the fresh tea leaves, laying the foundation for high-quality finished tea; the re-inspection marking and parameter adjustment operations enhance the system's adaptability and robustness. This step transforms the preceding image recognition results into actual sorting benefits, ultimately reflecting the recognition accuracy of the deep learning model in the increased market value of fresh tea leaves, completing a full closed loop from "accurate identification" to "accurate sorting."

[0041] In some embodiments of the present invention, the sub-category determination in step S12 above includes: for tea leaves that are coarsely classified as buds, subdividing them into at least one of the following categories based on the color feature value and / or shape feature value of the tea leaves: whole bud, one bud and one leaf, one bud and two leaves, and one bud and multiple leaves.

[0042] For example, the color feature value includes the predicted mean of at least one of hue (H), saturation (S), and value (V), and the shape feature value is the predicted area ratio of the tea leaf target within the detection box.

[0043] Specifically, hue (H) reflects the color type of fresh tea leaves. Tender buds usually have a light green hue (H value can be in the range of 60°~90°). As the leaves age and wilt, the H value gradually shifts towards yellowish-green or dark green (in the range of 90°~120°). Saturation (S) reflects the vividness of the color. Tender buds and leaves have a higher S value (full color). The S value decreases when the leaves are wilted or aged. Brightness (V) reflects the gloss of the leaves. Fresh buds and leaves have a moderate to bright V value. The V value decreases after the leaves lose water and wilt.

[0044] The shape feature is the predicted area ratio of the tea leaf target within the detection box, calculated as: Area Ratio = Target Pixel Area of ​​Tea Leaf / Total Area of ​​Detection Box. This feature characterizes the degree of bud and leaf unfolding and the number of leaves. A whole bud target has a compact shape, with unfurled leaves, and a small area ratio (e.g., <0.3); a one-bud-one-leaf target shows the outline of the bud connected to a tender leaf, with a moderate area ratio (e.g., in the range of 0.3~0.6); a one-bud-two-leaf or more target has more unfolded leaves, higher outline complexity, and a larger area ratio (e.g., >>0.6).

[0045] Based on the above color and shape feature values, the determination of subcategories can be achieved using one of the following two methods or a combination thereof: Method 1: Judgment based on rule thresholds Pre-define the feature value range for different subcategories. For example: if the area percentage is less than Th1 and the H value is within the interval [H1, H2], it is determined to be a whole bud; if Th1 ≤ area percentage < Th2 and the H value is within the interval [H2, H3], it is determined to be a one-bud-one-leaf bud; if the area percentage is greater than or equal to Th2, further analyze the contour complexity: calculate the ratio of the convex hull area of ​​the target contour to the original area. If the ratio is close to 1, the contour is compact and can be subdivided into one bud and two leaves; if the ratio is small, the contour is complex and can be subdivided into one bud and multiple leaves.

[0046] The thresholds Th1, Th2, and H value ranges can be flexibly adjusted according to different tea types (such as green tea and black tea) or customer needs. For example, for premium green tea, a stricter H value range can be set to select tender buds and leaves; for bulk tea, the threshold range can be appropriately relaxed.

[0047] Method 2: Classifier-based determination Color feature values ​​(mean values ​​of H, S, and V) and shape feature values ​​(area percentage) are combined into a multi-dimensional feature vector, which is then input into a pre-trained fine-grained classifier. The fine-grained classifier can employ lightweight machine learning models such as support vector machines, random forests, or shallow neural networks, and is trained on a small number of labeled samples. Due to the low dimensionality and clear physical meaning of the input features, the classifier can achieve high accuracy on tens to hundreds of samples, and its training speed is fast, allowing for retraining or fine-tuning at any time based on new grading standards.

[0048] Method 3: Combining rules and classifiers A two-stage strategy of "rule-based initial screening + classifier-based precise judgment" is adopted: first, a large number of targets are quickly judged using rule thresholds; for targets whose feature values ​​are near the boundary or whose rule judgment confidence is low, a classifier is then called for precise identification. This strategy balances processing speed and judgment accuracy.

[0049] After the sub-category determination is completed, the sub-category determination result can be output as a grade label (such as "whole bud - special grade" or "one bud and one leaf - grade one") or a continuous quality score (such as 0~100 points) for the subsequent step S13 to perform the corresponding processing operations.

[0050] The above-mentioned detailed classification method enables a refined grading of fresh tea leaves in the bud stage.

[0051] In some embodiments of the present invention, the tea leaf detection model is built on an improved YOLOv5s network, whose output layer is used to output the detection box position, overall confidence, category prediction probability, color feature value and shape feature value.

[0052] In other embodiments of the present invention, the tea leaf detection model can also be constructed based on improved networks such as Faster R-CNN and Mask R-CNN, whose output layer is used to output the detection box position, comprehensive confidence, category prediction probability, color feature value and shape feature value.

[0053] In some embodiments of the present invention, fully supervised learning training is employed. The training process of the tea leaf detection model includes: acquiring labeled images, which contain detection boxes for tea leaf targets and corresponding coarse classification labels; for each detection box, extracting the color feature value and shape feature value of the tea leaf target within the detection box as additional labels; and training the tea leaf detection model using the labeled images, coarse classification labels, and additional labels. The loss function of the tea leaf detection model includes bounding box regression loss, target confidence loss, classification loss, color feature loss, and shape feature loss.

[0054] In other embodiments of the present invention, transfer learning is employed to train the tea leaf detection model. The training process includes: acquiring a publicly available large-scale image dataset (such as ImageNet) or an agricultural image dataset in a related field, and pre-training the backbone network of the tea leaf detection model to enable it to have general feature extraction capabilities.

[0055] Subsequently, a small number of labeled images of fresh tea leaves were acquired. These images contained bounding boxes for the tea leaf targets and corresponding coarse classification labels. For each bounding box, color and shape feature values ​​were extracted as additional labels. These labeled images were then used to fine-tune the pre-trained model. During fine-tuning, some low-level parameters of the backbone network were frozen, and only the high-level feature extraction layers and the detection head were updated.

[0056] The loss function of the fresh tea leaf detection model includes bounding box regression loss, target confidence loss, classification loss, color feature loss, and shape feature loss. Since the backbone network already possesses excellent feature extraction capabilities, the fine-tuning process only requires a small number of samples to converge, effectively reducing the need for large-scale labeled data.

[0057] In some embodiments of the present invention, color feature loss includes hue loss. saturation loss and brightness loss Shape feature loss is area loss Loss function of the fresh tea leaf detection model Represented as:

[0058] in, For bounding box regression loss, For target confidence loss, For classifying losses, , , , , , , This represents the corresponding loss weight coefficient.

[0059] In other embodiments of the present invention, in order to enhance the discriminative power of color and shape features, a contrastive learning branch is introduced on the basis of the above-mentioned basic loss function to construct a contrastive loss. The loss function is expressed as:

[0060] Among them, comparative loss The calculation method is as follows: Positive sample pairs (different enhanced views of the same fresh tea leaf target) and negative sample pairs (views of different targets or different images) are selected from the training batch. Information Noise-Contrastive Estimation (InfoNCE) loss or Normalized Temperature-Scaled Cross Entropy (NT-Xent) loss is used to narrow the feature distance of positive sample pairs and widen the feature distance of negative sample pairs. The feature distance can be calculated based on the concatenation of color and shape feature vectors (e.g., directly calculating the Euclidean distance of the concatenated multidimensional vector), or it can be calculated separately and then weighted and fused (i.e., a weighted sum of color and shape distances, with weights preset or adaptively adjusted according to the actual scenario).

[0061] The introduction of contrastive loss enables the model to learn more robust and discriminative feature representations with limited labeled samples, which is especially suitable for distinguishing sticky tea leaves and fine classification of different grades of buds and leaves.

[0062] In some embodiments of the present invention, the processing operation includes: generating control instructions based on the subdivision classification result to drive the spray valve to sort the fresh tea leaves into the bin corresponding to the subdivision classification result.

[0063] In other embodiments of the present invention, the processing operation includes: determining the grasping path and posture of the robotic arm based on the sub-category determination result and the position coordinates of the detection box, driving the robotic arm to move to the location of the tea leaf target, grasping the target through an end effector (such as a flexible gripper or suction cup), and placing it into the collection container corresponding to the sub-category.

[0064] The following section describes the technical solution of this invention in detail, taking the tea leaf detection model built on an improved YOLOv5s network and trained using fully supervised learning and a basic loss function as an example.

[0065] 1) Training data preparation and feature extraction First, labeled images of fresh tea leaves are acquired. Each image contains manually marked bounding boxes and corresponding coarse classification labels (buds, leaves, stems, impurities). For each bounding box, the color and shape feature values ​​of the tea leaf targets within the box are further calculated as additional labels. The specific calculation method is as follows: Color feature extraction: The target image of fresh tea leaves within the detection box is converted to the HSV color space. The background is removed by the image segmentation algorithm. The mean hue (H), mean saturation (S), and mean brightness (V) of all pixels in the target area are calculated. The three components are normalized to the [0,1] interval to obtain the color feature vector (h,s,v).

[0066] Shape feature extraction: The tea leaf target within the detection box is binarized and segmented. The target pixel area is calculated and normalized by dividing it by the total area of ​​the manually marked box to obtain the shape feature value. (Value range 0~1), this feature represents the area ratio of fresh tea leaves within the detection box.

[0067] Finally, the complete feature vector corresponding to each detection box is denoted as (h,s,v, During the training of the tea leaf detection model, these calculated features, along with the images, are input into the model as supervisory signals.

[0068] 2) Network structure and loss function of the tea fresh leaf detection model The network structure of the tea fresh leaf detection model is as follows: Figure 3 As shown. To enable feature vectorization, this structure optimizes the official YOLOv5s network, with the main optimizations being the output layer and the loss function.

[0069] Specifically, the output format of the output layer is as follows: .in, The coordinates of the top left and bottom right corners of the detection box. To assess the overall confidence level, For category The predicted probability, The predicted mean in the HSV of the material within the prediction box (normalized to 0~1 by sigmoid). To predict the area of ​​the material within the box (normalized to 0~1 using sigmoid).

[0070] For sorting fresh tea leaves according to this invention, the optimized YoloV5S network outputs categories as buds, leaves, stems, and impurities. These category divisions are largely unambiguous and do not require frequent modeling for different samples. For further subdivision of buds, such as whole bud, one bud and one leaf, the network outputs h, s, v, ... The threshold value is adjusted to meet the specific needs of buds such as whole buds and one bud and one leaf.

[0071] The loss function is: .

[0072] The specific definitions of each loss term are as follows: Bounding box regression loss measures the positional deviation between the predicted box and the ground truth box. ,in, This is an empirical coefficient; , Indicates area, For the model's predicted bounding box, These are manually labeled boxes (i.e., actual boxes); , The coordinates of the center of the prediction box. The coordinates of the center of the manually labeled box; , and The maximum and minimum x-coordinates of the predicted bounding box and the manually labeled bounding box are given. and The maximum and minimum ordinates of the predicted bounding box and the manually labeled bounding box; , and To predict the width and length of the bounding box, and Define the width and length of the manual marker box.

[0073] Target confidence loss measures the accuracy of predicting whether a target exists in the prediction frame. ,in The total number of anchor boxes across all scale feature maps; No. Confidence predictions for each anchor box (model output, not processed by sigmoid); The sigmoid activation function ( ); For the first The true confidence score of each anchor box, representing positive samples (predicted boxes assigned to the true target). =1, negative samples (predicted bounding boxes that did not match the true target). =0, ignore the sample and do not participate in the calculation.

[0074] Classification loss measures the deviation between the predicted probability of the target class and the true class. ,in, This represents the number of positive sample anchor frames; This represents the total number of data categories. No. The positive sample anchor frame for the first The predicted score for the class.

[0075] Material color loss measures the deviation of the color of the material in the marked box from that in the predicted box. ,in, This represents the number of positive sample anchor frames. The hue value predicted by the network. The hue value is calculated during feature extraction for annotation.

[0076] Material saturation loss measures the deviation between the saturation of the material in the marked box and the saturation of the material in the predicted box. ,in, This represents the number of positive sample anchor frames. This represents the saturation value predicted by the network. This is the saturation value calculated during feature extraction during annotation.

[0077] Material brightness loss measures the deviation between the brightness of the material within the marked box and the brightness of the material within the predicted box. ,in, This represents the number of positive sample anchor frames. The brightness value predicted by the network. This refers to the brightness value calculated during feature extraction during annotation.

[0078] Material area loss measures the deviation between the area of ​​material within the marked box and the area of ​​material within the predicted box. ,in, This represents the number of positive sample anchor frames. The area value predicted by the network. This is the area value calculated during feature extraction during annotation.

[0079] , , , , The loss weighting coefficient can be configured as follows: , , , , In practical applications, the above weights can be adjusted appropriately according to different tea leaf varieties or grading accuracy requirements.

[0080] During model training, labeled images and corresponding manually extracted feature vectors (h,s,v, ...) are used. The inputs are fed into the network together and trained using an end-to-end multi-task learning approach. The optimizer can be Adam, and the initial learning rate can be set to 0.001, which gradually decreases with each training round.

[0081] During model inference, the input is an image of fresh tea leaves to be processed. After forward propagation, the network directly outputs the bounding box position, overall confidence score, coarse classification probability, and predicted color feature values ​​(h, s, v) and shape feature values ​​(h, s, v). This output information can be directly used for further category determination (such as achieving subdivision of whole buds, one bud and one leaf, etc. through threshold adjustment), without the need for additional feature calculation, thus meeting the requirements of real-time sorting.

[0082] For the tea leaf sorting application of this invention, the optimized YOLOv5s network output coarse classification is set to four categories: buds, leaves, stems, and impurities. These classifications are largely unambiguous and can be stably applied to different sample scenarios without requiring frequent remodeling for different customers. For more detailed bud classifications (e.g., whole bud, one bud and one leaf, one bud and two leaves), the network output h, s, v, ... The values ​​are adjusted using thresholds to flexibly adapt to diverse grading standards. By introducing supervision based on physically meaningful color and shape features, the model can still learn robust feature representations even with limited labeled samples, overcoming the technical bottlenecks of short shelf life and difficult sample collection for fresh leaves.

[0083] In summary, the tea leaf processing method proposed in this invention has the following beneficial effects: 1) By improving the deep learning model, color and shape feature values ​​are simultaneously quantified in addition to the output material location and coarse classification. When faced with different customers' grading requirements for bud grades, only the judgment thresholds of color and shape features need to be adjusted for flexible adaptation. There is no need to re-collect a large number of samples for each grading standard to build a model. This effectively overcomes the technical bottlenecks of short shelf life of fresh leaves and difficulty in sample collection, and greatly improves the model's scenario adaptability and reusability.

[0084] 2) The optimization algorithm of this invention only adds a small number of feature branches to the YOLOv5s output layer, and only adds a minimal amount of computation during the inference stage, thus not imposing high computational demands on the Artificial Intelligence (AI) platform. This makes the algorithm more suitable for deployment on a Neural-network Processing Unit (NPU) platform, offering significant advantages in cost and power consumption compared to platforms that rely on Graphics Processing Units (GPUs), facilitating large-scale application in industrial sorting equipment.

[0085] 3) The deep learning network is designed with a general architecture, and the model on the deployment side remains unchanged. When facing different hierarchical requirements, only the feature extraction algorithm needs to be adjusted on the training side and the model needs to be retrained. The deployment side does not need to frequently change the code and network structure, which greatly reduces the maintenance cost and version management complexity of the field equipment.

[0086] 4) The model can output the detection box position, coarse classification probability, color feature value and shape feature value simultaneously in a single forward inference, without the need for an additional feature extraction module, thus meeting the real-time requirements of high-speed sorting production lines.

[0087] 5) Color and shape characteristics have clear physical meaning and quantitative range, providing a quantifiable data basis for establishing tea leaf grading standards and promoting the development of grading towards digitalization and intelligence.

[0088] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the tea leaf processing method of the above embodiments.

[0089] Figure 4 This is a structural block diagram of a controller according to an embodiment of the present invention.

[0090] like Figure 4As shown, the controller 400 includes a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, for example, via a bus 402. Optionally, the controller 400 may also include a transceiver 404. It should be noted that in practical applications, the transceiver 404 is not limited to one, and the structure of the controller 400 does not constitute a limitation on the embodiments of the present invention.

[0091] Processor 401 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 401 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0092] Bus 402 may include a pathway for transmitting information between the aforementioned components. Bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 402 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0093] The memory 403 stores a computer program corresponding to the tea leaf processing method of the above embodiments of the present invention. This computer program is controlled and executed by the processor 401. The processor 401 executes the computer program stored in the memory 403 to implement the content shown in the aforementioned method embodiments. Figure 4 The controller 400 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0094] Figure 5 This is a schematic diagram of the structure of a tea leaf sorting device according to an embodiment of the present invention.

[0095] like Figure 5As shown, the tea leaf sorting equipment 500 includes: a feeding device 501, a conveyor belt 502, a camera 503, a spray valve 504, a hopper 505, and a main control unit 506.

[0096] The feeding device 501 is used to spread the fresh tea leaves out and lay them flat on the conveyor belt 502 for transport; the camera 503 is used to capture images of the fresh tea leaves on the conveyor belt 502; the main control unit 506 is used to receive the images of the fresh tea leaves, execute the fresh tea leaf processing method of the above embodiment, output the subdivision classification judgment result, and generate control commands based on the subdivision classification judgment result to drive the spray valve 504 to act, and sort the fresh tea leaves on the conveyor belt 502 into the hopper 505 corresponding to the subdivision classification judgment result.

[0097] Specifically, the feeding device 501 is used to evenly spread and flatten the fresh tea leaves to be sorted onto the conveyor belt 502. The feeding device 501 can adopt a vibrating feeder or a vibrating screen structure. By adjusting the vibration frequency and amplitude, the stacked and sticky fresh tea leaves are gradually dispersed under the action of vibration, reducing the probability of stacking between leaves and ensuring that the fresh tea leaves are presented as single-layered and separated as possible during subsequent image acquisition. The discharge port height of the feeding device 501 can be adjusted according to the variety and size of the fresh tea leaves to control the spreading thickness.

[0098] The conveyor belt 502 can be a conveyor belt moving at a constant speed, sequentially transporting the flattened fresh tea leaves to the shooting area of ​​the camera 503 and the execution area of ​​the spray valve 504. The color of the conveyor belt 502 is preferably a color with high contrast to the fresh tea leaves (such as dark blue or black) to facilitate subsequent image segmentation. An encoder can be installed on the conveyor belt 502 to provide real-time feedback on the conveyor belt's running speed and position information, providing a precise delay control reference for the main control component 506.

[0099] Camera 503 is used to acquire images of fresh tea leaves on conveyor belt 502. Camera 503 can be a color line scan camera or a color area scan camera, selected according to sorting accuracy and speed requirements. Line scan cameras are suitable for high-speed continuous shooting, with a scanning frequency of up to several kilohertz per line, capable of synchronizing with the speed of conveyor belt 502 to generate distortion-free high-resolution images; area scan cameras are suitable for smaller shooting areas, capturing the entire image at once. The optical parameters of camera 503 (such as gain, bias, and correction coefficients) can be remotely set and adjusted via the main control unit 506. A light source (such as an LED linear light source) is arranged around camera 503 to ensure uniform illumination of the shooting area and reduce the impact of shadows and reflections on image quality.

[0100] The spray valves 504 can be a high-pressure air valve array arranged along the width of the conveyor belt 502, with each spray valve corresponding to a sorting channel. The number and spacing of the spray valves 504 can be set according to the sorting accuracy requirements, such as 1 to 2 spray valves per millimeter. When the fresh tea leaves are conveyed to the execution area of ​​the spray valve 504, the main control unit 506 calculates the precise time when the target arrives at the spray valve 504 based on the subdivision classification result and the position of the detection frame, and triggers the corresponding spray valve 504 to blow the target into the corresponding hopper 505 through high-pressure airflow. The trigger delay of the spray valve 504 can be finely adjusted by the main control unit 506 to adapt to different conveyor belt speeds or target position deviations.

[0101] The material bin 505 may include multiple independent collection bins, each corresponding to a different sub-category. For example, it may include bins for whole buds, one bud and one leaf, one bud and multiple leaves, leaves, and stems and other materials. A guide vane may be installed at the inlet of each bin 505 to ensure that the target material blown away by the spray valve 504 accurately falls into the corresponding bin. A weight sensor may be installed at the bottom of the bin 505 to monitor the collection volume of fresh tea leaves of each grade in real time, facilitating statistical analysis and feedback control.

[0102] The main control unit 506 is the control core of the sorting equipment. It can be an industrial computer or an embedded controller, and has functions such as image acquisition, recognition and analysis, spray valve control, and parameter setting. The main control unit 506 can be connected to the camera 503 through a high-speed interface (such as CameraLink, GigE, or USB3.0) to acquire images of fresh tea leaves in real time; it can be connected to the spray valve 504 through a digital I / O interface to output control commands; and it can be connected to the drive motor of the feeding device 501 and the conveyor belt 502 through a serial port or Ethernet to achieve linkage control.

[0103] The parameter settings can include: camera parameter settings (gain, bias, correction coefficient), spray valve parameter settings (delay adjustment, jet duration), feeding device parameter settings (vibration frequency, amplitude), conveyor belt speed settings, etc., which can be adjusted in real time by the operator through the human-machine interface.

[0104] In some embodiments, the software architecture of the main control component 506 is as follows: Figure 6 and Figure 7 As shown, a multi-threaded parallel processing mechanism is adopted, which includes 5 threads and 3 cache areas. The specific implementation is as follows: Cache design: Three cache areas are allocated: display cache, recognition cache, and recognition result cache. The display cache stores the image data to be displayed; the recognition cache stores the image data to be recognized and the corresponding line number information; and the recognition result cache stores the recognition analysis results.

[0105] Thread Design: One main thread and four sub-threads. The main thread is responsible for system initialization, parameter configuration, thread management, and human-computer interaction interface response. Sub-thread 1 (display thread) is responsible for reading image data from the display cache and displaying it on the main interface in real time. Operators can monitor the conveying status and sorting effect of fresh tea leaves through the display interface. Sub-thread 2 (acquisition thread) is responsible for acquiring image data from camera 503 and storing the image data in both the display cache and the recognition cache. When storing in the recognition cache, the corresponding row number information of the image is also recorded to facilitate subsequent accurate positioning. Sub-thread 3 (recognition thread) is responsible for reading image data and the corresponding row number from the recognition cache, calling the pre-trained fresh tea leaf detection model for inference analysis, executing the fresh tea leaf processing method of the above embodiment of the present invention, and outputting the subdivided category judgment result of each fresh tea leaf target. The recognition thread associates the judgment result with the row number information and stores it in the recognition result cache. Sub-thread 4 (control thread) is responsible for reading the recognition results from the recognition result cache, combining the encoder feedback signal of the conveyor belt 502, calculating the precise delay of each target reaching the execution area of ​​the spray valve 504, and generating control commands to send to the spray valve 504 to drive the corresponding spray valve to sort the fresh tea leaves into the corresponding hopper 505.

[0106] In some examples, the main control unit 506 can also provide an offline analysis module, including single-image acquisition, intelligent learning, and single-image simulation functions. The single-image acquisition function can be used for on-site debugging and sample collection; the intelligent learning function supports adjusting fine classification criteria or incrementally training the model on a small number of new samples; and the single-image simulation function can simulate sorting effects offline, facilitating parameter optimization.

[0107] The tea leaf sorting equipment of the present invention can achieve the following technical effects: 1) From material feeding and laying, image acquisition, recognition and analysis to spray valve sorting, the entire process is automated, which greatly reduces manual intervention and improves sorting efficiency.

[0108] 2) Based on the improved YOLOv5s model and a coarse-fine grading strategy, the accurate identification of tea leaf grades and impurities is achieved. Combined with the precise positioning of the high-pressure spray valve, it ensures that tea leaves of different grades fall accurately into the corresponding bins, thereby improving the purity of sorting.

[0109] 3) The main control unit adopts a multi-threaded parallel processing architecture, with each thread for acquisition, display, identification, and control running independently. Combined with a three-level cache design, it effectively avoids data congestion and processing delays, meeting the real-time requirements of high-speed sorting.

[0110] 4) The subcategories are achieved by adjusting the thresholds of color and shape features, which can flexibly adapt to the grading standards of different customers and different types of tea, without the need to frequently change hardware or retrain the model.

[0111] 5) Offline analysis and parameter setting functions facilitate on-site debugging, troubleshooting, and process optimization; intelligent learning function supports rapid adaptation to new standards on a small number of samples, reducing system maintenance costs.

[0112] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0113] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method of processing tea fresh leaf, characterized by, Includes the following steps: The image of fresh tea leaves to be processed is input into a deep learning-based tea leaf detection model to obtain detection information for each tea leaf target in the image. The detection information includes the detection box position, comprehensive confidence, category prediction probability, color feature value, and shape feature value. The coarse category of the tea leaf target is determined based on the predicted category probability, and the tea leaf targets belonging to the same coarse category are further subdivided based on the color feature value and / or the shape feature value. Based on the results of the subcategorization, the corresponding processing operations are performed on the target fresh tea leaves.

2. The tea fresh leaf treatment method according to claim 1, characterized by, The color feature value includes the predicted mean of at least one of hue, saturation, and brightness, and the shape feature value is the predicted area ratio of the fresh tea leaf target within the detection frame.

3. A tea leaf processing method according to claim 1 or 2, characterised in that, The tea leaf detection model is built on an improved YOLOv5s network, and its output layer is used to output the detection box position, overall confidence, category prediction probability, color feature value and shape feature value.

4. The tea leaf processing method according to claim 3, wherein The training process of the tea leaf detection model includes: Obtain an labeled image, which contains a detection box for the tea leaf target and the corresponding coarse classification label; For each detection box, the color feature value and shape feature value of the tea leaf target within the detection box are extracted as additional labels; The tea leaf detection model is trained using the labeled image, the coarse classification label, and the additional label. The loss function of the tea leaf detection model includes bounding box regression loss, target confidence loss, classification loss, color feature loss, and shape feature loss.

5. The method for processing fresh tea leaves according to claim 4, characterized in that, The color feature loss includes hue loss. saturation loss and brightness loss The shape feature loss is an area loss. The loss function of the tea leaf detection model. Represented as: wherein, is a bounding box regression loss, is a target confidence loss, is a classification loss, , , , , , , is a corresponding loss weight coefficient.

6. The tea fresh leaf treatment method according to claim 1, characterized by, The coarse classification includes at least one of buds, leaves, stems, and impurities; the fine classification determination includes: For the tea leaves that are coarsely classified as buds, they are further subdivided into at least one of the following categories based on their color and / or shape characteristics: whole bud, one bud and one leaf, one bud and two leaves, and one bud and multiple leaves.

7. The method for processing fresh tea leaves according to claim 1, characterized in that, The processing operations include: Based on the sub-category determination result, a control command is generated to drive the spray valve to sort the fresh tea leaves into the bin corresponding to the sub-category determination result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the tea leaf processing method as described in any one of claims 1 to 7.

9. A controller comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the tea leaf processing method as described in any one of claims 1 to 7.

10. A tea leaf sorting device, characterized in that, include: The components include: feeding device, conveyor belt, camera, spray valve, hopper, and main control unit; among which, The feeding device is used to spread the fresh tea leaves out and lay them flat on the conveyor belt for transport. The camera is used to capture images of fresh tea leaves on the conveyor belt; The main control component is used to receive the image of the fresh tea leaves, execute the fresh tea leaf processing method as described in any one of claims 1 to 7, output a subdivision classification result, and generate a control command based on the subdivision classification result to drive the spray valve to sort the fresh tea leaf targets on the conveyor belt into the hopper corresponding to the subdivision classification result.

Images