Deep learning-based tea leaf fine manipulation degree grading method and device and electronic equipment

Abstract

The invention relates to the technical field of tea making, in particular to a tea leaf fine manipulation degree grading method and device based on deep learning and electronic equipment, and the method comprises the steps: obtaining a to-be-used tea leaf fine manipulation image; the tea leaf green-making image is input into a target detection model, the target detection model outputs color-changing leaves, the tea leaf green-making degree of the extracted color-changing leaves is judged by adopting an RGB color space model, and the target detection model comprises a backbone network, a neck network and a detection head; the backbone network is used for extracting multiple pieces of feature information of the tea green-making image, the neck network is used for fusing the multiple pieces of feature information, and the detection head locates a leaf color change area in the tea green-making image based on the fused feature information; and finally, carrying out color segmentation on the obtained color-changed leaves through an RGB color space model, and determining the grade of the green-making degree of the tea leaves through the segmentation area. Therefore, the problems that the tea quality cannot be guaranteed and the cost is high in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of tea making technology, and in particular to a method, device and electronic equipment for grading the degree of greenness of tea leaves based on deep learning. Background Art

[0002] As one of the three major beverages in the world, tea has a long history and culture around the world. The primary processing process of tea includes picking, sun-drying / air-drying, making tea, frying tea, rolling and drying. Among them, the making process is the key link in forming the unique quality of Wuyi Rock Tea, and it is also the main process that determines the appearance of "green leaves with golden edges".

[0003] The process of making tea leaves green is divided into: withering → shaking → standing. Under current technical conditions, the withered leaves need to be placed in a shaking barrel first, and the equipment parameters, such as rotation speed and time, need to be set according to the type of tea and processing requirements. During the processing, in order to monitor the status of the tea leaves, it is necessary to manually open the lid after each round of shaking to determine the intensity of the next shaking and the length of standing. Therefore, the traditional mechanical greening process still has the following problems: (1) The degree of greening is mainly judged by manual sensory experience, and its degree judgment cannot be standardized, which affects the stability of product quality. (2) It is necessary to open the lid multiple times to observe the fermentation of tea leaves in the barrel, which consumes a lot of manpower and increases production costs and human resource investment. (3) The system for controlling the duration and intensity of shaking is low in intelligence. Summary of the invention

[0004] The present application provides a method, device and electronic equipment for grading the degree of greenness of tea leaves based on deep learning, so as to solve the problems in the related art that the quality control in the greening process of tea leaves relies on manual experience, cannot guarantee the quality of tea leaves, and has high costs.

[0005] The first aspect of the present application provides a method for grading the degree of green tea leaves based on deep learning, comprising the following steps: obtaining a green tea image to be used; inputting the green tea image into a target detection model, and the target detection model outputs discolored tea leaves, wherein the target detection model comprises: a backbone network, a neck network and a detection head, the backbone network is used to extract multiple feature information of the green tea image, the neck network is used to fuse the multiple feature information, and the detection head locates the discolored tea leaves in the green tea image based on the fused feature information; color segmentation is performed on the tea leaves cropped by the target detection model through an RGB color space model, and the green tea degree grading is determined according to the segmented area.

[0006] Optionally, an encoder is added to the tail of the feature extraction of the backbone network, and the encoder is used to increase the information flow between different levels to enhance the extraction capability of the backbone network.

[0007] Optionally, the ELAN module of the backbone network is replaced with a MobileOne module to improve the performance of real-time network detection.

[0008] Optionally, in the feature fusion part, a BiFPN network is used as a feature fusion method for the neck component. This feature fusion method weights features of different scales according to their importance and reduces unnecessary node calculations, so that the network can make more accurate judgments.

[0009] Optionally, the SPPCSPC-AM module is used to integrate global information, which improves the performance of the model in handling densely overlapping targets in complex scenes.

[0010] Optionally, obtaining tea leaves to be used as greening images includes: obtaining actual images during the greening process; generating simulated tea leaves as greening images based on a noise suppression generator model; and generating a tea leaves as greening image dataset to be trained based on the simulated tea leaves as greening images and actual images.

[0011] Optionally, the generator model includes a mapping network and a synthesis network, the mapping network is composed of multiple layers of fully connected layers, and the multiple convolutional layers in the synthesis network are used to remove noise in the tea leaf image.

[0012] Optionally, a generator model based on noise suppression generates a simulated tea leaf greening image, including: obtaining random noise that conforms to the Gaussian distribution; inputting the random noise into the mapping network through regularization to obtain a latent vector of the same dimension as the random noise; converting the latent vector into a channel style parameter and inputting it into the synthesis network, the synthesis network outputs a simulated tea leaf greening image, and the discriminator outputs a scalar value by extracting image features to judge the probability that the input image is a real image to feed back the generator to continuously generate high-resolution images close to the real image. The generated image is filtered using the variance of the Laplace method and some generated blurred images are discarded; the MSRCP algorithm derived from the Retinex algorithm is used to achieve image enhancement, improve the lighting environment, and highlight the characteristics of leaf spots.

[0013] Optionally, determining the greening degree of tea leaves based on color features includes: performing color segmentation on tea leaves obtained by a target detection model through an RGB color space model; identifying target color features in the color change area; calculating the area ratio of characteristic pixel points of the target color features in the greening image, and determining the greening degree of tea leaves based on the area ratio.

[0014] Optionally, before inputting the tea leaves image into the target detection model, it also includes: obtaining a tea leaves image dataset, wherein the image dataset includes a training set, a test set and a verification set; using the training set to train the target detection model, and using the test set and the verification set to test and verify the trained target detection model, until the preset conditions are met to stop the training and obtain the target detection model.

[0015] The second aspect of the present application provides a tea-leaf greening degree grading device based on deep learning, including: an acquisition module, used to acquire a tea-leaf greening image to be used; a detection module, used to input the tea-leaf greening image into a target detection model, and the target detection model outputs discolored tea leaves, wherein the target detection model includes: a backbone network, a neck network and a detection head, the backbone network is used to extract multiple feature information of the tea-leaf greening image, the neck network is used to fuse the multiple feature information, and the detection head locates the discolored tea leaves in the tea-leaf greening image based on the fused feature information; a determination module, which performs color segmentation on the tea leaves clipped by the detection model through an RGB color space model, and determines the tea-leaf greening degree grading according to the color features.

[0016] Optionally, an encoder is added to the tail of the feature extraction of the backbone network, and the encoder is used to increase the information flow between different levels to enhance the extraction capability of the backbone network.

[0017] Optionally, the ELAN module of the backbone network is replaced with a MobileOne module to improve the performance of real-time network detection.

[0018] Optionally, in the feature fusion part, a BiFPN network is used as a feature fusion method for the neck component. This feature fusion method weights features of different scales according to their importance and reduces unnecessary node calculations, so that the network can make more accurate judgments.

[0019] Optionally, the SPPCSPC-AM module is used to integrate global information, which improves the performance of the model in handling densely overlapping targets in complex scenes.

[0020] Optionally, the acquisition module is further used to acquire actual images during the tea-making process; generate simulated tea-making images based on a noise-suppressed generator model; and generate tea-making images to be used based on the simulated tea-making images and actual images.

[0021] Optionally, the generator model includes a mapping network and a synthesis network, the mapping network is composed of multiple layers of fully connected layers, and the multiple convolutional layers in the synthesis network are used to remove noise in the tea leaf image.

[0022] Optionally, the acquisition module is further used to obtain random noise that conforms to the Gaussian distribution; the random noise is input into the mapping network through regularization to obtain a latent vector of the same dimension as the random noise; the latent vector is converted into a channel style parameter and input into the synthesis network, the synthesis network outputs a simulated tea leaf green image, the discriminator extracts image features and outputs a scalar value to judge the probability that the input image is a real image to feed back the generator to continuously generate high-resolution images close to the real image. The generated image is filtered using the variance of the Laplace method and some generated blurred images are discarded; the MSRCP algorithm derived from the Retinex algorithm is used to achieve image enhancement, improve the lighting environment, and highlight the characteristics of leaf spots.

[0023] Optionally, the determination module is further used to identify target color features in the discoloration area; perform color segmentation on the tea leaves obtained by the target detection model through the RGB color space model; identify the target color features in the discoloration area; calculate the area ratio of characteristic pixel points of the target color features in the tea leaf greening image, and determine the tea leaf greening degree grade according to the area ratio.

[0024] Optionally, the deep learning-based tea leaf greening degree grading device further includes: a training module, used to obtain a tea leaf greening image dataset before inputting the tea leaf greening image into the target detection model, wherein the image dataset includes a training set, a test set and a verification set; the target detection model is trained using the training set, and the trained target detection model is tested and verified using the test set and the verification set, until the preset conditions are met and the training is stopped to obtain the target detection model.

[0025] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a tea greenness grading method based on deep learning as described in the above embodiment.

[0026] The fourth aspect of the present application provides a computer-readable storage medium having a computer program or instruction stored thereon. When the computer program or instruction is executed, it is used to implement the tea greenness grading method based on deep learning as in the above-mentioned embodiment.

[0027] Therefore, this application has at least the following beneficial effects:

[0028] The embodiment of the present application inputs the tea-leaf greening image into the target detection model, outputs the specific location where the color of the tea leaves changes during the greening process based on the target detection model, and further analyzes the detected color-changing area to extract the color characteristics of the leaf epidermis in the area, thereby realizing accurate detection and grading of the degree of greening of the tea leaves, so as to improve the production efficiency in the greening process, reduce labor costs, and realize the standardization and intelligence of the degree of greening of the tea leaves. Thus, the problems in the related art of quality control in the greening process of tea leaves that rely on manual experience, cannot guarantee the quality of tea leaves, and have high costs are solved.

[0029] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0031] Figure 1 A flowchart of a method for grading the degree of greenness of tea leaves based on deep learning according to an embodiment of the present application;

[0032] Figure 2 A structural diagram of a generator model provided according to an embodiment of the present application;

[0033] Figure 3 A comparison diagram of the generator model training period provided according to an embodiment of the present application;

[0034] Figure 4 A structural diagram of a target detection model provided according to an embodiment of the present application;

[0035] Figure 5 This is an example diagram of an RGB color space model provided according to an embodiment of the present application;

[0036] Figure 6 A curve diagram of the accuracy of the training process of different test groups provided according to an embodiment of the present application;

[0037] Figure 7 A comparison chart of the accuracy of different network model algorithms provided according to an embodiment of the present application;

[0038] Figure 8 A loss comparison curve of different algorithms on a training set provided according to an embodiment of the present application;

[0039] Fig. 9 A loss comparison curve of different algorithms on a validation set provided according to an embodiment of the present application;

[0040] Fig.10 A schematic diagram of a method for grading the degree of greenness of tea leaves based on deep learning according to an embodiment of the present application;

[0041] Fig.11 A block diagram of a tea greenness grading device based on deep learning according to an embodiment of the present application;

[0042] Fig.12 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0044] The following describes the tea leaves green degree grading method, device, electronic device and storage medium based on deep learning according to the embodiments of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides a tea leaves green degree grading method based on deep learning. In this method, the tea leaves green image is input into the target detection model, and the specific location where the color changes during the tea leaves green process is output based on the target detection model, and the color characteristics of the leaf epidermis in the area are extracted by further analyzing the detected color change area, so as to achieve accurate detection and grading of the tea leaves green degree, so as to improve the production efficiency in the greening process, reduce labor costs, and achieve standardization and intelligentization of the tea leaves green degree. Thus, the problems that the quality control in the tea leaves greening process in the related technology relies on manual experience, cannot guarantee the quality of tea leaves, and has high costs are solved.

[0045] Specifically, Figure 1 A flowchart of a method for grading the degree of greenness of tea leaves based on deep learning provided in an embodiment of the present application.

[0046] like Figure 1 As shown, the tea greenness grading method based on deep learning includes the following steps:

[0047] In step S101, an image of tea leaves to be used is obtained.

[0048] Among them, the tea-leaf making images include: actual images taken during the actual tea-leaf making process and simulated tea-leaf making images expanded by the generator model.

[0049] In one embodiment of the present application, obtaining a tea leaf greening image to be used includes: obtaining an actual image during the greening process; generating a simulated tea leaf greening image based on a noise suppression generator model; and generating the tea leaf greening image to be used based on the simulated tea leaf greening image and the actual image.

[0050] It is understandable that since the number of tea leaves are limited, in order to increase the sample size and improve the accuracy of model prediction, the embodiment of the present application uses a noise suppression-based generator model to generate simulated tea leaves images, mixes the original images and the generated images, thereby achieving the purpose of expanding the number of images and laying the foundation for subsequent image detection.

[0051] Furthermore, the generator model based on noise suppression generates a simulated tea leaf greening image, including: obtaining random noise that conforms to the Gaussian distribution; inputting the random noise into the mapping network through regularization to obtain a latent vector of the same dimension as the random noise; converting the latent vector into a channel style parameter and inputting it into the synthesis network, the synthesis network outputs a simulated tea leaf greening image, and the discriminator outputs a scalar value by extracting image features to judge the probability that the input image is a real image to feed back the generator to continuously generate high-resolution images close to the real image. The generated image is filtered using the variance of the Laplace method and some generated blurred images are discarded; the MSRCP algorithm derived from the Retinex algorithm is used to achieve image enhancement, improve the lighting environment, and highlight the characteristics of leaf spots.

[0052] Among them, the generator model includes a mapping network and a synthesis network. The mapping network is composed of multiple layers of fully connected layers, and the multiple convolutional layers in the synthesis network are used to remove noise in the tea leaf image.

[0053] Specifically, the embodiment of the present application proposes a generator model based on noise suppression for the characteristics that the amount of tea leaf image data is limited and the basic features and shape contours of the data are relatively similar. The generator structure of the model is as follows: Figure 2As shown, it is composed of a mapping network and a synthesis network. First, the random noise z that conforms to the Gaussian distribution is input into the mapping network composed of 8 fully connected layers through regularization to obtain a potential vector w of the same dimension as z, and then the obtained potential w is converted into a channel style parameter through an affine transformation and input into the synthesis network. In the synthesis network, an output image is created through a series of convolutions, modulation and demodulation, nonlinear functions and noise. In the embodiment of the present application, according to the characteristics that the basic characteristics of the tea-making data and the outline are relatively similar and the color types show low diversity, the noise superposition is removed in the 4×4 to 32×32 layers in the synthesis network, and the generated image caused by the noise contains many unnecessary noise points, thereby affecting the image quality, thereby improving the quality and authenticity of the generated image. Secondly, since the receptive field size in StyleGAN2 is limited by the size of the convolution kernel, the local features are mainly captured during the training process. Therefore, the embodiment of the present application introduces SA (Self-Attention, self-attention mechanism) in the synthesis network to strengthen the generator's attention to the image texture details and long-distance pixel association information, and help the model capture the global features and complex dependencies of the image to generate a higher quality simulated tea-making image.

[0054] In the actual implementation process, the embodiment of the present application first needs to train the generator model. The images taken during the entire greening process are used as original images, and are divided into training sets and test sets according to a ratio of 9:1 for training the generator model. The training model parameters are set to train 2000 kimgs, batch-size is 32, the initial learning rate is 0.0025, and the Adam optimizer is used. The model is saved every 50 kimgs, and the generated images of the stored model are output. The quality of the generated images is observed through the FID score, such as Figure 3 As shown in the figure, the generated images at different training periods are selected for comparative analysis, which can intuitively observe the convergence effect of the generator model based on noise suppression. The FID score is used as the performance indicator. Some samples generated by the model saved during the training process are used, and the FID score is calculated through the generated samples for comparative analysis.

[0055] In order to objectively evaluate the quality of the generated image, the FID score and SSIM are used to determine the quality of the generated image. Table 1 is a quantitative comparison of the evaluation indicators of the original StyleGAN2-ADA model and the method of the embodiment of the present application.

[0056] Table 1

[0057] Evaluation indicators StyleGAN2-ADA Methods FID 56.58 50.95 SSIM 0.46 0.69

[0058] Furthermore, the embodiment of the present application selects a trained model to set a random seed, expands the green tea image, mixes the original image and the generated image to achieve the purpose of expanding the number of images, and lays the foundation for subsequent image detection.

[0059] In step S102, the tea leaves image is input into the target detection model, and the target detection model outputs the discolored tea leaves, wherein the target detection model includes: a backbone network, a neck network and a detection head, the backbone network is used to extract multiple feature information of the tea leaves image, the neck network is used to fuse the multiple feature information, and the detection head locates the discolored tea leaves in the tea leaves image based on the fused feature information.

[0060] Among them, the backbone network adds an encoder at the end of feature extraction, and the encoder is used to increase the information flow between different levels to enhance the extraction ability of the backbone network.

[0061] The target detection model of the embodiment of the present application is as follows Figure 4 As shown in the figure, it consists of three parts: a backbone network for feature extraction, a neck for feature enhancement, and a detection head for prediction. In the backbone network, firstly, a three-layer Transformer encoder connected by dense connections is introduced at the tail of feature extraction, which promotes the information flow between different layers in the network and helps the Transformer encoder receive richer feature information to enhance the feature extraction ability of the model. Secondly, the ELAN module of the backbone network is replaced by the MobileOne module to improve the performance of real-time detection of the network. In the feature fusion part, the BiFPN network is used as the feature fusion method of the neck component. This feature fusion method weights the features of different scales according to their importance and reduces unnecessary node calculations, so that the network can make more accurate judgments. At the same time, the SPPCSPC-AM module is used to improve the performance of the model in processing dense overlapping targets in complex scenes through the integration of global information.

[0062] During the actual implementation process, the embodiment of the present application can input the image of the tea leaves to be used into the target detection model. The backbone network processes the input image to generate a multi-level feature map. The neck network receives the feature map from the backbone network and forms a richer and more generalized feature representation by fusing these features. The detection head locates the target according to the fused feature map and outputs the position information of the color change area of ​​the tea leaves.

[0063] In step S103, the tea leaves obtained by the target detection model are color segmented using the RGB color space model, and the greenness of the tea leaves is graded according to the segmented area.

[0064] It can be understood that the RGB color space model of the embodiment of the present application extracts the color characteristics of the leaf epidermis by the RGB color space model. In the RGB color space model, each color is determined by a combination of different intensities of the three channels of red (R), green (G), and blue (B). The equation of the RGB color space is:

[0065] C=ω1R+ω2G+ω3B

[0066] Where: ω1, ω2 and ω3 represent the weighting coefficients of each component, and C represents the color obtained by weighting the three basic components. The RGB color space model is based on the additive color principle and forms new colors by superimposing different color light sources. Figure 5 shown.

[0067] In the RGB color space model, each component of red (R), green (G), and blue (B) is located on three coordinate axes, each with 256 levels of brightness. The maximum grayscale value of each component is 255 and the minimum is 0. If the intensity of the three colors is 0, the result is black. If the intensity of the three colors is 255, the result is white. Among them, pure red is represented by (1,0,0), green is represented by (0,1,0), and blue is represented by (0,0,1). By analyzing the RGB value of each pixel, the color of each pixel can be determined. In this way, different color areas of the image can be segmented according to different thresholds or conditions. Therefore, the RGB color space segmentation method is used to judge the degree of greenness of tea leaves.

[0068] In one embodiment of the present application, the greening degree of tea leaves is graded based on color features, including: identifying target color features in the color change area; calculating the area ratio of the target color features in the greening tea leaves image, and determining the greening degree of tea leaves based on the area ratio.

[0069] Among them, the target color feature can be red. The embodiment of the present application uses the RGB color space model to extract the number of red pixel values ​​and the number of green pixel values ​​of the leaf epidermis, and obtains the ratio of red pixels to green pixels of the tea leaves by comparison. Finally, the red area of ​​the tea leaves in the detection frame is segmented according to the color feature information and the ratio is output, which is used for visual judgment of the degree of greenness of the tea leaves, thereby realizing efficient detection and grading of the degree of greenness of the tea leaves.

[0070] In one embodiment of the present application, before inputting the tea leaves into the target detection model, it also includes: obtaining a tea leaves image dataset, wherein the image dataset includes a training set, a test set and a verification set; using the training set to train the target detection model, and using the test set and the verification set to test and verify the trained target detection model, until the preset conditions are met and the training is stopped to obtain the target detection model.

[0071] It is understandable that the embodiment of the present application can adopt a random division method, and all image data are divided into training set, test set and validation set in a ratio of 8:1:1. The training model parameters are set, and the training is 300 epochs, the batch-size is 16, and the initial learning rate is 0.001. The model is saved every 5 epochs, and the loss function change curve is used to determine whether the training has reached a stable point, and the best model is selected for subsequent positioning. Through the trained model, the green image of tea is detected, and the tea with "green leaves with gold edges" is preliminarily located.

[0072] Furthermore, the embodiment of the present application can cut out the green leaves with golden edges located in the above embodiment, and synchronously input them into the RGB color space model, extract the number of red pixels and the number of green pixels on the leaf epidermis from the RGB color space model, and obtain the ratio of red pixels to green pixels of the tea leaves by comparison. Finally, the red area of ​​the tea leaves in the detection frame is segmented according to the color feature information and the ratio is output for visual discrimination of the degree of greenness of the tea leaves.

[0073] In addition, the embodiment of the present application also analyzes the target detection model of the embodiment of the present application in detail in combination with specific test results, as shown in Table 2, ablation experiments under different model combinations:

[0074] Table 2

[0075]

[0076] like Figure 6 As shown, the accuracy curves of the training process of different experimental groups are shown. It can be seen that as the training progresses, the accuracy of each experimental group gradually increases and finally stabilizes. The method of the embodiment of the present application shows the highest accuracy during the training process and reaches a higher accuracy level in the earlier training rounds.

[0077] like Figure 7 As shown, a comparison of four different algorithms in the detection of the degree of greenness of tea leaves is demonstrated. It can be seen from the figure that the method of the embodiment of the present application is superior to the other three algorithms in terms of accuracy, recall rate, etc., and has the smallest number of parameters. It has better performance and efficiency in the task of detecting the degree of greenness of tea leaves and is more suitable for the detection of the degree of greenness of tea leaves.

[0078] like Figure 8 and Fig. 9As shown, the loss comparison curves of two different algorithms on the training and validation sets are shown, namely YOLO v7 and the method of the embodiment of the present application. The colors of the curves are red and blue, respectively. Red represents YOLO v7, and blue represents the method of the embodiment of the present application. Although YOLO v7 has a faster loss decline rate in the early stage of training, it has not fully converged. The loss of this method on both the training and validation sets decreases faster and reaches a stable minimum earlier, which shows that this method can better fit the data during the training process and avoid overfitting, thereby achieving lower losses on the validation set. In addition, the loss gap between the training and validation sets of this method is small, indicating that the model has good generalization ability.

[0079] Combine the following Fig.10 The tea greening degree grading method based on deep learning in the embodiment of the present application is described in detail. First, the problem of the limited number of collected image samples caused by the long tea greening process is addressed. A generator model based on noise suppression is proposed to remove noise addition in the 4×4 to 32×32 layers of the synthetic network, suppress the noise weight of low-level features, and reduce the diversity of basic features. Secondly, the self-attention mechanism is introduced to strengthen the generator's attention to image texture details and long-distance pixel correlation information to generate better quality tea greening images.

[0080] Secondly, in order to optimize the processing flow of Wuyi rock tea and realize the standardization of the grading of green tea, a green tea degree detection and grading algorithm based on convolutional neural network is proposed. The algorithm consists of two stages. In the first stage, the key feature extraction of the target area is improved by adding a densely connected Transformer encoder at the end of the YOLO v7 backbone network; secondly, the MobileOne module is used to replace the ELAN (Effective Long-Range Aggregation Network) module in the backbone network to improve the inference speed of the model; thirdly, in view of the mutual occlusion of tea leaves, in order to capture a wider range of background information, the proposed SPPCSPC-AM module is used to reduce the loss of key semantic information caused by relying only on edge information and ignoring background information; in the second stage, the leaves with green leaves with gold edges detected by the detection network are first cropped out; secondly, the RGB color space segmentation method is used to segment the discolored areas in the leaves; finally, the segmented area ratio of a single image is calculated, and the grading of the green tea degree is completed through the area ratio.

[0081] According to the tea greening degree grading method based on deep learning proposed in the embodiment of the present application, by inputting the tea greening image into the target detection model, the specific location where the color of the tea leaves changes during the greening process is output based on the target detection model, and by further analyzing the detected color change area, the color characteristics of the leaf epidermis in the area are extracted to achieve accurate detection and grading of the tea greening degree, so as to improve the production efficiency in the greening process, reduce labor costs, and achieve standardization and intelligence of the tea greening degree. Thus, the problems in the related art that the quality control in the tea greening process depends on manual experience, cannot guarantee the quality of tea leaves, and has high costs are solved.

[0082] Next, a tea leaf greenness grading device based on deep learning proposed in accordance with an embodiment of the present application is described with reference to the accompanying drawings.

[0083] Fig.11 It is a block diagram of a tea leaves greenness grading device based on deep learning in an embodiment of the present application.

[0084] like Fig.11 As shown, the tea greenness grading device 10 based on deep learning includes: an acquisition module 100, a detection module 200 and a determination module 300.

[0085] Among them, the acquisition module 100 is used to obtain the tea leaves to be used; the detection module 200 is used to input the tea leaves to be used into the target detection model, and the target detection model outputs the discolored tea leaves, wherein the target detection model includes: a backbone network, a neck network and a detection head, the backbone network is used to extract multiple feature information of the tea leaves to be used, the neck network is used to fuse the multiple feature information, and the detection head locates the discolored tea leaves in the tea leaves to be used based on the fused feature information; the determination module 300 is used to perform color segmentation on the tea leaves obtained by the target detection model through the RGB color space model, and determine the degree of tea leaves to be used according to the segmented area.

[0086] Optionally, an encoder is added to the tail of the feature extraction of the backbone network, and the encoder is used to increase the information flow between different levels to enhance the extraction capability of the backbone network.

[0087] Optionally, the ELAN module of the backbone network is replaced with a MobileOne module to improve the performance of real-time network detection.

[0088] Optionally, in the feature fusion part, a BiFPN network is used as a feature fusion method for the neck component. This feature fusion method weights features of different scales according to their importance and reduces unnecessary node calculations, so that the network can make more accurate judgments.

[0089] Optionally, the SPPCSPC-AM module is used to integrate global information, which improves the performance of the model in handling densely overlapping targets in complex scenes.

[0090] Optionally, the acquisition module 100 is further used to acquire actual images during the tea-making process; generate simulated tea-making images based on a noise suppression generator model; and generate a tea-making image dataset to be trained based on the simulated tea-making images and actual images.

[0091] Optionally, the generator model includes a mapping network and a synthesis network, the mapping network is composed of multiple layers of fully connected layers, and the multiple convolutional layers in the synthesis network are used to remove noise in the tea leaf image.

[0092] Optionally, the acquisition module 100 is further used to obtain random noise that conforms to the Gaussian distribution; the random noise is input into the mapping network through regularization to obtain a latent vector of the same dimension as the random noise; the latent vector is converted into a channel style parameter and input into the synthesis network, the synthesis network outputs a simulated tea green image, and the discriminator outputs a scalar value by extracting image features to judge the probability that the input image is a real image to feed back the generator to continuously generate a high-resolution image close to the real image. The generated image is filtered using the variance of the Laplace method and some generated blurred images are discarded; the MSRCP algorithm derived from the Retinex algorithm is used to achieve image enhancement, improve the lighting environment, and highlight the characteristics of leaf spots.

[0093] Optionally, the determination module 300 is further used to identify the target color features in the discoloration area, perform color segmentation on the tea leaves obtained by the target detection model through the RGB color space model; identify the target color features in the discoloration area; calculate the area ratio of the characteristic pixel points of the target color features in the tea leaf greening image, and determine the tea leaf greening degree grade according to the area ratio.

[0094] Optionally, the deep learning-based tea leaf greening degree grading device 10 also includes: a training module, used to obtain a tea leaf greening image dataset before inputting the tea leaf greening image into the target detection model, wherein the image dataset includes a training set, a test set and a verification set; the target detection model is trained using the training set, and the trained target detection model is tested and verified using the test set and the verification set, until the preset conditions are met and the training is stopped to obtain the target detection model.

[0095] It should be noted that the aforementioned explanation of the embodiment of the method for grading the degree of greenness of tea leaves based on deep learning is also applicable to the device for grading the degree of greenness of tea leaves based on deep learning in this embodiment, and will not be repeated here.

[0096] According to the tea greening degree grading device based on deep learning proposed in the embodiment of the present application, by inputting the tea greening image into the target detection model, the specific location where the color of the tea leaves changes during the greening process is output based on the target detection model, and by further analyzing the detected color change area, the color characteristics of the leaf epidermis in the area are extracted to achieve accurate detection and grading of the tea greening degree, so as to improve the production efficiency in the greening process, reduce labor costs, and achieve standardization and intelligence of the tea greening degree. Thus, the problems in the related art that the quality control in the tea greening process depends on manual experience, cannot guarantee the quality of tea leaves, and has high costs are solved.

[0097] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0098] A memory 1201 , a processor 1202 , and a computer program stored in the memory 1201 and executable on the processor 1202 .

[0099] When the processor 1202 executes the program, the tea greenness grading method based on deep learning provided in the above embodiment is implemented.

[0100] Furthermore, the electronic device further comprises:

[0101] The communication interface 1203 is used for communication between the memory 1201 and the processor 1202 .

[0102] The memory 1201 is used to store computer programs that can be executed on the processor 1202 .

[0103] The memory 1201 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0104] If the memory 1201, the processor 1202 and the communication interface 1203 are implemented independently, the communication interface 1203, the memory 1201 and the processor 1202 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0105] Optionally, in a specific implementation, if the memory 1201, the processor 1202 and the communication interface 1203 are integrated on a chip, the memory 1201, the processor 1202 and the communication interface 1203 can communicate with each other through an internal interface.

[0106] The processor 1202 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0107] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for grading the degree of greenness of tea leaves based on deep learning.

[0108] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0109] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0110] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0111] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0112] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0113] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for grading the degree of greenness of tea leaves based on deep learning, characterized in that: The following steps are involved: Obtain the tea leaves to be used for making green tea; The tea leaves are input into a target detection model, and the target detection model outputs the discolored tea leaves, wherein the target detection model comprises: a backbone network, a neck network, and a detection head, wherein the backbone network is used to extract multiple feature information of the tea leaves, the neck network is used to fuse the multiple feature information, and the detection head locates the discolored tea leaves in the tea leaves image based on the fused feature information; The tea leaves obtained by the target detection model are segmented by color through the RGB color space model, and the degree of greenness of the tea leaves is determined according to the segmented area.

2. The method for grading the degree of greenness of tea leaves based on deep learning according to claim 1, characterized in that: The backbone network adds an encoder at the end of feature extraction, and the encoder is used to increase the information flow between different levels to enhance the extraction ability of the backbone network; the backbone network adopts the MobileOne module to improve the performance of real-time detection of the network; the backbone network adopts the BiFPN network to fuse the feature maps of different layers and reduce node calculations; the backbone network adopts the SPPCSPC-AM module, which improves the performance of the model in processing densely overlapping targets in complex scenes through the integration of global information.

3. The method for grading the degree of greenness of tea leaves based on deep learning according to claim 1, characterized in that: The step of obtaining a tea leaf greening image dataset to be used comprises: Get actual images of the greening process; The noise suppression-based generator model generates simulated tea leaves green images; The tea leaf greening image data set is generated based on the simulated tea leaf greening image and the actual image.

4. The method for grading the degree of greenness of tea leaves based on deep learning according to claim 3 is characterized in that: The generator model includes a mapping network and a synthesis network, wherein the mapping network is composed of multiple fully connected layers, and the multiple convolutional layers in the synthesis network are used to remove noise points in the tea leaf image; The noise suppression-based generator model generates a simulated tea leaf greening image, including: Get random noise that conforms to Gaussian distribution; Inputting the random noise into the mapping network through regularization to obtain a latent vector with the same dimension as the random noise; The latent vector is converted into a channel style parameter and input into a synthesis network, and the synthesis network outputs the simulated tea leaf greening image.

5. The method for grading the degree of greenness of tea leaves based on deep learning according to claim 4, characterized in that: Before generating the tea leaf greening image dataset according to the simulated tea leaf greening image and the actual image, the method further includes: The variance of Laplace method is used to filter and discard the blurred images in the simulated tea leaves. The MSRCP algorithm derived from the Retinex algorithm is used to enhance the image of simulated tea leaves, improve the lighting environment, and highlight the characteristics of leaf spots.

6. The method for grading the degree of greenness of tea leaves based on deep learning according to claim 1, characterized in that: Determining the grading of the degree of greenness of tea leaves according to the segmented area includes: Calculate the area ratio of characteristic pixel points of the target color feature in the tea leaf greening image; The greenness grade of the tea leaves is determined according to the area ratio.

7. The method for grading the degree of greenness of tea leaves based on deep learning according to claim 1, characterized in that: Before the green tea image is input into the target detection model, the method further includes: Acquire an image dataset of tea leaves being greened, wherein the image dataset includes a training set, a test set, and a validation set; The target detection model is trained using the training set, and the trained target detection model is tested and verified using the test set and the validation set, until the training is stopped when a preset condition is met to obtain the target detection model.

8. A tea greenness grading device based on deep learning, characterized in that: include: An acquisition module, used for acquiring the tea leaves to be used for making green tea; A detection module, used for inputting the tea leaves image into a target detection model, and the target detection model outputs the discolored tea leaves, wherein the target detection model comprises: a backbone network, a neck network and a detection head, the backbone network is used for extracting a plurality of feature information of the tea leaves image, the neck network is used for fusing the plurality of feature information, and the detection head locates the discolored tea leaves in the tea leaves image based on the fused feature information; The determination module is used to perform color segmentation on the tea leaves obtained by the target detection model through the RGB color space model, and determine the greenness grade of the tea leaves according to the segmented area.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for grading the degree of greenness of tea leaves based on deep learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed, the method for grading the degree of greenness of tea leaves based on deep learning as described in any one of claims 1 to 7 is implemented.

Images