Improved ShuffleNet V2-based tea leaf fine manipulation degree identification method

Abstract

The invention discloses a tea leaf fine manipulation degree identification method based on improved ShuffleNet V2, and the method comprises the steps: 1, obtaining a tea leaf fine manipulation image, obtaining a tea leaf fine manipulation data set through data enhancement, and dividing the tea leaf fine manipulation data set into a training set, a verification set and a test set; 2, a CA attention mechanism, a residual structure, a multi-scale shallow feature extraction module, a ReLU6 activation function and a Ghost module are adopted for improvement, and an improved ShuffleNet V2 model is obtained; 3, training the improved ShuffleNet V2 model by using the training set, and verifying the trained model by using the verification set so as to optimize model parameters to obtain a final improved ShuffleNet V2 model; and step 4, inputting test samples of the test set into the finally improved ShuffleNet V2 model, predicting and outputting a classification identification result, and completing identification of the degree of fine manipulation of the tea leaves. The light-weight model is adopted to realize intelligent judgment of the green-making degree of the tea leaves.

Description

Technical Field

[0001] The present invention relates to the technical field of large-scale tea recognition models, and in particular to a method for identifying the degree of greenness of tea leaves based on an improved ShuffleNet V2. Background Art

[0002] During the process of making tea leaves of Tieguanyin, the characteristics of tea leaves in adjacent stages are not much different. For example, the characteristics of tea leaves after the first shaking and before the second shaking are basically the same, both of which are fresh and lively, with leaves slightly upright; after the second shaking, there are a few brown spots on the leaves and stems; before the third shaking, the number of brown spots on the leaves increases, a little brown appears on the edges of the leaves, and the middle and lower parts of the stems are brown; after the third shaking, some brown appears on the edges of the leaves, and half of the stems are brown; at the end of the making process, the edges of the leaves are completely covered with brown (commonly known as red edging), and two-thirds or more of the stems are brown. Therefore, how to accurately extract these subtle feature information has become the key to accurately identifying the degree of making tea leaves of Tieguanyin. At the same time, the recognition model should be made as lightweight as possible to save hardware resources and costs.

[0003] The existing technology generally adopts the following technical solutions to identify the quality of tea leaves: ① Manual identification: This method is inefficient. The identification of the degree of green tea is greatly affected by human factors. The quality of green tea is unstable, which will lead to uneven quality and taste of the tea.

[0004] ② Aroma combined with water loss rate: When collecting aroma, in order to avoid the influence of the surrounding environment, this method needs to put this batch of tea into a relatively closed box and use an aroma meter to collect aroma data. As a result, the environment of the measured tea is obviously different from that of the tea in the actual greening process. This is the main reason why the accuracy of this method in identifying the degree of greening is not high.

[0005] ③Machine vision combined with neural network technology: Use RGB and HIS color parameters to predict the degree of greening. However, during the process of making Tieguanyin tea, not only the color of the tea leaves will change, but also the shape of the tea leaves will change. Therefore, it is not comprehensive to judge the degree of greening based on color parameters alone. In addition, the color characteristics of the tea leaves in the first and second shaking stages are not much different, and the color of the tea leaves in the same stage will also vary in depth, resulting in a low accuracy rate in identifying the degree of greening using color parameters. Summary of the invention

[0006] The purpose of the present invention is to provide a method for identifying the degree of green tea leaves based on the improved ShuffleNet V2, which uses the improved ShuffleNet V2 model to classify and identify the color, shape and distribution position of brown spots of green tea leaves, and finally realizes the intelligent judgment of the degree of green tea leaves of Tieguanyin. At the same time, by improving the lightweight model, the model is easy to deploy on mobile terminal devices.

[0007] The technical solution adopted by the present invention is: A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2, comprising the following steps: Step 1, obtain tea leaves images and obtain a tea leaves dataset through data enhancement, and divide the tea leaves dataset into a training set, a validation set, and a test set according to a set ratio; Step 2, the CA attention mechanism, residual structure, multi-scale shallow feature extraction module, ReLU6 activation function and Ghost module are used to improve the ShuffleNet V2 convolutional neural network to obtain an improved ShuffleNet V2 model; Step 3, use the training set to train the improved ShuffleNet V2 model, and use the validation set to validate the trained model, so as to optimize the model parameters to obtain the final improved ShuffleNet V2 model; Step 4: Input the test samples of the test set into the final improved ShuffleNet V2 model, predict and output the classification and recognition results of the color, shape and distribution position of brown spots of the test samples, and complete the recognition of the degree of greenness of the tea leaves.

[0008] Furthermore, in step 1, an industrial camera is used to collect images of a batch of tea leaves 20 minutes after each shaking and before the cooling.

[0009] Furthermore, the tea-leaf greening image in step 1 is processed by data enhancement means such as horizontal mirror flipping, vertical mirror flipping, brightness adjustment, contrast adjustment and saturation adjustment to generate a tea-leaf greening dataset.

[0010] Furthermore, the tea-making dataset in step 1 is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0011] Furthermore, the specific steps to obtain the improved ShuffleNet V2 model are as follows: Step 21, build the ShuffleNet V2 backbone network, which includes an input layer, a convolutional layer 1, a stage 2 layer, a stage 3 layer, a stage 4 layer, a convolutional layer 2, a global pooling layer, and a fully connected layer; Step 22, replace the convolutional layer 1 of the ShuffleNet V2 model with a multi-scale shallow feature extraction module (Multi-scale Shallow Feature Extraction, MSFE); the input features of the multi-scale shallow feature extraction module are divided into three paths, one path passes through a 3×3 maximum pooling layer with a stride of 2 and a 1×1 convolution layer; another path passes through a 3×3 convolution layer with a stride of 2 in sequence; the last path passes through a 3×3 convolution layer with a stride of 2 and a stride of 1 in sequence; the output features of the three paths are concatenated to become the output features of the multi-scale shallow feature extraction module; Step 23, adding CA attention mechanism and residual structure connection to the basic feature extraction modules of the second layer, the third layer, and the fourth layer of the ShuffleNet V2 model; the basic feature extraction module includes a basic unit and a downsampling unit; In step 24, the first 1×1 convolution of each branch in the basic unit and downsampling unit of the ShuffleNet V2 model is replaced with a Ghost module, and the second 1×1 convolution in each branch is deleted to make the model lightweight, thereby obtaining an improved ShuffleNet V2 model.

[0012] Furthermore, in step 22, the activation functions of the multi-scale shallow feature extraction module and the stage two layer, stage three layer, stage four layer, and convolutional layer two are replaced with ReLU6.

[0013] Furthermore, the CA attention mechanism in step 23 specifically performs the following steps: Step 23-1, change the size to The input feature maps are respectively along the height and width Average pooling is performed in two directions to obtain two sizes of and The characteristic map of and width The pooling operation formula for the direction is as follows: in, is the feature map in the height direction; Feature map in width direction, is the number of channels; Indicates the width position of the current pooling operation, Indicates the height position of the current pooling operation, and It is used to represent the index along the height and width direction during the pooling operation; Step 23-2, the two feature maps containing specific direction information are concat-concatenated in the third dimension, and then the dimension is reduced by 1×1 convolution to obtain a size of The feature map of is the dimension reduction coefficient when the channel is downsampled; Step 23-3, generate intermediate feature vector using nonlinear activation function , along the spatial dimension, the feature map Perform channel separation to generate two feature vectors and ; Step 23-4, using 1×1 convolution and Perform dimensionality increase operation and obtain the sizes of and The feature map of Step 23-5, then use the Sigmoid activation function to get the attention vector and , will obtain and The feature vector is multiplied by the feature weight to obtain an image with enhanced feature expression at the output of CA.

[0014] Furthermore, the Ghost module in step 24 includes two parts. One part uses ordinary convolution to compress the number of channels of the input image and generates some intrinsic feature maps through identity mapping; the other part uses a series of linear operations to Ф Generate a ghost feature map to achieve feature expansion while increasing the number of channels to obtain more feature maps and further enrich the feature expression; finally, concatenate the ghost feature map with the intrinsic feature map in the channel dimension to obtain an output of the same dimension as the traditional convolution.

[0015] Furthermore, in step 3, the Adam optimizer is used for training, the cross entropy loss function is used as the loss function, the batch size is set to 24, the learning rate is 0.001, and the number of iterations is 300.

[0016] The present invention adopts the above technical scheme, uses an industrial camera to collect tea greening images, and through data enhancement, makes a tea greening data set for model training, verification and testing. Secondly, by adding CA attention mechanism and residual structure connection to the basic feature extraction module (including basic unit and downsampling unit) of ShuffleNet V2 model, using multi-scale shallow feature extraction (MSFE) module in the shallow convolution of the model and using ReLU6 to replace ReLU activation function, etc., the recognition accuracy of the model for the degree of tea greening is improved. Finally, the first 1×1 convolution of each branch in the basic feature extraction module is replaced with Ghost module, and the second 1×1 convolution of each branch is deleted to make the model lightweight. The present invention uses the improved ShuffleNet V2 model to classify and identify the color, morphology and distribution position of brown spots of tea green, and finally realizes the intelligent judgment of the degree of tea greening. At the same time, through the lightweight improvement of the model, the model is easy to be deployed in mobile terminal devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 This is a structural diagram of an improved ShuffleNet V2 model in the present invention; Figure 2 A schematic diagram of the green image comparison of Tieguanyin at different stages; Figure 3 It is a schematic diagram of the structure of the CA attention mechanism module; Figure 4 A visual comparison diagram of the feature capture before and after the addition of the CA attention mechanism to the ShuffleNet V2 model; Figure 5 Schematic diagram of the structure after introducing residual connection into the basic feature extraction module of ShuffleNet V2 model; Figure 6 A comparison diagram of the training loss curves before and after the introduction of residual connections for the ShuffleNet V2 model with CA attention mechanism; Figure 7 It is a structural schematic diagram of the multi-scale shallow feature extraction MSFE module of the present invention; Figure 8 A visual comparison diagram of the feature capture of the ShuffleNet V2 model with CA attention mechanism before and after adding the MSFE module; Fig. 9This is a schematic diagram comparing the activation values ​​of the ShuffleNet V2 model using the ReLU activation function and the ReLU6 activation function; Fig.10 It is a schematic diagram of the structure of the Ghost module; Fig.11 This is a schematic diagram of the structure of the basic feature extraction module of the ShuffleNet V2 model after lightweight improvement; Fig.12 The figure is a schematic diagram comparing the confusion matrices of two different models, the present invention and the existing ResNet18. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0019] like Figures 1 to 12 As shown in one, the present invention discloses a method for identifying the degree of tea leaves being green based on an improved ShuffleNet V2. An industrial camera is used to collect tea leaves being green images, and a tea leaves being green dataset is prepared through data enhancement for model training, verification and testing. Secondly, by adding CA attention mechanism and residual structure connection to the basic feature extraction module (basic unit and downsampling unit) of the ShuffleNet V2 model, a multi-scale shallow feature extraction (MSFE) module is used in the shallow convolution of the model, and ReLU6 is used to replace the ReLU activation function, etc., four improvements are made to improve the recognition accuracy of the model for the degree of tea leaves being green. Finally, the first 1×1 convolution of each branch in the basic feature extraction module is replaced with a Ghost module, and the second 1×1 convolution of each branch is deleted to make the model lightweight. A method for identifying the degree of tea leaves being green based on an improved ShuffleNet V2 of the present invention specifically includes the following steps: Step 1, obtain tea leaves images and obtain a tea leaves dataset through data enhancement, and divide the tea leaves dataset into a training set, a validation set, and a test set according to a set ratio; Step 2, the CA attention mechanism, residual structure, multi-scale shallow feature extraction module, ReLU6 activation function and Ghost module are used to improve the ShuffleNet V2 convolutional neural network to obtain an improved ShuffleNet V2 model; Step 3, use the training set to train the improved ShuffleNet V2 model, and use the validation set to validate the trained model, so as to optimize the model parameters to obtain the final improved ShuffleNet V2 model; Step 4: Input the test samples of the test set into the final improved ShuffleNet V2 model, predict and output the classification and recognition results of the color, shape and distribution position of brown spots of the test samples, and complete the recognition of the degree of greenness of the tea leaves.

[0020] Furthermore, in step 1, an industrial camera is used to collect images of a batch of tea leaves 20 minutes after each shaking and before the cooling.

[0021] Furthermore, the tea-leaf greening image in step 1 is processed by data enhancement means such as horizontal mirror flipping, vertical mirror flipping, brightness adjustment, contrast adjustment and saturation adjustment to generate a tea-leaf greening dataset.

[0022] Furthermore, the tea-making dataset in step 1 is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0023] Furthermore, the specific steps to obtain the improved ShuffleNet V2 model are as follows: Step 21, build the ShuffleNet V2 backbone network, which includes an input layer, a convolutional layer 1, a stage 2 layer, a stage 3 layer, a stage 4 layer, a convolutional layer 2, a global pooling layer, and a fully connected layer; Step 22, replace the convolutional layer 1 of the ShuffleNet V2 model with a multi-scale shallow feature extraction module (Multi-scale Shallow Feature Extraction, MSFE); the input features of the multi-scale shallow feature extraction module are divided into three paths, one path passes through a 3×3 maximum pooling layer with a stride of 2 and a 1×1 convolution layer; another path passes through a 3×3 convolution layer with a stride of 2 in sequence; the last path passes through a 3×3 convolution layer with a stride of 2 and a stride of 1 in sequence; the output features of the three paths are concatenated to become the output features of the multi-scale shallow feature extraction module; Step 23, adding CA attention mechanism and residual structure connection to the basic feature extraction modules of the second layer, the third layer, and the fourth layer of the ShuffleNet V2 model; the basic feature extraction module includes a basic unit and a downsampling unit; In step 24, the first 1×1 convolution of each branch in the basic unit and downsampling unit of the ShuffleNet V2 model is replaced with a Ghost module, and the second 1×1 convolution in each branch is deleted to make the model lightweight, thereby obtaining an improved ShuffleNet V2 model.

[0024] Furthermore, in step 22, the activation functions of the multi-scale shallow feature extraction module and the stage two layer, stage three layer, stage four layer, and convolutional layer two are replaced with ReLU6.

[0025] Furthermore, the CA attention mechanism in step 23 specifically performs the following steps: Step 23-1, change the size to The input feature maps are respectively along the height and width Average pooling is performed in two directions to obtain two sizes of and The characteristic map of and width The pooling operation formula for the direction is as follows: in, is the feature map in the height direction; Feature map in width direction, is the number of channels; Indicates the width position of the current pooling operation, Indicates the height position of the current pooling operation, and It is used to represent the index along the height and width direction during the pooling operation; Step 23-2, the two feature maps containing specific direction information are concat-concatenated in the third dimension, and then the dimension is reduced by 1×1 convolution to obtain a size of The feature map of is the dimension reduction coefficient when the channel is downsampled; Step 23-3, generate intermediate feature vector using nonlinear activation function , along the spatial dimension, the feature map Perform channel separation to generate two feature vectors and ; Step 23-4, using 1×1 convolution and Perform dimensionality increase operation and obtain the sizes of and The feature map of Step 23-5, then use the Sigmoid activation function to get the attention vector and , will obtain and The feature vector is multiplied by the feature weight to obtain an image with enhanced feature expression at the output of CA.

[0026] Furthermore, the Ghost module in step 24 includes two parts. One part uses ordinary convolution to compress the number of channels of the input image and generates some intrinsic feature maps through identity mapping; the other part uses a series of linear operations to Ф Generate a ghost feature map to achieve feature expansion while increasing the number of channels to obtain more feature maps and further enrich the feature expression; finally, concatenate the ghost feature map with the intrinsic feature map in the channel dimension to obtain an output of the same dimension as the traditional convolution.

[0027] Furthermore, in step 3, the Adam optimizer is used for training, the cross entropy loss function is used as the loss function, the batch size is set to 24, the learning rate is 0.001, and the number of iterations is 300.

[0028] The specific principle of the present invention is described in detail below: Specifically, as an embodiment, the tea image dataset used in the present invention adopts the Tieguanyin image dataset, which was collected at the Tieguanyin production demonstration base in Anxi County, Quanzhou City, Fujian Province. A batch of images was collected after each shaking and 20 minutes before the end of the cooling. The images taken after the first shaking are called after the first shaking (after the first shaking), and the images taken before the end of the first cooling are called before the second shaking (before the second shaking), and so on. Image data of six stages of the Tieguanyin green making process were collected respectively, namely, after the first shaking, before the second shaking, after the second shaking, before the third shaking, after the third shaking, and at the end of the green making. The image samples collected at each stage are as follows. Figure 2 shown.

[0029] The training of the task of identifying the degree of greening of Tieguanyin belongs to supervised learning, which requires a large number of samples for training. The present invention collects and screens samples of the greening process of Tieguanyin spring tea and autumn tea, and obtains a total of 4561 images. In order to improve the generalization ability of the model and avoid overfitting, the present invention uses data enhancement methods such as horizontal mirror flipping, vertical mirror flipping, adjusting brightness, contrast and saturation to process the above 4561 images to generate new training samples, and the enhanced data set has a total of 7618 images. The enhanced data set is divided into training set, validation set and test set in a ratio of 7:2:1. The enhanced Tieguanyin greening data set is shown in Table 1.

[0030] Table 1 Tieguanyin tea making dataset Green stage Training set / sheet Validation set / sheet Test set / sheet total After a shake 866 247 123 1236 Before the second shake 887 253 126 1266 After the second shake 911 259 129 1299 Before the third shake 930 265 132 1327 After three shakes 878 250 125 1253 Finished 867 247 123 1237 total 5339 1521 758 7618 The Tieguanyin greening degree recognition model ISTO proposed in the present invention is generated based on the ShuffleNet V2 model. This is because ShuffleNet V2 has good feature extraction capabilities and can better identify the subtle and complex features of Tieguanyin at different greening stages, thereby improving the accuracy and reliability of classification. In addition, the lightweight design of ShuffleNet V2 also facilitates the deployment of the model on mobile terminal devices, which can reduce production costs and is easy to promote.

[0031] Due to the fact that the characteristics of tea leaves in adjacent stages are not very different and the color depth of tea leaves in the same stage varies, it is difficult to identify the degree of greenness of Tieguanyin. The ShuffleNet V2 model still has problems in missing features and not paying enough attention to feature points in identifying the degree of greenness of Tieguanyin, resulting in an identification accuracy of only 90.90%.

[0032] like Figure 1 As shown, in order to improve the recognition accuracy and lightweight of the model, the present invention will improve the ShuffleNet V2 model from five aspects: CA attention mechanism, residual structure, multi-scale shallow feature extraction module, ReLU6 activation function and Ghost module.

[0033] Introducing CA attention mechanism: In the identification of the degree of greenness of Tieguanyin, subtle differences in the color, shape, and number and distribution of brown spots of tea leaves will affect the identification of the degree of greenness. Figure 3 As shown in the figure, in order to enable the attention mechanism to capture the features of different positions in the image more finely and enhance the model's ability to capture local features, the CA (coordinate attention) attention mechanism is introduced in the basic feature extraction module of the ShuffleNet V2 model.

[0034] First, the size is The input feature maps are respectively along the height H and width W Average pooling is performed in two directions to obtain two sizes of and It is at a high and width The pooling operation of the direction is as shown in formula (1) and formula (2).

[0035] Secondly, the two feature maps containing specific direction information obtained in the previous step are concatenated in the third dimension, and then the dimension is reduced by 1×1 convolution to obtain a size of The feature map of is the dimension reduction coefficient when the channel is downsampled.

[0036] Next, a nonlinear activation function is used to generate an intermediate feature vector , along the spatial dimension, the feature map f Perform channel separation operation to generate two feature vectors and .

[0037] Then, using 1×1 convolution and Perform dimensionality increase operation and obtain the sizes of and feature map.

[0038] Finally, the attention vector is obtained through the Sigmoid activation function and . and The feature vector is multiplied by the feature weight to obtain an image with enhanced feature expression at the output of CA.

[0039] By introducing the CA attention mechanism, the improved model can more accurately capture the key feature information of the shaking and cooling green of Tieguanyin. The visual comparison of the feature capture before and after the model improvement is as follows: Figure 4 As shown in the figure, the first row is the image after three shakes, and the second row is the image at the end of the tea making process. In these two adjacent stages, the tea color, shape and "red border" features of the original image are very similar. The original ShuffleNet V2 model can only identify the features of the part with more significant color changes; after adding CA attention, the attention to the brown part of the leaf stalk and leaf edge is significantly enhanced, which verifies the effectiveness of adding the CA attention mechanism.

[0040] Introducing residual connections: In order to ensure that the model can maintain and transmit important feature information and effectively alleviate the gradient vanishing problem, the ISTO model proposed in this paper introduces residual connections in both the basic unit and the downsampling unit of the ShuffleNet V2 model. Figure 5 As shown, the dotted box is the structure of the basic unit and downsampling unit of the original model. In the basic unit, the ISTO model adds a rightmost branch (residual connection), which can directly add the input feature map and the result of the input feature map after transformation by the middle branch. In the downsampling unit, since the stride of the 3×3 depth convolution is 2 (Stride=2), the dimension of the output feature map is reduced and is no longer consistent with the input feature map. Therefore, the input and output feature maps cannot be directly added. In order to achieve the connection between the two, the present invention designs a specific residual structure ( Figure 5The rightmost branch of b), which contains a maximum pooling layer for downsampling and a 1×1 ordinary convolution for feature extraction, ensures that the feature map can retain semantic information after the spatial dimension is reduced.

[0041] The comparison curve of the model's training loss value before and after the introduction of residual connection is as follows Figure 6 As shown. Figure 6 It can be seen that the curve with residual connection drops more rapidly in the early stage of training (about the first 50 rounds), indicating that the network converges faster. In the middle and late stages of training, the network loss curve with residual structure is more stable and has less fluctuation, so the introduction of residual structure can help the network learn more stably. At the same time, the value of the orange curve steadily decreases throughout the training process but does not stagnate, so the residual structure effectively alleviates the problem of gradient disappearance.

[0042] Replace shallow convolution: Since the shallow convolution of the ShuffleNet V2 model consists of 3×3 ordinary convolution and maximum pooling layer, the features extracted by this structure are relatively simple and the receptive field is fixed. This results in the shallow convolution being unable to fully capture the subtle color and morphological changes of the tea leaves during the shaking and cooling processes. Therefore, the present invention designs a multi-scale shallow feature extraction (MSFE) module, such as Figure 7 This module combines a maximum pooling layer and multiple 3×3 convolutional layers, and uses a ReLU6 activation function to enhance the response of the shallow network to the changes in leaf characteristics at different stages of the tea-making process, and can accurately capture features of different granularities.

[0043] The visualization comparison of the model's feature capture before and after adding the MSFE module is shown below. Figure 8 shown. Figure 8 b is the leaf feature detection result with only CA attention mechanism added. Figure 8 c is the leaf feature detection result with both CA attention and MSFE modules added. It can be clearly seen from the visualization that the addition of the MSFE module further enhances the recognition of the "red border" of the leaf, and also recognizes the brown stripe feature in the middle of the leaf that CA did not notice, which shows the effectiveness of the improvement.

[0044] Modify the activation function: ShuffleNet V2 model uses ReLU activation function. ReLU is efficient and simple to calculate, and is widely used in various neural network models. It can effectively alleviate the gradient vanishing problem. However, its disadvantage is that the output upper limit is not limited (such as Fig. 9 a), which results in an activation value that is too large, thus increasing the risk of computational overflow.

[0045] The ISTO model proposed in the present invention changes the activation function of the ShuffleNet V2 model to ReLU6. In contrast, ReLU6 limits the activation value to between 0 and 6 (e.g. Fig. 9 b), this limitation effectively prevents the activation value from being too large and reduces the risk of computational overflow. In addition, ReLU6 has better control over the amplitude of the model weights, which helps stabilize the training process and reduce the risk of overfitting.

[0046] Lightweight improvement: In order to make the model lightweight, the ISTO model proposed in this paper replaces the first 1×1 convolution of each branch in the basic unit and downsampling unit of the ShuffleNet V2 model with a Ghost module. The Ghost module is a lightweight convolution module that reduces the amount of calculation and parameters of the model by reducing redundant calculations while ensuring the recognition accuracy of the model. Under the premise of ensuring that the size of the output feature map remains unchanged, the number of parameters and computational complexity required by the Ghost module are lower than those of traditional convolutional neural networks. The structure of the Ghost module is as follows: Fig.10 As shown in the figure, it consists of two main parts. One part uses ordinary convolution to compress the number of channels of the input image and generates some intrinsic feature maps through identity mapping; the other part uses a series of linear operations to Ф While achieving feature expansion, the number of channels is increased to obtain more feature maps and further enrich the feature expression. Finally, the ghost feature map and the intrinsic feature map are concatenated in the channel dimension to obtain an output of the same dimension as the traditional convolution.

[0047] Considering that the second 1×1 convolution in the right branch of the basic unit and downsampling unit of the ShuffleNet V2 model does not change the number of channels and only plays the role of information fusion between channels, which may cause information redundancy, the ISTO model proposed in this invention deletes this convolution, making the improved model further lightweight. Fig.11 As shown in FIG. 1 , the basic unit and downsampling unit structures after the above two improvements are shown.

[0048] In order to verify the impact of the above lightweight improvements on model complexity, the model that only undergoes the above lightweight adjustments is compared with the original model, and the test results are shown in Table 2. From the test results, it can be seen that after the lightweight improvement, the number of model parameters and model size are reduced by nearly half.

[0049] Table 2 Lightweight improvement experimental results Model Parameter quantity / M Model size / MB ShuffleNet V2 5.357 20.57 ShuffleNet V2+ lightweight 3.006 11.60 Model performance: In order to verify that the series of improvement measures of the ShuffleNet V2 model in the present invention are effective in improving the recognition performance of the degree of greenness of Tieguanyin, an ablation test was conducted, and the test results are shown in Table 3. The table uses accuracy, F1 score, number of parameters and model size to evaluate the performance of the model. The F1 score is a comprehensive indicator that combines precision and recall. The higher the F1 score value, the better the overall performance of the model.

[0050] From the data in Table 3, we can see that the ISTO model (the last row of data) has significantly improved the performance of Tieguanyin tea green degree recognition compared with the ShuffleNet V2 model (the first row of data). The recognition accuracy rate has increased by 3.03%, the F1 score has increased by 3.04%, and the number of parameters and model size have been reduced by nearly 30%.

[0051] Table 3 Ablation experiments of ISTO model Table 4 is a comparison of the performance of the ISTO model and other classification models. From the data in Table 4, it can be seen that among the conventional models GoogLeNet, ResNet18 and the lightweight model EffientNet B0, ResNet18 has the best performance. However, compared with the ResNet18 model, the ISTO model proposed in the present invention not only improves the accuracy by 11.34%, but also reduces the model size and parameter amount by more than half. It can be seen that the ISTO model has achieved good performance in both recognition accuracy and model complexity.

[0052] Table 4 Performance comparison of different models Model Accuracy / % Accuracy / % Recall rate / % <![CDATA[F1 score / %]]> Parameter quantity / M Model size / MB GoogLeNet 62.80 63.62 63.06 62.03 5.980 43.37 ResNet18 82.59 82.75 82.63 82.64 11.180 42.68 EfficientNet B0 71.90 74.10 72.21 72.09 4.015 15.48 ISTO 93.93 93.98 93.97 93.95 3.902 15.05 Among the three comparison models mentioned above, since the ResNet18 model has the best performance, its confusion matrix is ​​compared with the ISTO model confusion matrix, as shown in the figure below: Fig.12 As shown in the figure. As can be seen from the figure, due to the high similarity of tea characteristics after the first shake and before the second shake of Tieguanyin, and the fact that the tea characteristics of two adjacent stages do not change significantly, both models are confused. However, in comparison, among the 758 pictures in the test set, the ISTO model only made 46 mistakes, while the ResNet18 model made 132 mistakes, which is nearly three times the number of the ISTO model. It can be seen that the ISTO model performs better in identifying the degree of greenness of Tieguanyin. Fig.12 1 is after one shake; 2 is before two shakes; 3 is after two shakes; 4 is before three shakes; 5 is after three shakes; 6 is the end of the green.

[0053] The present invention adopts the above technical solution, uses industrial cameras to collect tea greening images, and through data enhancement, makes a tea greening dataset for model training, verification and testing. Secondly, by adding CA attention mechanism and residual structure connection to the basic feature extraction module (basic unit and downsampling unit) of ShuffleNet V2 model, using multi-scale shallow feature extraction (MSFE) module in the shallow convolution of the model and using ReLU6 to replace ReLU activation function, etc., four improvements are made to improve the model's recognition accuracy of the degree of tea greening. Finally, the 1×1 convolution in the basic feature extraction module is replaced with the Ghost module, and the second 1×1 convolution is deleted to make the model lightweight.

[0054] The present invention uses the improved ShuffleNet V2 model to classify and identify the color, shape and distribution position of brown spots of green tea, and finally realizes the intelligent judgment of the degree of greenness of tea leaves. At the same time, by improving the lightweight of the model, the model is easy to deploy on mobile terminal devices.

[0055] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2, characterized in that: It includes the following steps: Step 1, obtain tea leaves images and obtain a tea leaves dataset through data enhancement, and divide the tea leaves dataset into a training set, a validation set, and a test set according to a set ratio; Step 2, the CA attention mechanism, residual structure, multi-scale shallow feature extraction module, ReLU6 activation function and Ghost module are used to improve the ShuffleNet V2 convolutional neural network to obtain an improved ShuffleNet V2 model; Step 3, use the training set to train the improved ShuffleNet V2 model, and use the validation set to validate the trained model, so as to optimize the model parameters to obtain the final improved ShuffleNet V2 model; Step 4: Input the test samples of the test set into the final improved ShuffleNet V2 model, predict and output the classification and recognition results of the color, shape and distribution position of brown spots of the test samples, and complete the recognition of the degree of greenness of the tea leaves.

2. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 1, characterized in that: In step 1, an industrial camera is used to collect images of a batch of tea leaves 20 minutes after each shaking and before the end of the cooling.

3. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 1, characterized in that: In step 1, the tea-leaf greening image is processed by horizontal mirror flipping, vertical mirror flipping, adjusting brightness, adjusting contrast and adjusting saturation to generate the tea-leaf greening dataset.

4. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 1, characterized in that: In step 1, the tea-making dataset is divided into training set, validation set and test set in a ratio of 7:2:

1.

5. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 1, characterized in that: Step 2 The specific steps to obtain the improved ShuffleNet V2 model are as follows: Step 21, build the ShuffleNet V2 backbone network, which includes an input layer, a convolutional layer 1, a stage 2 layer, a stage 3 layer, a stage 4 layer, a convolutional layer 2, a global pooling layer, and a fully connected layer; Step 22, replace the convolution layer 1 of the ShuffleNet V2 model with a multi-scale shallow feature extraction module MSFE; the input features of the multi-scale shallow feature extraction module are divided into three paths, one path passes through a 3×3 maximum pooling layer with a stride of 2 and a 1×1 convolution layer; another path passes through a 3×3 convolution layer with a stride of 2 in sequence; the last path passes through a 3×3 convolution layer with a stride of 2 and a stride of 1 in sequence; the output features of the three paths are concatenated to form the output features of the multi-scale shallow feature extraction module; Step 23, adding CA attention mechanism and residual structure connection to the basic feature extraction modules of the second layer, third layer, and fourth layer of the ShuffleNet V2 model; The basic feature extraction module includes a basic unit and a downsampling unit; In step 24, the first 1×1 convolution of each branch in the basic unit and downsampling unit of the ShuffleNet V2 model is replaced with a Ghost module, and the second 1×1 convolution in each branch is deleted to make the model lightweight, thereby obtaining an improved ShuffleNet V2 model.

6. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 5, characterized in that: In step 22, the activation functions of the multi-scale shallow feature extraction module and the second layer, third layer, fourth layer, and convolutional layer 2 are replaced with ReLU6.

7. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 5, characterized in that: In step 23, the CA attention mechanism specifically performs the following steps: Step 23-1, change the size to The input feature maps are respectively along the height and width Average pooling is performed in two directions to obtain two sizes of and The characteristic map of and width The pooling operation formula for the direction is as follows: in, is the feature map in the height direction; Feature map in width direction, is the number of channels; Indicates the width position of the current pooling operation, Indicates the height position of the current pooling operation, and It is used to represent the index along the height and width direction during the pooling operation; Step 23-2, the two feature maps containing specific direction information are concat-concatenated in the third dimension, and then the dimension is reduced by 1×1 convolution to obtain a size of The feature map of is the dimension reduction coefficient when the channel is downsampled; Step 23-3, generate intermediate feature vector using nonlinear activation function , along the spatial dimension, the feature map Perform channel separation to generate two feature vectors and ; Step 23-4, using 1×1 convolution and Perform dimensionality increase operation and obtain the sizes of and The feature map of Step 23-5, then use the Sigmoid activation function to get the attention vector and , will obtain and The feature vector is multiplied by the feature weight to obtain an image with enhanced feature expression at the output of CA.

8. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 5, characterized in that: The Ghost module in step 24 consists of two parts. One part uses ordinary convolution to compress the number of channels of the input image and generates some intrinsic feature maps through identity mapping; the other part uses a series of linear operations to Ф Generate a ghost feature map to achieve feature expansion while increasing the number of channels to obtain more feature maps and further enrich the feature expression; finally, concatenate the ghost feature map with the intrinsic feature map in the channel dimension to obtain an output of the same dimension as the traditional convolution.

9. A method for identifying the degree of greenness of tea leaves based on improved ShuffleNet V2 according to claim 1, characterized in that: In step 3, the Adam optimizer is used for training, the cross entropy loss function is used as the loss function, the batch size is set to 24, the learning rate is 0.001, and the number of iterations is 300.

Images