A method, system and terminal device for identifying fresh tea leaves based on deep learning

Abstract

The present invention discloses a tea green recognition method, system and terminal device based on deep learning, including: collecting tea green sample images by using a collecting device, and enhancing the tea green sample images by using a traditional data augmentation method; embedding dual coordinate attention into a generative adversarial network to construct a reinforced generative adversarial network, and further augmenting the tea green sample images to generate a tea green image augmented data set; under a deep learning framework, constructing a multi-channel ghost network with a multi-channel attention module as the core, and pre-training the multi-channel ghost network; using the pre-trained multi-channel ghost network to train the tea green image augmented data set to realize the recognition of tea green images. The present invention combines traditional data enhancement technology and a reinforced generative adversarial network for data augmentation, effectively enhances the effect of small-scale images, generates new effective tea green image data, improves the quality and diversity of images, and can better focus on the regions of interest in the images.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a tea green recognition method, system and terminal device based on deep learning. Background Art

[0002] With the progress of technology, tea picking has changed from manual operation in the past to mechanical tea picking. While improving the picking efficiency, it inevitably mixes different grades of tea together. Currently, traditional data augmentation methods are used for data expansion in tea green image acquisition. Through geometric transformation, color transformation, main pixel transformation, etc., enhancement is carried out on the existing images, which improves the algorithm robustness within a certain range and enhances the generalization ability of the algorithm. However, due to the limited amount of data and single data type in the enhancement algorithm, the dataset expansion effect obtained in specific cases is not good, and based on the same dataset for expansion, the correlation of the images is too strong and the diversity is insufficient.

[0003] Most of the traditional grading of tea green adopts the manual evaluation method, which has defects such as subjectivity and empiricism, and consumes a large amount of manpower and material resources. Using the traditional convolutional neural network to recognize tea green, the model cannot well express the importance of complex tea green image features in different channels on a small dataset, and it is easy to deviate from the overall positioning of key features, resulting in a reduction in tea green recognition performance. Summary of the Invention

[0004] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part, as well as in the abstract and title of the specification of this application, to avoid obscuring the purpose of this part, the abstract and the title of the specification, and such simplifications or omissions shall not be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the technical problems solved by the present invention are: the training samples in the existing small datasets are overfitted and have poor generalization ability, the images generated by traditional data augmentation methods lack diversity, the traditional convolutional neural network cannot effectively judge the importance of complex tea green image features in different channels, it is easy to deviate from the overall positioning of key features, and the tea green recognition performance is low.

[0007] To solve the above technical problems, the present invention provides the following technical solutions, including:

[0008] Collect tea green sample images using a collection device, and enhance the tea green sample images using traditional data augmentation methods;

[0009] Embed dual coordinate attention into the generative adversarial network to construct a reinforced generative adversarial network, and further expand the tea green sample images to generate a tea green image augmented dataset;

[0010] Under the deep learning framework, construct a multi-channel ghost network with a multi-channel attention module as the core, and pre-train the multi-channel ghost network;

[0011] Use the pre-trained multi-channel ghost network to train the tea green image augmented dataset to achieve the recognition of tea green images.

[0012] As a preferred solution of the tea green recognition method based on deep learning according to the present invention, wherein:

[0013] Enhance the tea green sample images using traditional data augmentation methods, including: performing traditional augmentation on the collected tea green sample images through 10 data enhancement methods such as flipping, translation, rotation, random noise, scaling, grayscaling, shearing, brightness change, affine transformation, and blurring.

[0014] As a preferred solution of the tea green recognition method based on deep learning according to the present invention, wherein:

[0015] Embed dual coordinate attention into the generative adversarial network to construct a reinforced generative adversarial network, and further augment and enhance the tea green samples, including: a generative model G and a discriminative model D;

[0016] The generative model G embeds an attention module and uses transposed convolution to expand a one-dimensional vector into a two-dimensional image, expressed as:

[0017] G1(x) = ReLU(BN(Transpose2d(f(Transform(x)))))

[0018] G2(x) = ReLU(BN(Transpose2d(G1(x))))

[0019] G3(x) = ReLU(BN(Transpose2d(G2(x))))

[0020] G4(x) = ReLU(BN(Transpose2d(G3(x))))

[0021] G5(x) = Tanh(Transpose2d(G4(x)))

[0022] Among them, G is the generation model in the enhanced generative adversarial network, G1 is the feature map obtained by training the first layer of the generation model, G2 is the feature map obtained by training the second layer of the generation model, G3 is the feature map obtained by training the third layer of the generation model, G4 is the feature map obtained by training the fourth layer of the generation model, G5 is the feature map finally obtained by the generation model, Transpose2d is the transposed convolution, x is the input image feature, Transform is the matrix transformation, f is the input coordinate attention, BN is the normalization process, ReLU and Tanh are activation functions;

[0023] The discriminative model D is expressed as:

[0024] D1(x) = LeakyReLU(BN(conv(f(x))))

[0025] D2(x) = LeakyReLU(BN(conv(D1(x))))

[0026] D3(x) = LeakyReLU(BN(conv(D2(x))))

[0027] D4(x) = LeakyReLU(BN(conv(D3(x))))

[0028] D5(x) = Sigmoid(conv(D4(x)))

[0029] Among them, D is the discriminative model in the enhanced generative adversarial network, D1 is the feature map obtained after training the first layer of the discriminative model, D2 is the feature map obtained after training the second layer of the discriminative model, D3 is the feature map obtained after training the third layer of the discriminative model, D4 is the feature map obtained after training the fourth layer of the discriminative model, D5 is the feature map finally obtained by the discriminative model, Sigmoid is the activation function, x is the input feature, f is the input coordinate attention, conv is the convolution process, BN is the normalization process, and LeakyReLU is the activation function.

[0030] As a preferred solution of the tea green recognition method based on deep learning according to the present invention, among them: The multi-channel attention module includes:

[0031] According to the channel attention mechanism, perform a pooling operation on the input feature, map the weights of the importance of each channel through a multi-layer perceptron, perform a non-linear transformation using the activation function, compress the weight range into the interval [0, 1], and finally construct an identity mapping layer to achieve the adaptive combination of the convolutional layer to obtain the eigenvalue x1;

[0032] Use the pooling layer to encode the input feature along the horizontal coordinate and the vertical coordinate respectively, and perform non-linear superposition using cross-layer skip connections to obtain the eigenvalue x2;

[0033] The eigenvalues x1 and x2 are stacked, and the dimensionality is reduced using point-by-point convolution to finally obtain the output eigenvalue y, thereby accurately locking the key information.

[0034] As a preferred solution of the method for identifying green tea leaves based on deep learning described in the present invention, wherein:

[0035] The eigenvalue x1 is expressed as:

[0036] x1=Sigmoid(MLP[MaxPool(x)]+MLP[AvgPool(x)])+x

[0037] Among them, MaxPool represents maximum pooling processing, AvgPool represents average pooling processing, MLP represents multi-layer perceptron, Sigmoid is the activation function, and x is the input feature;

[0038] The eigenvalue x2 is expressed as:

[0039] x2=Sigmoid(conv(BN(conv(AvgPool(X)+AvgPool(Y)))))+x

[0040] Among them, AvgPool represents average pooling processing, Sigmoid is the activation function, AvgPool represents average pooling processing, conv is convolution processing, BN is normalization processing, and x is the input feature;

[0041] The output eigenvalue y is expressed as:

[0042] y=BN(conv(BN(x1+x2)))

[0043] Among them, conv is convolution processing and BN is normalization processing.

[0044] As a preferred solution of the method for identifying green tea leaves based on deep learning described in the present invention, wherein:

[0045] The multi-channel ghost network includes: feature information w ij The ghost module and ghost bottleneck that are connected with the intrinsic feature w to output feature information;

[0046] The intrinsic feature w is expressed as:

[0047] w=conv(f)

[0048] Among them, f is the input coordinate attention, conv is the convolution process, and w is the intrinsic feature of the inherent number generated by point-by-point convolution;

[0049] The characteristic information w ij , expressed as:

[0050]

[0051] where \(w\) i ′ is the \(i\)-th feature in the intrinsic feature \(w\), is a linear operation for generating the \(i\)-th associated feature; [[ID=k11]]

[0052] Build a ghost bottleneck using the ghost module, and stack the ghost bottlenecks to build a multi-channel ghost network.

[0053] As a preferred solution of the tea green identification method based on deep learning according to the present invention, wherein:

[0054] The pre-training of the multi-channel ghost network includes:

[0055] Using the Adam optimizer, iteratively update the weights of the multi-channel network ghost network with the help of training data, expressed as:

[0056]

[0057] where ] is the bias correction of the first-order momentum, is the bias correction of the second-order momentum, \(m\) t is the first-order moment estimate of the gradient, \(v\) t is the second-order moment estimate of the gradient, \(\omega\) t is the parameter of the \(t\)-th round, \(\omega\) t+1 is the updated parameter, and \(lr\) is the learning rate;

[0058] Use the loss function for training, expressed as: *

[0059] FL(x i ) = -(1 - x i ) α lg(x i )

[0060] where \(x\) i is the probability that the tea green image belongs to a certain category, \(i = 1, 2, 3, 4\), \(\alpha\) is the focusing parameter and \(0\leq\alpha\leq5\), \((1 - x i ) α is the modulation factor, and its value range is \([0, 1]\).

[0061] As a preferred solution of the tea green identification method based on deep learning according to the present invention, wherein:

[0062] The pre-trained multi-channel ghost network is trained on the augmented dataset of tea green images to achieve the identification of tea green images, including:

[0063] Migrate the saved weight file to the multi-channel GhostNet, import the augmented dataset of tea green images, and replace the last fully connected layer of the multi-channel GhostNet for training;

[0064] When the learning rate is too high, the gradient descent speed is fast but it is difficult to reach the optimal solution, resulting in the network being unable to converge. When the learning rate is too low, local optimal solutions are likely to occur, resulting in poor tea green recognition effects;

[0065] After training and comparison, set the learning rate to 0.0001 to achieve the recognition of tea green images.

[0066] To solve the above technical problems, the present invention provides the following technical solutions: A tea green recognition system based on deep learning, characterized by comprising:

[0067] An image acquisition module, used to acquire tea green sample images and send the tea green sample images to the data enhancement module;

[0068] A data enhancement module, used to perform enhancement processing and analysis on the acquired tea green sample images by using traditional data enhancement methods and reinforcement generative adversarial networks, and generate an augmented dataset of tea green images;

[0069] A multi-channel GhostNet module, used to take the multi-channel attention module as the core, perform channel information localization on each input tea green image, and mine effective feature information;

[0070] A pre-training module, used to pre-train the multi-channel GhostNet module;

[0071] A tea green recognition module, used to use transfer learning to apply the knowledge learned on the dataset to the augmented dataset of tea green images to achieve the recognition of tea green images;

[0072] A hierarchical performance verification module, used to perform hierarchical performance verification on the tea green recognition method.

[0073] The present invention also proposes a terminal device based on deep learning, characterized by comprising

[0074] A memory and a processor; store the tea green recognition system in the memory and run it on the processor to achieve the visualization of the tea green recognition system; when the tea green recognition system based on deep learning is executed by the processor, the tea green recognition method is implemented.

[0075] Advantages of the present invention: The present invention combines an enhanced generative adversarial network and a multi-channel attention mechanism. By embedding the attention mechanism, it focuses on the positions where important hierarchical features are located, accelerates the convergence speed of the network, improves the diversity of the generated data and the classification accuracy of the trainer; constructs a multi-channel attention module based on coordinate attention and channel attention, and introduces a shortcut connection to prevent the loss of information during the feature extraction process, solves the problem of deep network degradation, can capture information across channels when strengthening feature allocation, while enhancing the influence of important channels in the feature map, can also highlight the importance of the position where it is located to the output result, and achieves accurate positioning of the region of interest; embeds the multi-channel attention module into the ghost module to achieve the purpose of focusing on strong classification feature regions and suppressing weak classification regions, further extracts feature information, reduces the computational redundancy of the attention module, and uses the visual attention mechanism that most conforms to humans to recognize things, realizing the recognition of fresh tea leaves. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0077] Figure 1 It is a technical roadmap of a fresh tea leaf recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0078] Figure 2 It is a schematic diagram of a fresh tea leaf recognition system of a fresh tea leaf recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0079] Figure 3 It is a schematic diagram of a multi-channel attention module of a fresh tea leaf recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0080] Figure 4 It is a schematic diagram of a ghost bottleneck of a fresh tea leaf recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0081] Figure 5 It is a schematic diagram of a fresh tea leaf image of a fresh tea leaf recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0082] Figure 6 It is a schematic diagram of a fresh tea leaf image after being generated by the enhanced generative network of a fresh tea leaf recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0083] Figure 7 Schematic diagram for comparing the tea green recognition accuracy of a tea green recognition method, system and terminal device based on deep learning according to an embodiment of the present invention;

[0084] Figure 8 Schematic diagram for comparing the tea green recognition loss values of a tea green recognition method, system and terminal device based on deep learning according to an embodiment of the present invention. Detailed implementation manners

[0085] To make the above objects, features and advantages of the present invention more obvious and understandable, the following detailed description of the specific implementation manners of the present invention will be given in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention shall fall within the protection scope of the present invention.

[0086] In the following description, many specific details are set forth to facilitate a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0087] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure or characteristic that may be included in at least one implementation manner of the present invention. The "in one embodiment" appearing in different places in this specification does not all refer to the same embodiment, nor is it a separate or selectively exclusive embodiment from other embodiments.

[0088] The present invention will be described in detail in conjunction with the schematic diagrams. When detailing the embodiments of the present invention, for the sake of convenience of explanation, the cross-sectional views showing the device structures will be enlarged locally out of the general scale, and the schematic diagrams are only examples, which should not limit the protection scope of the present invention herein. In addition, in actual production, three-dimensional spatial dimensions including length, width and depth should be included.

[0089] Meanwhile, in the description of the present invention, it should be noted that the orientation or positional relationship indicated by terms such as "upper, lower, inner and outer" is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and thus cannot be construed as a limitation to the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0090] Unless otherwise clearly defined and limited in this invention, the terms "installation, connection, and coupling" shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may also be a mechanical connection, an electrical connection, or a direct connection, or it may be indirectly connected through an intermediate medium, or it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in this invention can be understood according to specific situations.

[0091] Example 1

[0092] Referring to Figures 1 to 4 , which is the first embodiment of the present invention. This embodiment provides a tea green recognition method based on deep learning, including:

[0093] S1: Use a collection device to collect tea green sample images, and use traditional data augmentation methods to enhance the tea green sample images;

[0094] Furthermore, using traditional data augmentation methods to enhance the tea green sample images includes: performing traditional augmentation on the collected tea green sample images through 10 data augmentation methods, namely flipping, translation, rotation, random noise, scaling, grayscale conversion, shearing, brightness change, affine transformation, and blurring.

[0095] S2: Embed double coordinate attention into the generative adversarial network to construct a strengthened generative adversarial network, and further augment the tea green sample images to generate a tea green image augmented dataset;

[0096] Furthermore, embedding double coordinate attention into the generative adversarial network to construct a strengthened generative adversarial network and further augmenting and enhancing the tea green samples includes: a generative model G and a discriminative model D;

[0097] It should be noted that by embedding the attention mechanism, focusing on the location of important hierarchical features, the convergence speed of the network is accelerated, the diversity of the generated data and the classification accuracy of the trainer are improved, and effective samples can be generated after about 200 iterations.

[0098] The generative model G embeds an attention module and uses transposed convolution to expand a one-dimensional vector into a two-dimensional image, which is expressed as:

[0099] G1(x) = ReLU(BN(Transpose2d(f(Transform(x)))))

[0100] G2(x) = ReLU(BN(Transpose2d(G1(x))))

[0101] G3(x) = ReLU(BN(Transpose2d(G2(x))))

[0102] G4(x) = ReLU(BN(Transpose2d(G3(x))))

[0103] G5(x) = Tanh(Transpose2d(G4(x)))

[0104] Where G is the generator model in the enhanced generative adversarial network, G1 is the feature map obtained from the first layer training of the generator model, G2 is the feature map obtained from the second layer training of the generator model, G3 is the feature map obtained from the third layer training of the generator model, G4 is the feature map obtained from the fourth layer training of the generator model, G5 is the final feature map obtained by the generator model, Transpose2d is the transposed convolution, x is the input image feature, Transform is the matrix transformation, f is the input coordinate attention, BN is the normalization process, ReLU and Tanh are activation functions;

[0105] The discriminator model D is expressed as:

[0106] D1(x) = LeakyReLU(BN(conv(f(x))))

[0107] D2(x) = LeakyReLU(BN(conv(D1(x))))

[0108] D3(x) = LeakyReLU(BN(conv(D2(x))))

[0109] D4(x) = LeakyReLU(BN(conv(D3(x))))

[0110] D5(x) = Sigmoid(conv(D4(x)))

[0111] Where D is the discriminator model in the enhanced generative adversarial network, D1 is the feature map obtained after the first layer training of the discriminator model, D2 is the feature map obtained after the second layer training of the discriminator model, D3 is the feature map obtained after the third layer training of the discriminator model, D4 is the feature map obtained after the fourth layer training of the discriminator model, D5 is the final feature map obtained by the discriminator model, Sigmoid is the activation function, x is the input feature, f is the input coordinate attention, conv is the convolution process, BN is the normalization process, LeakyReLU is the activation function.

[0112] It should be noted that the enhanced generative adversarial network mainly consists of two parts: a generative model G and a discriminative model D. The generative model G is responsible for receiving noise z and generating an image G(z). The discriminative model D is responsible for judging the probability that the input image is real. During the entire training process, the goal of the generative model G is to generate an image G(z) that can be mistaken for real, making it impossible for the discriminative model D to distinguish, that is, D(G(z)) = 0.5. At this time, a generative model is obtained to generate tea green images to expand the dataset. By constructing an enhanced generative adversarial network to expand the tea green sample image data, the quality and diversity of the generated images are ensured, and the training process is accelerated.

[0113] S3: Under the deep learning framework, construct a multi-channel ghost network with a multi-channel attention module as the core, and pre-train the multi-channel ghost network;

[0114] Furthermore, the multi-channel attention module includes:

[0115] According to the channel attention mechanism, perform a pooling operation on the input features, map the weights of the importance of each channel through a multi-layer perceptron, perform a non-linear transformation using an activation function, compress the weight range into the interval [0, 1], and finally construct an identity mapping layer to achieve the adaptive combination of convolutional layers, obtaining the eigenvalue x1;

[0116] It should be noted that the average pooling operation is to obtain the eigenvalue describing the background information of the image, and the max pooling operation is to obtain the eigenvalue describing the texture information of the image, generating two different spatial feature maps. The eigenvalue x1 can enhance the expression of the tea green area and suppress the expression of useless information.

[0117] Use the pooling layer to encode the input features along the horizontal and vertical coordinates respectively, and perform non-linear superposition using cross-layer skip connections to obtain the eigenvalue x2;

[0118] It should be noted that using the pooling layer to encode the input image along the horizontal and vertical coordinates respectively can obtain accurate position information of the global encoding, splice the extracted feature information, and use pointwise convolution to transform the information and expand the output result.

[0119] Stack the eigenvalues x1 and x2, use pointwise convolution to reduce the dimension, and finally obtain the output eigenvalue y to achieve precise locking of key information.

[0120] It should be noted that while the output eigenvalue x enhances the influence of important channels in the feature map, it can also emphasize the importance of its position on the output result. [[ID=2s]]

[0121] Furthermore, the eigenvalue x1 is expressed as:

[0122] x1 = Sigmoid(MLP[MaxPool(x)] + MLP[AvgPool(x)]) + x

[0123] Among them, MaxPool represents max pooling processing, AvgPool represents average pooling processing, MLP represents a multi-layer perceptron, Sigmoid is an activation function, and x is an input feature;

[0124] The eigenvalue x2 is expressed as:

[0125] x2 = Sigmoid(conv(BN(conv(AvgPool(X) + AvgPool(Y))))) + x

[0126] Among them, AvgPool represents average pooling processing, Sigmoid is an activation function, AvgPool represents average pooling processing, conv is convolution processing, BN is normalization processing, and x is an input feature;

[0127] The output eigenvalue y is expressed as:

[0128] y = BN(conv(BN(x1 + x2)))

[0129] Among them, conv is convolution processing, and BN is normalization processing.

[0130] It should be noted that a multi-channel attention module is constructed based on coordinate attention and channel attention, and a shortcut connection is introduced to prevent information loss during the feature extraction process, solve the problem of deep network degradation, capture information across channels while strengthening feature allocation, emphasize the importance of the position where it is located for the output result while enhancing the influence of important channels in the feature map, and achieve precise positioning of the region of interest.

[0131] Furthermore, the multi-channel ghost network includes: feature information w ij A ghost module and a ghost bottleneck that are connected to the intrinsic feature w to output feature information;

[0132] The intrinsic feature w is expressed as:

[0133] w = conv(f)

[0134] Among them, f is the input coordinate attention, conv is convolution processing, and w is the intrinsic feature generated by pointwise convolution with an inherent number;

[0135] Feature information w ij , is expressed as:

[0136]

[0137] Among them, w i′ is the \(i\)-th feature in the intrinsic feature \(w\), is a linear operation to generate the \(i\)-th associated feature;

[0138] Use the Ghost module to construct a Ghost bottleneck, and stack Ghost bottlenecks to build a multi-channel Ghost network.

[0139] It should be noted that the input features are processed by the multi-channel attention module, and then the pointwise convolution is used to compress the number of channels of the feature layer to obtain the intrinsic feature \(w\), and then a linear transformation is performed to obtain more feature maps, and finally the generated feature information \(x\) ij is connected with the intrinsic feature \(w\) and the feature information is output.

[0140] When the stride is 1, two Ghost modules are connected in series to form a Ghost bottleneck. The first Ghost module expands the number of channels, and the second Ghost module reduces the number of channels to the same as the input channel number; the residual side part is the same as ResNet, and finally the channel information of the two branches is communicated through the concat operation.

[0141] For spatial downsampling, a Ghost bottleneck with a stride of 2 is designed. A layer-by-layer convolution with stride = 2 is added between two connected Ghost modules, which can compress the height and width of the feature map to reduce its size to 1 / 2 of the input; in the residual side part, a layer-by-layer convolution with a stride of 2×2 and a 1×1 ordinary convolution are added, and an Add operation is performed to increase the dimension of the output features, thereby constructing a Ghost bottleneck.

[0142] Finally, stack Ghost bottlenecks to obtain a new feature layer, use a 1×1 convolution block to adjust the number of channels, and use a fully connected layer for classification after tiling. Build a multi-channel Ghost network, with the multi-channel attention module as the core to build a multi-channel Ghost network, generate a large number of feature maps through a series of simple linear transformations, greatly reduce the computational amount of the network, and at the same time imitate human visual cognition, focus on strong features, suppress weak features, and make full use of small data sets to achieve the identification of tea greens.

[0143] Furthermore, the pre-training of the multi-channel Ghost network includes:

[0144] Using the Adam optimizer, iteratively update the weights of the multi-channel Ghost network with the help of training data, expressed as:

[0145]

[0146] where, is the bias correction of the first-order momentum, is the bias correction of the second-order momentum, \(m\) t is the first-order moment estimate of the gradient, \(v\) t is the second-order moment estimate of the gradient, \(\omega\)t is the parameter for the t-th round, ω t+1 is the updated parameter, and lr is the learning rate;

[0147] It should be noted that for pre-training using the multi-channel Ghost network, the obtained model weights are saved in the PTH file format. During the network training process, the Adam optimizer with lr = 0.01, bets = (0.9, 0.999), and eps = 1e-8 is selected, and the network weights are iteratively updated with the help of the training data.

[0148] The bias of the first-order momentum is corrected and expressed as:

[0149]

[0150] where β1 is the decay rate on m, and β1 ∈ [0, 1];

[0151] The bias of the second-order momentum is corrected and expressed as:

[0152]

[0153] where β2 is the decay rate on v, and β2 ∈ [0, 1];

[0154] Training is carried out using the loss function, expressed as:

[0155] FL(x i ) = -(1 - x i ) α lg(x i )

[0156] where x i is the probability that the tea green image belongs to a certain category, i = 1, 2, 3, 4, α is the focusing parameter and 0 ≤ α ≤ 5, and (1 - x i ) α is the modulation factor, and its value range is [0, 1].

[0157] It should be noted that when using the Focalloss loss function for training, during the network feature extraction, it can effectively solve the problem of class imbalance, reduce the weights of easy-to-classify samples, and focus on difficult-to-classify samples.

[0158] S4: Use the pre-trained multi-channel Ghost network to train the augmented dataset of tea green images to achieve the recognition of tea green images.

[0159] Furthermore, using the pre-trained multi-channel Ghost network to train the augmented dataset of tea green images to achieve the recognition of tea green images includes:

[0160] Migrate the saved weight file to the multi-channel ghost network, import the tea leaf image expansion dataset, and replace the last fully connected layer of the multi-channel ghost network for training;

[0161] When the learning rate is too high, the gradient descent speed is fast but it is difficult to reach the optimal solution, resulting in the network failing to converge. When the learning rate is too low, a local optimal solution is likely to appear, resulting in poor tea leaf recognition results.

[0162] After training and comparison, the learning rate was set to 0.0001 to achieve the recognition of green tea images.

[0163] Example 2

[0164] Reference Figures 1 to 4 , which is the second embodiment of the present invention, differs from the first embodiment in that it provides a green tea identification system based on deep learning, including:

[0165] An image acquisition module is used to obtain green tea sample images and send the green tea sample images to a data enhancement module;

[0166] The data enhancement module is used to enhance and analyze the collected green tea sample images using traditional data enhancement methods and enhanced generative adversarial networks to generate an expanded dataset of green tea images;

[0167] It should be noted that the attention module is embedded in the generative model and the discriminative model, allocating more weights to important feature channels, focusing on important feature information, and making full use of the limited data set for image enhancement. Finally, traditional data enhancement technology and enhanced generative adversarial network are combined for data expansion, which effectively enhances the small-scale image effect, generates new effective tea image data, improves the quality and diversity of the image, accelerates the convergence speed of the network, and improves the diversity of the generated data and the classification accuracy of the trainer.

[0168] The multi-channel ghost network module is used to locate the channel information of each input tea leaf image and mine effective feature information based on the multi-channel attention module;

[0169] It should be noted that a multi-channel attention module is constructed based on coordinate attention and channel attention, and shortcut connections are introduced to prevent information loss during feature extraction, solve the degradation problem of deep networks, capture information across channels while strengthening feature allocation, enhance the influence of important channels in the feature map, and emphasize the importance of their position to the output results, thereby achieving precise positioning of the region of interest.

[0170] Pre-training module, used to pre-train the multi-channel ghost network module;

[0171] The tea green recognition module is used to apply the knowledge learned from the dataset to the tea green image augmented dataset by using transfer learning to achieve the recognition of tea green images;

[0172] It should be noted that by using transfer learning to apply the knowledge learned from the Oxford-17flower dataset to the tea green image augmented dataset, a large number of feature maps are generated through a series of simple linear transformations, greatly reducing the computational amount of the network. At the same time, it imitates human visual cognition, focuses on strong features, suppresses weak features, and fully utilizes the small dataset to achieve tea green recognition.

[0173] The grading performance verification module is used to verify the grading performance of the tea green recognition method.

[0174] It should be noted that the comprehensive performance of the tea green recognition method is measured by indicators such as accuracy, loss value, grading accuracy, precision, recall, and F1-score.

[0175] Example 3

[0176] Refer to Figures 5 to 8 , which is the third embodiment of the present invention. Through comparative experiments, scientific demonstration is carried out to verify the beneficial effects.

[0177] Pick several tea greens of 4 categories of single buds, one bud with one leaf, one bud with two leaves, and one bud with three leaves from the tea factory, identify the intact tea greens, lay them flat independently on white paper, the acquisition device is the camera device of a OnePlus 9R mobile phone, and use the mobile phone's camera function to collect images. Each type of tea green sample collects 800 images, and they are saved in the tea green image database under the corresponding category name; combine two data enhancement methods of traditional data augmentation method and enhanced generative adversarial network to augment the tea green dataset. Each type of tea green database has 1500 images, of which the training dataset is 1125 images and the test dataset is 375 images; under the Pytorch deep learning framework, build a tea green recognition network based on deep learning, establish a system database according to the current tea green's color, texture and other features, pre-train on the Oxford-17flower dataset using a multi-channel ghost network, the tea green recognition network is used to perform the classification and grading of tea greens, deploy the grading system to the hardware platform, and visually display the tea green recognition data.

[0178] According to different tea green recognition methods, grade the tea greens, and the grading accuracy is shown in Table 1:

[0179] Table 1 Comparison data of grading accuracy between the present invention and the prior art

[0180]

[0181] As can be seen from Table 1, due to the simple characteristics of single buds, each model has a high recognition rate. In the classification of one bud with multiple leaves, the classification accuracy rate of the present invention is the highest, and the classification performance far exceeds other traditional models.

[0182] The comparison of comprehensive performance is shown in Table 2:

[0183] Table 2 Comparison data of comprehensive performance between the present invention and the prior art

[0184]

[0185]

[0186] As can be seen from Table 2, in the classification of one bud with one leaf, the present invention achieved precision rates, recall rates and F1-scores of 98.62%, 95.47% and 97.02% respectively; in the classification of one bud with two leaves, the present invention achieved precision rates, recall rates and F1-scores of 96.68%, 93.07% and 94.84% respectively; in the classification of one bud with three leaves, the present invention achieved precision rates, recall rates and F1-scores of 90.8%, 97.33% and 93.95% respectively. Compared with other traditional models, the present invention shows the best classification performance for processing small-scale fresh tea leaf images.

[0187] Combined Figures 5 - 8 As shown: when the iteration period is set to 300, the best accuracy rate of the method of the present invention is 0.968, and the lowest cross-entropy loss value is 0.1003; the best accuracy rate of Inceptionv3-Fresh Tea Leaves is 0.9367, and the lowest cross-entropy loss value is 0.1687; the best accuracy rate of Penut-AlexNet is 0.95, and the lowest cross-entropy loss value is 0.1317; the best accuracy rate of EfficientNet-CBAM is 0.9593, and the lowest cross-entropy loss value is 0.1103. The initial accuracy rate of the accuracy rate and loss value change curve of the present invention is the highest, the loss is the lowest, the model fluctuation is small, the anti-fitting ability is the strongest, the ability to process small sample data is the best, and it can better focus on the interesting regions in the image and improve the quality and diversity of the image.

[0188] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying green tea leaves based on deep learning, characterized in that: include: Collecting green tea sample images using an acquisition device, and enhancing the green tea sample images using a traditional data expansion method; Embedding dual coordinate attention into a generative adversarial network to construct an enhanced generative adversarial network, and further expanding the green tea sample image to generate an expanded green tea image dataset; Under the deep learning framework, a multi-channel ghost network is constructed with a multi-channel attention module as the core, and the multi-channel ghost network is pre-trained; The multi-channel attention module includes: pooling the input features according to the channel attention mechanism, mapping the weights of the importance of each channel through a multi-layer perceptron, using the activation function for nonlinear transformation to compress the weight range to the interval [0, 1], and finally constructing an identity mapping layer to achieve the adaptive combination of convolutional layers to obtain the eigenvalue x1; Use the pooling layer to encode the input features along the horizontal and vertical coordinates respectively, and use cross-layer skip connections for nonlinear superposition to obtain the eigenvalue x2; The eigenvalues x1 and x2 are stacked, and the dimensionality is reduced by point-by-point convolution to finally obtain the output eigenvalue y, thereby accurately locking the key information. The multi-channel ghost network includes: feature information w ij The ghost module and ghost bottleneck that are connected with the intrinsic feature w to output feature information; The intrinsic feature w is expressed as: w=conv(f) Among them, f is the input coordinate attention, conv is the convolution process, and w is the intrinsic feature of the inherent number generated by point-by-point convolution; The characteristic information w ij , expressed as: Among them, w i ′ is the i-th feature in the intrinsic feature w, It is a linear operation that generates the i-th associated feature; Using ghost modules to construct ghost bottlenecks, and stacking the ghost bottlenecks to construct a multi-channel ghost network; The pre-trained multi-channel ghost network is used to train the expanded data set of green tea images to realize the recognition of green tea images.

2. The method for identifying green tea leaves based on deep learning according to claim 1, characterized in that: The green tea sample image is enhanced using traditional data augmentation methods, including: performing traditional augmentation on the collected green tea sample image through 10 data augmentation methods including flipping, translation, rotation, random noise, scaling, grayscale, shearing, brightness change, affine transformation and blurring.

3. The method for identifying green tea leaves based on deep learning according to claim 1 or 2, characterized in that: Embedding dual coordinate attention into the generative adversarial network to build an enhanced generative adversarial network, further expanding and enhancing the green tea samples, including: generative model G and discriminative model D; The generative model G is embedded in the attention module and uses transposed convolution to expand the one-dimensional vector into a two-dimensional image, which can be expressed as: G1(x)=Re LU(BN(Transpose2d(f(Transform(x))))) G2(x)=Re LU(BN(Transpose2d(G1(x)))) G3(x)=Re LU(BN(Transpose2d(G2(x)))) G4(x)=Re LU(BN(Transpose2d(G3(x)))) G5(x)=Tanh(Transpose2d(G4(x))) Among them, G is the generative model in the enhanced generative adversarial network, G1 is the feature map obtained by training the first layer of the generative model, G2 is the feature map obtained by training the second layer of the generative model, G3 is the feature map obtained by training the third layer of the generative model, G4 is the feature map obtained by training the fourth layer of the generative model, G5 is the final feature map obtained by the generative model, Transpose2d is the transposed convolution, x is the input image feature, Transform is the matrix transformation, f is the input coordinate attention, BN is the normalization process, Re LU and Tanh are activation functions; The discriminant model D is expressed as: D1(x)=LeakyReLU(BN(conv(f(x)))) D2(x)=LeakyReLU(BN(conv(D1(x)))) D3(x)=LeakyReLU(BN(conv(D2(x)))) D4(x)=LeakyReLU(BN(conv(D3(x)))) D5(x)=Sigmoid(conv(D4(x))) Among them, D is the discriminant model in the enhanced generative adversarial network, D1 is the feature map obtained after the first layer of the discriminant model is trained, D2 is the feature map obtained after the second layer of the discriminant model is trained, D3 is the feature map obtained after the third layer of the discriminant model is trained, D4 is the feature map obtained after the fourth layer of the discriminant model is trained, D5 is the final feature map obtained by the discriminant model, Sigmoid is the activation function, x is the input feature, f is the input coordinate attention, conv is the convolution processing, BN is the normalization processing, and LeakyRe LU is the activation function.

4. The method for identifying green tea leaves based on deep learning according to claim 3, characterized in that: The eigenvalue x1 is expressed as: x1=Sigmoid(MLP[MaxPool(x)]+MLP[AvgPool(x)])+x Among them, MaxPool represents maximum pooling processing, AvgPool represents average pooling processing, MLP represents multi-layer perceptron, Sigmoid is the activation function, and x is the input feature; The eigenvalue x2 is expressed as: x2=Sigmoid(conv(BN(conv(AvgPool(X)+AvgPool(Y)))))+x Among them, AvgPool represents average pooling processing, Sigmoid is the activation function, AvgPool represents average pooling processing, conv is convolution processing, BN is normalization processing, x is the input feature; the output feature value y, Expressed as: y=BN(conv(BN(x1+x2))) Among them, conv is convolution processing and BN is normalization processing.

5. The method for identifying green tea leaves based on deep learning according to claim 4, characterized in that: Pre-training a multi-channel ghost network involves: Using the Adam optimizer, the weights of the multi-channel network ghost network are iteratively updated with the help of training data, which can be expressed as: in, To correct the deviation of the first-order momentum, To correct the deviation of the second-order momentum, m t is the first-order moment estimate of the gradient, v t is the second-order moment estimate of the gradient, ω t is the parameter of the tth round, ω t+1 is the updated parameter, lr is the learning rate; The loss function is used for training, which is expressed as: FL(x i )=-(1-x i ) α lg(x i ) Among them, x i is the probability that the tea green image belongs to a certain category, i=1,2,3,4, α is the focus parameter and 0≤α≤5, (1-x i ) α is the modulation factor, and its value range is [0, 1].

6. The method for identifying green tea leaves based on deep learning according to claim 5, characterized in that: The pre-trained multi-channel ghost network is trained on the expanded dataset of green tea images to realize the recognition of green tea images, including: Migrate the saved weight file to the multi-channel ghost network, import the tea leaf image expansion dataset, and replace the last fully connected layer of the multi-channel ghost network for training; When the learning rate is too high, the gradient descent speed is fast but it is difficult to reach the optimal solution, resulting in the network failing to converge. When the learning rate is too low, a local optimal solution is likely to appear, resulting in poor tea leaf recognition results. After training and comparison, the learning rate was set to 0.0001 to achieve the recognition of green tea images.

7. A green tea identification system based on deep learning, applied to the method according to any one of claims 1 to 6, characterized in that: include: An image acquisition module is used to obtain green tea sample images and send the green tea sample images to a data enhancement module; The data enhancement module is used to enhance and analyze the collected green tea sample images using traditional data enhancement methods and enhanced generative adversarial networks to generate an expanded dataset of green tea images; The multi-channel ghost network module is used to locate the channel information of each input tea leaf image and mine effective feature information based on the multi-channel attention module; Pre-training module, used to pre-train the multi-channel ghost network module; The green tea recognition module is used to apply the knowledge learned from the dataset to the expanded dataset of green tea images through transfer learning to achieve green tea image recognition; The grading performance verification module is used to verify the grading performance of the tea leaf identification method.

8. A terminal device based on deep learning, characterized in that: include: Memory, processor; The green tea identification system is stored in the memory and runs on the processor to realize the visualization of the green tea identification system; when the green tea identification system based on deep learning is executed by the processor, the green tea identification method as described in any one of claims 1-6 is realized.

Images