Deep learning-based tea non-standardized sampling image identification and classification method
Through a deep learning-based method, the tea classification data set and sub-cluster clustering module are used to solve the problem of low recognition accuracy in traditional tea image recognition methods, and a higher tea image recognition accuracy is achieved.
Patent Information
- Application Number
- CN202411952563.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-22
AI Technical Summary
Traditional tea image recognition methods are limited by factors such as complex background, lighting changes, and tea occlusion, resulting in low recognition accuracy.
Using a deep learning-based method, we collect non-standardized tea images, establish tea classification data sets, and divide them into error-prone subsets and regular subsets to build tea classification models, and use error-prone subsets and regular subsets to improve recognition accuracy.
Through the sub-cluster clustering module and texture enhancement classifier, the difficulty of subsequent classification is reduced and the classification accuracy is improved, and the problem of low recognition accuracy in traditional methods is solved.
Smart Images
Figure CN119863663A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of tea classification and identification, and in particular relates to a method for recognizing and classifying tea non-standardized sampling images based on deep learning. Background Art
[0002] Tea can be divided into six categories according to different processing techniques, namely: green tea, black tea, white tea, yellow tea, dark tea and oolong tea. Each major tea category is further subdivided, each with its own unique flavor and regional characteristics, demonstrating the profoundness and regional diversity of tea culture. The category of tea can be intuitively judged by observing its appearance, so image recognition based on computer vision can realize the recognition and classification of tea images, but this often requires professional equipment and standardized image acquisition conditions. Traditional tea image recognition methods are often limited by factors such as complex background, lighting changes, and tea occlusion, resulting in low recognition accuracy, which restricts the optimization and application of this technology. Summary of the invention
[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a deep learning-based tea non-standardized sampling image recognition and classification method which solves the problem that traditional tea image recognition methods are often limited by factors such as complex background, lighting changes, and tea occlusion, resulting in low recognition accuracy.
[0004] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for recognizing and classifying tea non-standardized sampling images based on deep learning, comprising:
[0005] Collect non-standardized images of different types of dry tea and build a tea classification dataset;
[0006] Divide the tea classification dataset into error-prone subsets and regular subsets;
[0007] Construct a tea classification model, and use the error-prone subset and the regular subset to train the tea classification model to obtain a trained tea classification model;
[0008] Get the tea image to be identified and use the trained tea classification model to get the tea type.
[0009] Furthermore, the tea classification dataset is divided into an error-prone subset and a regular subset, specifically: the convolutional neural network is trained using the tea classification dataset, and after the kth iteration, the images that are incorrectly recognized during training are divided into the error-prone subset until a preset iteration number threshold is reached, and the images contained in the error-prone subset in the tea classification dataset are eliminated to obtain the regular subset.
[0010] Furthermore, the tea classification model includes a feature extraction module, a subset clustering module, a plurality of texture enhancement classifiers, and a classification layer;
[0011] The feature extraction module is used to extract the image features of the error-prone subset and the regular subset respectively, and obtain the initial features of each image in the error-prone subset and the initial features of each image in the regular subset;
[0012] The subset clustering module is used to cluster the images according to the initial features of each image in the error-prone subset to obtain a number of error-prone clusters;
[0013] Each texture enhancement classifier is used to extract texture features and deep features from each error-prone cluster, and fuse the texture features and deep features to obtain fused features, and obtain the classification results of each image in the error-prone cluster based on the fused features;
[0014] The classification layer is used to perform classification according to the features of the conventional image to obtain the classification result of the conventional image.
[0015] Furthermore, clustering is performed according to the initial features of each image in the error-prone subset to obtain a number of error-prone clusters, specifically: the initial features of each image in the error-prone subset are regarded as clusters, and the distance between each cluster is calculated; the clusters that meet the merging conditions are merged until the number of clusters is less than or equal to the preset number of clusters.
[0016] Furthermore, the distance between the clusters is expressed as:
[0017] D=mind(V A ,V B )
[0018] Where D is the distance between clusters; min is the minimum function; d is the distance calculation function; V A is the initial feature vector of the image in cluster A; V B is the initial feature vector of the image in cluster B.
[0019] Further, the texture enhancement classifier includes a texture feature extraction unit, a deep feature extraction unit, a cross-attention feature fusion unit and a sub-classification layer;
[0020] The texture feature extraction unit is used to extract subtle visual features of the error-prone image in the error-prone cluster using local binary patterns and gray-level co-occurrence matrices, respectively, to obtain local texture features and local intensity distribution features of adjacent pixel pairs, and to fuse the local texture features and the local intensity distribution features of adjacent pixel pairs using a pre-trained ResNet18 network to obtain final texture features;
[0021] The deep feature extraction unit is used to extract the semantic deep features of the error-prone images in the error-prone cluster class using the pre-trained backbone network;
[0022] The cross-attention feature fusion unit is used to fuse the final texture feature and the semantic deep feature based on the cross-attention mechanism to obtain a fused feature;
[0023] The sub-classification layer is used to obtain the classification result of the error-prone image according to the fusion features by using the global average pooling operation and the classification head.
[0024] Furthermore, the expression of the final texture feature is:
[0025] F T '=Resnet18([F LBP ,F Con ,F Cor ,F ASM ,F Dis ,F Hom ,F Ent ])
[0026] F S =S(F GLCM ),S∈{Con,Cor,ASM,Dis,Hom,Ent}
[0027] F GLCM =GLCM(x img )
[0028] F LBP =LBP(x img )
[0029] Among them, F T ' is the final texture feature; Resnet18 is the pre-trained ResNet18 network; F LBP is the local texture feature; F Con is the contrast feature; F Cor is the correlation feature; F ASM is the angular second-order moment characteristic; F Dis is a differential feature; F Hom F is a homogeneous characteristic; Ent is the entropy characteristic; F S is the total feature map; S is the feature selection operation; F GLCM is the local intensity distribution feature of adjacent pixel pairs; Con is the contrast mark; Cor is the correlation mark; ASM is the angular second moment mark; Dis is the difference mark; Hom is the homogeneity mark; Ent is the entropy mark; GLCM is the gray level co-occurrence matrix; LBP is the local binary pattern; x img For error-prone images.
[0030] Furthermore, the expression of the semantic deep feature is:
[0031] F′ D=Backbone(x img )
[0032] Among them, F′ D is the deep semantic feature; Backbone is the backbone network; x img For error-prone images.
[0033] Furthermore, the expression of the fusion feature is:
[0034] F fusion (F D ,F T )=FC(concat(CA1,CA2))+F D
[0035]
[0036] F T =LN(FL(F T '))
[0037] F D =LN(FL(F D ' D ))
[0038] Among them, F fusion is the fusion feature function; F D is the semantic deep feature after flattening and linear normalization; F T is the final texture feature after flattening and linear normalization; FC is a fully connected layer with a channel dimension of D; concat is a concatenation operation; CA1 and CA2 are both cross-attention components; Attention is a cross-attention mechanism; Q T F T The query matrix K D F D The bond matrix of V D F D The value matrix; softmax is the activation function; D is the number of channels of the fully connected layer FC; Q D F D The query matrix K T F T The bond matrix of V T F T The value matrix of Q D The weight matrix of K D The weight matrix of V D The weight matrix of Q T The weight matrix of KT The weight matrix of V T The weight matrix; LN is the linear normalization operation; FL is the expansion operation; F′ D is the deep semantic feature; F′ T is the final texture feature.
[0039] The beneficial effects of the present invention are as follows: first, a tea classification data set is used to train a convolutional neural network of a basic classification model, and the data set is classified for the first time on this basis, which can reduce the complexity of the overall operation of the model and, at the same time, speed up the detection process for conventional images; a subset clustering module is used to perform a second classification on the error-prone subsets, which reduces the difficulty of subsequent classification and improves the classification accuracy; and a texture enhancement classifier is used to adaptively extract textures of various types of error-prone subset clusters, which can improve the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 The figure is a flow chart of the method of the present invention.
[0041] Figure 2 This is a model structure diagram of the present invention. DETAILED DESCRIPTION
[0042] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0043] like Figure 1 As shown, in one embodiment of the present invention, a method for recognizing and classifying tea non-standardized sampling images based on deep learning includes:
[0044] Collect non-standardized images of different types of dry tea and build a tea classification dataset;
[0045] Divide the tea classification dataset into error-prone subsets and regular subsets;
[0046] Construct a tea classification model, and use the error-prone subset and the regular subset to train the tea classification model to obtain a trained tea classification model;
[0047] The tea image to be identified is obtained, and the tea type is obtained using the trained tea classification model. In this embodiment, a large number of images are collected to establish a tea classification data set, and the types of tea that can be used for classification and identification are shown in Table 1:
[0048] Table 1
[0049]
[0050]
[0051] In order to provide a comprehensive tea dataset, a tea classification dataset was constructed, which covers 35 tea categories in the six major tea categories in China. In order to solve the influence of the acquisition environment on the recognition results in previous image recognition, non-standard sampling images were obtained from multiple different network platforms, including online e-commerce websites, social media platforms, and tea industry websites.
[0052] After identifying low-quality images that do not meet the standards for resolution, focus, and visual clarity, a pre-trained ResNet50 model is used to systematically locate and exclude anomalous samples that do not match the target tea category.
[0053] The tea classification data set is divided into an error-prone subset and a regular subset, specifically: the tea classification data set is used to train a convolutional neural network, and after the kth iteration, the images that are incorrectly recognized during training are divided into the error-prone subset until a preset iteration number threshold is reached, and the images contained in the error-prone subset in the tea classification data set are eliminated to obtain the regular subset.
[0054] In this embodiment, only the misclassified images after k iterations are taken as error-prone images in order to avoid errors caused by too low classification accuracy of the convolutional neural network.
[0055] The tea classification model includes a feature extraction module, a subset clustering module, a plurality of texture enhancement classifiers, and a classification layer;
[0056] The feature extraction module is used to extract the image features of the error-prone subset and the regular subset respectively, and obtain the initial features of each image in the error-prone subset and the initial features of each image in the regular subset;
[0057] The subset clustering module is used to cluster the images according to the initial features of each image in the error-prone subset to obtain a number of error-prone clusters;
[0058] Each texture enhancement classifier is used to extract texture features and deep features from each error-prone cluster, and fuse the texture features and deep features to obtain fused features, and obtain the classification results of each image in the error-prone cluster based on the fused features;
[0059] The classification layer is used to perform classification according to the features of the conventional image to obtain the classification result of the conventional image.
[0060] In this example, this ensemble-based approach addresses the unique challenges of homogeneity and similarity in fine-grained tea classification, and achieves improved performance compared to single-stage models.
[0061] The method performs clustering according to the initial features of each image in the error-prone subset to obtain a number of error-prone clusters, specifically: the initial features of each image in the error-prone subset are regarded as clusters, and the distance between each cluster is calculated; the clusters that meet the merging conditions are merged until the number of clusters is less than or equal to the preset number of clusters.
[0062] The expression of the distance between the clusters is:
[0063] D=mind(V A ,V B )
[0064] Where D is the distance between clusters; min is the minimum function; d is the distance calculation function; V A is the initial feature vector of the image in cluster A; V B is the initial feature vector of the image in cluster B.
[0065] In this embodiment, subset clustering aims to divide the training data into multiple error-prone subsets and a regular subset, and generate a category distribution map between the subsets. This unsupervised clustering process utilizes the image similarity measured by the features embedded by the pre-trained backbone network.
[0066] like Figure 2 As shown, the texture enhancement classifier includes a texture feature extraction unit, a deep feature extraction unit, a cross-attention feature fusion unit and a sub-classification layer;
[0067] The texture feature extraction unit is used to extract subtle visual features of the error-prone image in the error-prone cluster using local binary patterns and gray-level co-occurrence matrices, respectively, to obtain local texture features and local intensity distribution features of adjacent pixel pairs, and to fuse the local texture features and the local intensity distribution features of adjacent pixel pairs using a pre-trained ResNet18 network to obtain final texture features;
[0068] The deep feature extraction unit is used to extract the semantic deep features of the error-prone images in the error-prone cluster class using the pre-trained backbone network;
[0069] The cross-attention feature fusion unit is used to fuse the final texture feature and the semantic deep feature based on the cross-attention mechanism to obtain a fused feature;
[0070] The sub-classification layer is used to obtain the classification result of the error-prone image according to the fusion features by using the global average pooling operation and the classification head.
[0071] The expression of the final texture feature is:
[0072] F′ T =Resnet18([FLBP ,F Con ,F Cor ,F ASM ,F Dis ,F Hom ,F Ent ])
[0073] F S =S(F GLCM ),S∈{Con,Cor,ASM,Dis,Hom,Ent}
[0074] F GLCM =GLCM(x img )
[0075] F LBP =LBP(x img )
[0076] Among them, F′ T is the final texture feature; Resnet18 is the pre-trained ResNet18 network; F LBP is the local texture feature; F Con is the contrast feature; F Cor is the correlation feature; F ASM is the angular second-order moment characteristic; F Dis is a differential feature; F Hom F is a homogeneous characteristic; Ent is the entropy characteristic; F S is the total feature map; S is the feature selection operation; F GLCM is the local intensity distribution feature of adjacent pixel pairs; Con is the contrast mark; Cor is the correlation mark; ASM is the angular second moment mark; Dis is the difference mark; Hom is the homogeneity mark; Ent is the entropy mark; GLCM is the gray level co-occurrence matrix; LBP is the local binary pattern; x img For error-prone images.
[0077] In this embodiment, local texture features are crucial when distinguishing similar categories in the error-prone subset because they can capture subtle visual features for distinction. Two complementary techniques are used to capture these features: local binary pattern (LBP) and gray-level co-occurrence matrix (GLCM). In order to adaptively fuse and optimize local texture features, the separate LBP and GLCM features are concatenated and a pre-trained ResNet18 network is used to obtain a more comprehensive feature representation.
[0078] The expression of the semantic deep feature is:
[0079] F′ D =Backbone(x img )
[0080] Among them, F′ D is the deep semantic feature; Backbone is the backbone network; x img For error-prone images.
[0081] In this embodiment, in addition to local texture, deep semantic features are also important for fine-grained classification in the error-prone subset. Considering the small number of categories that need to be classified in each subset, a pre-trained small backbone network is used to learn the semantics and global deep features of the error-prone images. This helps to balance the performance and efficiency of the overall network.
[0082] The expression of the fusion feature is:
[0083] F fusion (F D ,F T )=FC(concat(CA1,CA2))+F D
[0084]
[0085] F T =LN(FL(F T '))
[0086] F D =LN(FL(F′ D ))
[0087] Among them, F fusion is the fusion feature function; F D is the semantic deep feature after flattening and linear normalization; F T is the final texture feature after flattening and linear normalization; FC is a fully connected layer with a channel dimension of D; concat is a concatenation operation; CA1 and CA2 are both cross-attention components; Attention is a cross-attention mechanism; Q T F T The query matrix K D F D The bond matrix of V D F D The value matrix; softmax is the activation function; D is the number of channels of the fully connected layer FC; Q D F D The query matrix K T F T The bond matrix of V T F T The value matrix of Q D The weight matrix of K D The weight matrix of V D The weight matrix of Q T The weight matrix of K T The weight matrix of V T The weight matrix; LN is the linear normalization operation; FL is the expansion operation; F′ D is the deep semantic feature; F T ' is the final texture feature.
[0088] In this example, a cross-attention feature fusion unit is proposed to integrate texture and semantic features for accurate tea classification in the error-prone subset. The unit has two independent heads CA1 and CA2, which are used to learn the correlations between texture features and semantic features. These correlations are then used to optimize texture features and semantic features respectively, thereby capturing more discriminative visual features for fine-grained classification.
Claims
1. A method for recognizing and classifying tea non-standardized sampling images based on deep learning, characterized in that: include: Collect non-standardized images of different types of dry tea and build a tea classification dataset; Divide the tea classification dataset into error-prone subsets and regular subsets; Construct a tea classification model, and use the error-prone subset and the regular subset to train the tea classification model to obtain a trained tea classification model; Get the tea image to be identified and use the trained tea classification model to get the tea type.
2. The method for recognizing and classifying tea leaves based on deep learning according to claim 1 is characterized in that: The tea classification data set is divided into an error-prone subset and a regular subset, specifically: the tea classification data set is used to train a convolutional neural network, and after the kth iteration, the images that are incorrectly recognized during training are divided into the error-prone subset until a preset iteration number threshold is reached, and the images contained in the error-prone subset in the tea classification data set are eliminated to obtain the regular subset.
3. The method for recognizing and classifying tea leaves based on deep learning according to claim 1 is characterized in that: The tea classification model includes a feature extraction module, a subset clustering module, a plurality of texture enhancement classifiers, and a classification layer; The feature extraction module is used to extract the image features of the error-prone subset and the regular subset respectively, and obtain the initial features of each image in the error-prone subset and the initial features of each image in the regular subset; The subset clustering module is used to cluster the images according to the initial features of each image in the error-prone subset to obtain a number of error-prone clusters; Each texture enhancement classifier is used to extract texture features and deep features from each error-prone cluster, and fuse the texture features and deep features to obtain fused features, and obtain the classification results of each image in the error-prone cluster based on the fused features; The classification layer is used to perform classification according to the features of the conventional image to obtain the classification result of the conventional image.
4. The method for recognizing and classifying tea leaves based on deep learning according to claim 3 is characterized in that: The method performs clustering according to the initial features of each image in the error-prone subset to obtain a number of error-prone clusters, specifically: the initial features of each image in the error-prone subset are regarded as clusters, and the distance between each cluster is calculated; the clusters that meet the merging conditions are merged until the number of clusters is less than or equal to the preset number of clusters.
5. The method for recognizing and classifying tea leaves based on deep learning according to claim 4 is characterized in that: The expression of the distance between the clusters is: D=mind(V A ,V B ) Where D is the distance between clusters; min is the minimum function; d is the distance calculation function; V A is the initial feature vector of the image in cluster A; V B is the initial feature vector of the image in cluster B.
6. The method for recognizing and classifying tea leaves based on deep learning according to claim 4 is characterized in that: The texture enhancement classifier includes a texture feature extraction unit, a deep feature extraction unit, a cross-attention feature fusion unit and a sub-classification layer; The texture feature extraction unit is used to extract subtle visual features of the error-prone image in the error-prone cluster using local binary patterns and gray-level co-occurrence matrices, respectively, to obtain local texture features and local intensity distribution features of adjacent pixel pairs, and to fuse the local texture features and the local intensity distribution features of adjacent pixel pairs using a pre-trained ResNet18 network to obtain final texture features; The deep feature extraction unit is used to extract the semantic deep features of the error-prone images in the error-prone cluster class using the pre-trained backbone network; The cross-attention feature fusion unit is used to fuse the final texture feature and the semantic deep feature based on the cross-attention mechanism to obtain a fused feature; The sub-classification layer is used to obtain the classification result of the error-prone image according to the fusion features by using the global average pooling operation and the classification head.
7. The method for recognizing and classifying tea leaves based on deep learning according to claim 6 is characterized in that: The expression of the final texture feature is: In T =Resnet18([F LBP ,F Con ,F Cor ,F ASM ,F Dis ,F Hom ,F Ent ]) F S =S(F GLCM ),S∈{Con,Cor,ASM,Dis,Hom,Ent} F GLCM =GLCM(x img ) F LBP =LBP(x img ) Among them, F' T is the final texture feature; Resnet18 is the pre-trained ResNet18 network; F LBP is the local texture feature; F Con is the contrast feature; F Cor is the correlation feature; F ASM is the angular second-order moment characteristic; F Dis is a differential feature; F Hom F is a homogeneous characteristic; Ent is the entropy characteristic; F S is the total feature map; S is the feature selection operation; F GLCM is the local intensity distribution feature of adjacent pixel pairs; Con is the contrast mark; Cor is the correlation mark; ASM is the angular second moment mark; Dis is the difference mark; Hom is the homogeneity mark; Ent is the entropy mark; GLCM is the gray level co-occurrence matrix; LBP is the local binary pattern; x img For error-prone images.
8. The method for recognizing and classifying tea leaves based on deep learning according to claim 6, characterized in that: The expression of the semantic deep feature is: F' D =Backbone(x img ) Among them, F' D is the deep semantic feature; Backbone is the backbone network; x img For error-prone images.
9. The method for recognizing and classifying tea leaves based on deep learning according to claim 6, characterized in that: The expression of the fusion feature is: F fusion (F D ,F T )=FC(concat(CA1,CA2))+F D F T =LN(FL(F' T )) F D =LN(FL(F' D )) Among them, F fusion is the fusion feature function; F D is the semantic deep feature after flattening and linear normalization; F T is the final texture feature after flattening and linear normalization; FC is a fully connected layer with a channel dimension of D; concat is a concatenation operation; CA1 and CA2 are both cross-attention components; Attention is a cross-attention mechanism; Q T F T The query matrix K D F D The bond matrix of V D F D The value matrix; softmax is the activation function; D is the number of channels of the fully connected layer FC; Q D F D The query matrix K T F T The bond matrix of V T F T The value matrix of Q D The weight matrix of K D The weight matrix of V D The weight matrix of Q T The weight matrix of K T The weight matrix of V T The weight matrix; LN is the linear normalization operation; FL is the expansion operation; F' D is the deep semantic feature; F' T is the final texture feature.