Tea multi-view quantitative evaluation method and device and storage medium

Abstract

The invention discloses a tea multi-view quantitative evaluation method and device and a storage medium. The method comprises the following steps: acquiring a dry tea view image, a leaf bottom view image and a soup color view image of to-be-evaluated tea; respectively extracting and fusing the dry tea visual angle image, the leaf bottom visual angle image and the soup color visual angle image by the trained tea sensory evaluation model to obtain fusion feature maps of multiple scales under different visual angles, and outputting the scores of the dry tea, the leaf bottom and the soup color of the tea based on the fusion feature maps; and splicing the fused feature maps of different visual angles of the same scale along the channel dimension to obtain a multi-scale visual angle fused feature map, and outputting the aroma fraction, the taste fraction and the tea identification result of the tea based on the multi-scale visual angle fused feature map and the fused feature map of each visual angle. According to the method, through multi-scale fusion and multi-view fusion, the overall quality of the tea leaves can be reflected more comprehensively and accurately, and modernization and standardization of the tea industry are effectively promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of tea quality assessment, and in particular to a multi-view quantitative evaluation method, device and storage medium for tea. Background Art

[0002] Tea sensory evaluation is a crucial link in the tea industry, which directly determines the market positioning of tea, consumer experience and market recognition. The traditional tea sensory evaluation method is based on the Chinese national standard GB / T 23776-2018. Professional tea tasters conduct sensory evaluation of tea from five main aspects: dry tea, soup color, aroma, taste, and leaf bottom based on their long-term accumulated sensory experience. However, the traditional evaluation method is highly dependent on the experience of the tea taster, which makes the evaluation results often have strong subjectivity and are easily affected by the tea taster's state, experience differences and environmental factors, resulting in instability and inconsistency in the evaluation results.

[0003] In order to improve the consistency and stability of tea evaluation results, instrumental analysis methods such as gas chromatography-mass spectrometry, high-performance liquid chromatography, electronic tongue, and electronic nose have been introduced into the tea evaluation process. The results of instrumental analysis can provide a certain scientific basis for tea quality. However, the high testing cost, long testing cycle, high experimental environment requirements, and complex testing process make it difficult for such methods to meet the rapidity and large-scale evaluation requirements of tea evaluation, and the application scenarios are limited.

[0004] With the development of artificial intelligence technology, algorithms based on machine learning and deep learning are gradually applied to tea quality management in order to reduce costs and improve efficiency. Patent CN118351369A discloses a tea intelligent identification and grading method based on machine vision. This grading method only focuses on the characteristics of dry tea, tea soup, and leaf bottom from the perspective, and does not consider the impact of aroma and taste on the overall quality of tea. Patent CN118797413A uses electronic eyes to extract the shape, color and texture characteristics of tea leaves, uses electronic tongues to collect flavor signals, uses electronic noses to collect aroma signals, and combines near-infrared spectrum information to comprehensively evaluate the grade of Gongfu black tea. This method is based on multiple instrument analysis methods. Not only is the detection time long, the cost is high, and the process is cumbersome, but the output evaluation results can only qualitatively divide the grade of black tea. It is difficult to quantitatively evaluate the complex characteristics and features of the same grade of tea in the five main dimensions of dry tea, soup color, aroma, taste, and leaf bottom, and the mutual influence between the dimensions cannot be well captured and reflected, resulting in poor accuracy of tea sensory evaluation. Summary of the invention

[0005] One object of the present invention is to provide a multi-perspective quantitative evaluation method for tea. The tea sensory evaluation model constructed by the present invention is based on the three visual feature information inputs of dry tea, soup color, and leaf bottom. Through multi-scale fusion processing and multi-perspective fusion processing, it can quantitatively capture the characteristics of tea in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom and identify the type of tea, comprehensively and accurately reflect the overall quality of tea, thereby effectively promoting the modernization and standardization of the tea industry.

[0006] The above-mentioned object of the present invention is achieved by the following technical solutions:

[0007] A multi-view quantitative evaluation method for tea, comprising the following steps:

[0008] Collect images of the tea leaves to be evaluated from the perspective of dry tea leaves, tea leaves bottom, and tea soup color;

[0009] The trained tea sensory evaluation model extracts and fuses the dry tea perspective image, the leaf bottom perspective image, and the tea soup color perspective image respectively to obtain multiple scales of dry tea fusion feature maps, leaf bottom fusion feature maps, and tea soup color fusion feature maps, and outputs the dry tea score, leaf bottom score, and tea soup color score of the tea based on the dry tea fusion feature map, leaf bottom fusion feature map, and tea soup color fusion feature map respectively;

[0010] The trained tea sensory evaluation model splices dry tea fusion feature maps, leaf bottom fusion feature maps and soup color fusion feature maps of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, and outputs the aroma score, taste score and tea identification result of the tea based on the perspective fusion feature maps of multiple scales and the dry tea fusion feature maps, leaf bottom fusion feature maps and soup color fusion feature maps;

[0011] The training method of the tea sensory evaluation model comprises the following steps:

[0012] Obtaining several kinds of tea for model training, labeling the tea based on the tea sensory evaluation method, and obtaining a tea review data set;

[0013] Collecting dry tea perspective images, leaf bottom perspective images, and tea soup color perspective images of several types of tea as a visual image dataset, and obtaining a training set based on the visual image dataset and the tea review dataset;

[0014] The tea sensory evaluation model is trained using the training set to obtain a trained tea sensory evaluation model.

[0015] In this technical solution, it is necessary to collect the dry tea perspective image, leaf bottom perspective image and tea soup color perspective image of the tea to be evaluated as the input of the tea sensory evaluation model. Among them, the dry tea perspective image is an image obtained by directly collecting the dry tea sample. After the dry tea sample is soaked, the tea soup is poured out to collect the tea soup color perspective image, and the remaining leaf bottom is used to collect the leaf bottom perspective image. In some embodiments, the three perspective images are all taken by a camera under consistent lighting conditions.

[0016] In this technical solution, the trained tea sensory evaluation model is used to process the dry tea perspective image, the leaf bottom perspective image and the tea soup color perspective image. Specifically, the visual backbone network is used to extract the dry tea feature maps of different scales from the dry tea perspective image, and then the multi-scale feature fusion of the dry tea feature maps of different scales is performed to obtain the dry tea fusion feature maps of multiple scales, and finally the score of the dry tea is output based on the dry tea fusion feature maps of multiple scales.

[0017] Similarly, the visual backbone network is used to extract leaf bottom feature maps of different scales from the leaf bottom perspective image, and then multi-scale feature fusion is performed on the leaf bottom feature maps of different scales to obtain leaf bottom fusion feature maps of multiple scales, and finally the leaf bottom score is output based on the leaf bottom fusion feature maps of multiple scales. The visual backbone network is used to extract soup color feature maps of different scales from the soup color perspective image, and then multi-scale feature fusion is performed on the soup color feature maps of different scales to obtain soup color fusion feature maps of multiple scales, and finally the soup color score is output based on the soup color fusion feature maps of multiple scales.

[0018] In this technical solution, by utilizing multi-scale feature fusion, the visual information from three perspectives can be more effectively used to capture the global and local detail features of tea leaves.

[0019] In one or more embodiments, ResNet101 is used as a visual backbone network in a CLIP (Contrastive Language-Image Pre-training) pre-training model.

[0020] In the present technical solution, the trained tea sensory evaluation model splices the fusion feature maps of the three perspectives of the same scale together along the channel dimension to obtain perspective fusion feature maps of multiple scales. In some embodiments, the perspective fusion feature maps of different scales generate the weights of the pixel-by-pixel weighted fusion of the perspective fusion feature maps of each scale after pooling. After normalization along the channel dimension, fine-grained weighted aggregation is performed using dynamic weights to obtain a multi-scale fusion perspective fusion feature map. Subsequently, the perspective fusion feature map is spliced ​​together with the features of the dry tea fusion feature map, the leaf bottom fusion feature map, and the soup color fusion feature map to output the aroma score, taste score, and tea identification results of the tea. By splicing fusion feature maps of different perspectives, the model's ability to represent the multi-dimensional quality characteristics of tea can be effectively enhanced. Taking tea as a whole, the aroma and taste of tea are predicted based on the features of dry tea, leaf bottom, and soup color, and the type of tea is comprehensively judged in combination with the five main dimensions.

[0021] In this technical solution, when training the tea sensory evaluation model, the training set mainly includes two parts: tea evaluation data set and visual image data set. Among them, the tea evaluation data set is the expert's scoring of five dimensions of dry tea, soup color, aroma, taste, and leaf bottom based on the Chinese national standard GB / T 23776-2018, and the calculation formula for the overall score of each tea is:

[0022] Overall score = 0.25×dry tea + 0.1×tea soup color + 0.25×aroma + 0.3×flavor + 0.1×leaf bottom

[0023] The collection method of the dry tea perspective images, leaf bottom perspective images and tea color perspective images in the visual image dataset is the same as the collection method of the tea to be reviewed. In some embodiments, the three perspective images of other types of tea are also collected in the same way as the test set for model training.

[0024] In this technical solution, during the model training process, the visual image dataset is used as the input of the model. Through the loss function and adjustment of the model parameters, the tea score output by the model is evaluated using the expert review dataset, and the various performance indicators of the model are monitored to finally obtain the trained tea sensory evaluation model.

[0025] In this technical solution, by multi-scale fusion of feature information input from three perspectives of dry tea, soup color, and leaf bottom, the visual information from three perspectives can be effectively used to capture the global and local detail features of tea. At the same time, based on the feature fusion of three perspectives, the model's ability to represent the multi-dimensional quality features of tea can be enhanced. Therefore, only the visual features from three perspectives of dry tea, soup color, and leaf bottom need to be input to quantitatively capture the features of tea in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom and identify the type of tea, reflecting the overall quality of tea more comprehensively and accurately. Not only that, it can also greatly shorten the detection time, reduce the cost of detection, simplify the detection process, and facilitate the efficient evaluation of large-scale tea quality and promote the modernization and standardization of the tea industry.

[0026] Furthermore, traditional visual models usually abstract high-level semantic features by gradually downsampling feature maps as the network depth increases. Although this method effectively abstracts global semantic information, it often leads to the loss of local detail information, especially for tea images where local detail information only occupies a small number of pixels. Therefore, in order to solve this problem, multi-scale feature maps need to be used to enrich the semantic representation so that the model can focus on both global and local details.

[0027] As a preferred implementation of multi-scale fusion in the present invention, a bidirectional feature pyramid network is used to fuse dry tea feature maps of different scales extracted from the dry tea perspective image, leaf bottom feature maps of different scales extracted from the leaf bottom perspective image, and soup color feature maps of different scales extracted from the soup color perspective image, and in the top-down fusion process, a convolutional block attention module is used to adjust the fused feature maps.

[0028] In this technical solution, a bidirectional feature pyramid network is used to perform multi-scale feature map fusion, thereby enhancing the semantic representation of different scales. Specifically, for each feature map level downsampled in the same perspective, such as the dry tea perspective image, bottom-up fusion is achieved by upsampling the feature map of the adjacent lower layer to a size that matches the feature map of the current layer, and then summing these feature maps using learnable normalized weights, followed by deep fusion through a 3×3 depthwise separable convolution and a 1×1 convolution; in contrast, top-down fusion involves pooling and downsampling the feature map of the adjacent upper layer to match the size of the current layer, followed by operations similar to the bottom-up fusion process, and its calculation formula is

[0029]

[0030] In the formula, and They represent the bottom-up and top-down fused feature maps of the i-th layer, UP and DOWN represent upsampling and pooling downsampling operations to ensure the consistency of the size of the feature map, PC 1×1 Represents a point-by-point convolution with a convolution kernel size of 1×1, DC 3×3 represents a depth-wise separable convolution with a kernel size of 3×3 and parameters β1, β2, β3, β j is a learnable weight used to adjust the contribution of the fused feature map, φ is the ReLU activation function, which ensures that the weight remains non-negative, ∈ is a small constant included to prevent division by zero during weight normalization, and σ represents the swish activation function, which is usually used in neural networks to enhance the nonlinear representation ability of the network.

[0031] In this technical solution, a convolutional block attention module (CBAM) is also introduced in the top-down fusion process. The convolutional block attention module can apply channel and spatial attention in sequence to capture the attention distribution in the global feature map. By using the convolutional block attention module in the top-down fusion process to adjust the fused feature map, a more refined semantic feature representation can be generated.

[0032] In some preferred embodiments, the calculation formula of the convolution block attention module is:

[0033]

[0034] In the formula, Represents the top-down fusion feature map of the i-th layer, A c is the channel attention, A s is spatial attention, A is the global attention that combines channel attention and spatial attention, and f (5) represents a one-dimensional convolution with a convolution kernel length of 5, which is used to generate channel attention weights, f (7×7) Represents a two-dimensional convolution with a convolution kernel size of 7×7, which is used to generate spatial attention weights, AvgPool c Represents global average pooling along the height and width directions, used to capture the global information of each channel, AvgPool s Represents average pooling along the channel dimension, MaxPool s represents the maximum pooling along the channel dimension, σ s is a sigmoid activation function, which is used to compress the global attention weight obtained by element-by-element multiplication of the channel and spatial attention weights into a gated value between 0 and 1. The symbol ⊕ represents a concatenation operation, that is, combining the feature maps obtained by two different pooling modes along the channel dimension, and the symbol ⊙ represents element-by-element multiplication.

[0035] In this technical solution, by improving the convolutional block attention module, not only the number of parameters is reduced and the parallelism of calculation is improved, thereby improving the efficiency of the model, but also the output amount of information in the global feature map is adjusted, the correlation of image features in channels and spatial dimensions is enhanced, and the fine-grained semantic feature representation is enriched, thereby improving the generalization performance of the network.

[0036] As a preferred implementation of the tea sensory evaluation model in the present invention, a priori review elements and cross-modal attention mechanisms are introduced to further enhance the model's ability to understand the sensory attributes of tea, thereby further exploring the flavor characteristics of tea.

[0037] Specifically, the steps include:

[0038] Obtaining the review elements of dry tea, leaf bottom, and tea soup color in the tea sensory evaluation method, and encoding the review elements to obtain a first semantic vector;

[0039] The following processing is performed on the dry tea fusion feature map, the leaf bottom fusion feature map, and the tea color fusion feature map of the multiple scales:

[0040] The larger scale fusion feature map of each perspective is used as the key and value, and the first semantic vector is used as the query vector. Cross-modal attention calculation is performed to obtain the prior visual semantic features of each perspective.

[0041] The smaller-scale fused feature maps of each perspective are pooled to obtain the second semantic vector of each perspective;

[0042] The prior visual semantic features and the corresponding second semantic vectors of each perspective are spliced ​​respectively, and after passing through a linear mapping layer, the dry tea score, leaf bottom score, and tea soup color score of the tea are output.

[0043] In this technical solution, the review elements of three perspectives are obtained based on the sensory evaluation method of tea. Among them, the review elements of dry tea include shape, tenderness, color, wholeness and purity, the review elements of leaf bottom include tenderness, color, brightness and uniformity, and the review elements of soup color include color type and chromaticity, brightness and clarity. After obtaining the review elements of three perspectives, the review elements are encoded to obtain the first semantic vector, which is used as the query vector in the cross-modal attention mechanism.

[0044] In this technical solution, multiple scales of dry tea fusion feature maps, multiple scales of leaf bottom fusion feature maps, and multiple scales of soup color fusion feature maps are obtained through attention-enhanced multi-scale fusion processing. For the larger-scale fusion feature maps in the fusion feature maps of each perspective, they are used as keys and values, combined with the corresponding first semantic vectors for cross-modal attention calculation, to obtain the prior visual semantic features of the current perspective. The larger-scale fusion feature maps have more detailed information. Through cross-modal attention calculation, they can better imitate the expert's review logic and understand the visual features from the dimensions of multiple review elements corresponding to the current perspective. At the same time, for the smaller-scale fusion feature maps in the fusion feature maps of each perspective, they have more global information, and the second semantic vector is obtained after pooling. Finally, the prior visual semantic features and the second semantic features of each perspective are spliced, and after a linear mapping layer, the quality identification score of the corresponding perspective is output.

[0045] In this technical solution, by taking the review elements corresponding to each perspective as query vectors and combining them with a larger-scale fusion feature map for cross-modal attention calculation, each perspective can be given a visual feature representation corresponding to the prior semantic enhancement provided by multiple review elements, so that the model can analyze the visual features of tea from the perspective of review elements of each perspective in a way that is closer to expert review, effectively improving the model's ability to understand the visual attributes of tea.

[0046] Furthermore, for the review elements of the same perspective, the following processing is performed:

[0047] Using the prior visual semantic features as node information;

[0048] Perform self-attention calculation on the first semantic vector to obtain a weighted adjacency matrix;

[0049] A graph attention network is used to learn the associations between the review elements.

[0050] In this technical solution, a Graph Attention Network (GAT) is used to learn the correlation between each review element under the same perspective, so as to further enhance the model's ability to understand the visual attributes of tea. Specifically, taking the dry tea perspective as an example, the prior visual semantic features corresponding to the dry tea perspective are used as node information, and self-attention calculations are performed on the five review elements of the encoded shape, tenderness, color, wholeness and clarity, so as to generate a weighted adjacency matrix representing the correlation between the review elements. Subsequently, the graph attention network is used to learn the relationship between the semantic information of each review element to further enhance the prior visual semantic features.

[0051] Furthermore, the method further comprises the following steps:

[0052] Pooling the perspective fusion feature maps of the multiple scales to obtain a third semantic vector;

[0053] The third semantic vector is concatenated with the prior visual semantic features of the dry tea, the tea leaves, and the tea soup color to obtain a global visual semantic feature;

[0054] The aroma score, flavor score, and tea type identification result of the tea are output based on the global visual semantic features.

[0055] In this technical solution, after obtaining the perspective fusion feature maps of multiple scales, pooling is performed, such as spatial pyramid pooling and average pooling to obtain the third semantic vector, and then the third semantic vector and the prior visual semantic features of the dry tea perspective, the prior visual semantic features of the leaf bottom perspective, and the prior visual semantic features of the soup color perspective are spliced ​​together to obtain a complete global perspective semantic feature, and finally the aroma score and taste score are output through the linear mapping layer, and the tea identification result is output through the linear mapping layer and the softmax layer.

[0056] Furthermore, the calculation formula of the perspective fusion feature map of multiple scales is:

[0057]

[0058] Where W i Represents the dynamic weight of fine-grained aggregation, with dimensions h and w are the height and width of the feature map, n is the number of viewing angles, and F in represents the feature map of the i-th scale and the n-th perspective, f (1×1) represents a two-dimensional convolution with a convolution kernel size of 1×1, W i (j) represents the weight of fine-grained aggregation of feature maps at the i-th scale and j-th perspective, which is composed of W i Sliced ​​along the channel dimension. F i is the feature map of the i-th scale after fusion of multiple perspectives. ⊕ and ⊙ represent the concatenation operation and element-wise product operation respectively.

[0059] In this technical solution, a fine-grained dynamic fusion mechanism is proposed, which splices the feature maps of the same scale from different perspectives together along the channel dimension, and then generates the weights of the pixel-by-pixel weighted fusion of the multi-perspective feature maps through two layers of convolution with a kernel size of 1×1, and normalizes them along the channel dimension using the softmax function. Finally, these dynamic weights are used for fine-grained weighted aggregation to obtain multi-scale feature maps after multi-perspective fusion.

[0060] In this technical solution, through the above-mentioned dynamic fusion mechanism, multi-view features at the pixel level can be dynamically weighted and aggregated, which can effectively enhance the multi-granularity and multi-semantics of visual features and improve generalization performance.

[0061] As a preferred implementation of the loss function in the present invention, a loss function is introduced in the process of training the tea sensory evaluation model. The loss function L total for:

[0062]

[0063] In this technical solution, considering the need to predict the tea type and the scores of five dimensions at the same time, and the uneven distribution of quality evaluation score data, a multi-task attention network (MTAN) is used to perform multi-task balance. At the same time, a multi-task balanced focal regression loss function is proposed to solve the problem of category imbalance in classification tasks.

[0064] In the formula, l cls is the loss function for tea classification, which is the standard cross entropy loss function. aux and l i The regression loss function used to predict the score is calculated in the same way, where l aux It is a regression loss function of the weighted total score, which is used to provide the network model with information on the different importance of each scoring dimension to assist the optimization process; i is the regression loss function of the i-th scoring dimension. α, β, and γ are the hyperparameters that balance the three loss functions and their sum is 1. sl is the smooth L1 loss, where ρ is the hyperparameter for adjusting the shape of the function. θ is the hyperparameter for adjusting the shape of the exponential distribution function. i is the true value of the label, Y is the difference between the predicted value and the true value of the label, and μ is the factor for adjusting the focusing strength.

[0065] With this setting, when the regression loss value of the sample is closer to 0, the weight will become smaller, and vice versa. This directly amplifies the difference between the two, allowing the network model to adaptively mine difficult samples during the optimization process.

[0066] In order to balance multiple tasks, a dynamic weight λ is assigned to the loss value of each regression task at the time step during training. i , w in the equation i (·) represents the rate of decrease of the loss of the i-th task, and t is the iteration index. In the experiment, the rate of decrease of the task loss is expressed by the loss value of the next iteration compared to the loss value of the previous iteration. In the formula, T is the temperature coefficient, which is used to control the softness of task weighting. A larger T will lead to a more uniform distribution among different tasks. Finally, after softmax normalization, it is multiplied by K to ensure that ∑kλi(t)=K, where K is the number of tasks.

[0067] With this setting, if the loss of a task decreases faster, the weight value assigned to the task will become smaller, thereby slowing down its learning optimization speed. Conversely, if the loss decreases slower, the learning optimization speed will be faster. This dynamic adjustment of the learning weights of each task during the optimization process is intended to achieve balanced optimization of each task, preventing some tasks from being over-optimized and others from being under-optimized.

[0068] Another object of the present invention is to provide a multi-perspective quantitative evaluation device for tea, which processes the input images of dry tea, tea soup color, and leaf bottom based on a trained tea sensory evaluation model, outputs scores of the tea in five dimensions of dry tea, tea soup color, aroma, taste, and leaf bottom, and identifies the type of tea.

[0069] The above-mentioned object of the present invention is achieved by the following technical solutions:

[0070] A multi-view quantitative evaluation device for tea, comprising:

[0071] A collection unit, used to collect images from the perspective of dry tea, images from the perspective of leaf bottom, and images from the perspective of tea soup color of the tea to be reviewed;

[0072] A tea sensory evaluation unit, comprising a trained tea sensory evaluation model, wherein the trained tea sensory evaluation model is used to output a dry tea score, a leaf bottom score, a soup color score, an aroma score, a taste score, and a tea type identification result based on the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image;

[0073] Wherein, the trained tea sensory evaluation model includes:

[0074] A feature extraction module is used to extract dry tea feature maps of multiple scales from the dry tea perspective image, extract leaf bottom feature maps of multiple scales from the leaf bottom perspective image, and extract soup color feature maps of multiple scales from the soup color perspective image;

[0075] A feature fusion module is used to fuse dry tea feature maps of multiple scales to obtain dry tea fusion feature maps of multiple scales, fuse leaf bottom feature maps of multiple scales to obtain leaf bottom fusion feature maps of multiple scales, and fuse soup color feature maps to obtain soup color fusion feature maps of multiple scales;

[0076] The semantic enhancement module is used to use the larger-scale dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map as the key and value of the corresponding perspective, encode the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method, and obtain the first semantic vector as the query vector of the corresponding perspective, perform cross-modal attention calculation to obtain the prior visual semantic features of each perspective; obtain the second semantic vector of each perspective after pooling the smaller-scale dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map; and respectively splice the prior visual semantic features and the corresponding second semantic vector of each perspective, and output the dry tea score, leaf bottom score, and soup color score of the tea;

[0077] The perspective fusion module is used to splice the dry tea fusion feature map, leaf bottom fusion feature map and soup color fusion feature map of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, pool the perspective fusion feature maps of multiple scales to obtain a third semantic vector, splice the third semantic vector and the prior visual semantic features of each perspective to obtain a global visual semantic feature, and output the aroma score, taste score and tea identification result of the tea based on the global visual semantic feature.

[0078] The present invention also provides a storage medium, comprising a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute any of the aforementioned multi-view quantitative evaluation methods for tea.

[0079] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0080] 1. The present invention can effectively use the visual information of the three perspectives to capture the global and local detail features of tea leaves by multi-scale fusion of feature information input from the three perspectives of dry tea, soup color, and leaf bottom. At the same time, based on the feature fusion of the three perspectives, the model's ability to represent the multi-dimensional quality features of tea leaves can be enhanced. Therefore, only the visual features of the three perspectives of dry tea, soup color, and leaf bottom need to be input to quantitatively capture the features of tea leaves in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom and identify the type of tea leaves, thereby more comprehensively and accurately reflecting the overall quality of tea leaves.

[0081] 2. The present invention adopts a bidirectional feature pyramid network to fuse feature maps of different scales under different perspectives, and introduces a convolutional block attention module in the top-down fusion process to adjust the fused feature maps, thereby generating a more refined semantic feature representation, while enabling the model to focus on both global and local details at the same time;

[0082] 3. The present invention uses the review elements corresponding to each perspective as query vectors and combines them with a larger scale fusion feature map to perform cross-modal attention calculation, which can give each perspective a visual feature representation corresponding to the prior semantic enhancement provided by multiple review elements, so that the model can analyze the visual features of tea from the perspective of review elements of each perspective in a way that is closer to expert review, effectively improving the model's ability to understand the visual attributes of tea;

[0083] 4. The present invention uses the graph attention network to learn the correlation between the review elements under the same perspective, which can further enhance the model's ability to understand the perception attributes of tea leaves;

[0084] 5. The present invention can dynamically weight and aggregate multi-view features at the pixel level through a dynamic fusion mechanism, which can effectively enhance the multi-granularity and multi-semantics of visual features and improve generalization performance;

[0085] 6. The multi-task balanced focus regression loss function designed by the present invention enables the network model to adaptively mine difficult samples during the optimization process, and the dynamic adjustment of the learning weights of each task during the optimization process is intended to achieve balanced optimization of each task, preventing some tasks from being over-optimized and other tasks from being under-optimized;

[0086] 7. The present invention can greatly shorten the detection time, reduce the detection cost, simplify the detection process, and is conducive to the efficient evaluation of large-scale tea quality and promote the modernization and standardization of the tea industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0088] Figure 1 It is a flowchart of a multi-view quantitative evaluation method in a specific embodiment of the present invention;

[0089] Figure 2 It is a schematic diagram of the model framework of the trained tea sensory evaluation model in a specific embodiment of the present invention;

[0090] Figure 3 The information of tea samples in a specific embodiment of the present invention and the scores and images of representative tea samples in the training set are shown, wherein: (a) the number of black tea, yellow tea and green tea samples; (b) the sensory scores of the expert panel on five dimensions of the representative tea samples; (c) representative images of the tea samples;

[0091] Figure 4 The sensory score distribution of all tea samples in a specific embodiment of the present invention is shown, including five dimensions and a total score;

[0092] Figure 5 The figure shows the comparison of the accuracy of flavor factor scoring of Tea MvMtNet in a specific embodiment of the present invention and two other representative models;

[0093] Figure 6 The figure shows the comparison between the predicted value and the true value of the evaluation score of three representative tea samples by Tea MvMtNet in a specific embodiment of the present invention;

[0094] Figure 7 The figure shows a comparison of the classification accuracy of Tea MvMtNet and the other two models in tea type recognition evaluated by confusion matrix in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0095] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0096] In the description of the present invention, it should be understood that the terms "front", "rear", "left", "right", "up", "down", "vertical", "horizontal", "high", "low", "inside", "outside", etc., indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0097] Embodiment 1:

[0098] like Figure 1 and Figure 2 A multi-view quantitative evaluation method for tea leaves is shown, comprising the following steps:

[0099] Collect images of the tea leaves to be evaluated from the perspective of dry tea leaves, tea leaves bottom, and tea soup color;

[0100] The trained tea sensory evaluation model extracts and fuses the dry tea perspective image, the leaf bottom perspective image, and the tea soup color perspective image respectively to obtain multiple scales of dry tea fusion feature maps, leaf bottom fusion feature maps, and tea soup color fusion feature maps, and outputs the dry tea score, leaf bottom score, and tea soup color score of the tea based on the dry tea fusion feature map, leaf bottom fusion feature map, and tea soup color fusion feature map respectively;

[0101] The trained tea sensory evaluation model splices dry tea fusion feature maps, leaf bottom fusion feature maps and soup color fusion feature maps of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, and outputs the aroma score, taste score and tea identification result of the tea based on the perspective fusion feature maps of multiple scales and the dry tea fusion feature maps, leaf bottom fusion feature maps and soup color fusion feature maps;

[0102] The training method of the tea sensory evaluation model comprises the following steps:

[0103] Obtaining several kinds of tea for model training, labeling the tea based on the tea sensory evaluation method, and obtaining a tea review data set;

[0104] Collecting dry tea perspective images, leaf bottom perspective images, and tea soup color perspective images of several types of tea as a visual image dataset, and obtaining a training set based on the visual image dataset and the tea review dataset;

[0105] The tea sensory evaluation model is trained using the training set to obtain a trained tea sensory evaluation model.

[0106] In some embodiments, the three perspective images are scaled to the same size in equal proportion. For images with different heights and widths, gray borders can be used to fill and adjust them to the same size, and then the pixel values ​​are normalized for the visual backbone network to extract feature maps at different perspectives. In some preferred embodiments, the three perspective images are all adjusted to 512×512 pixels.

[0107] In one or more embodiments, high-level semantic features are extracted from the normalized dry tea perspective image, leaf bottom perspective image, and tea soup color perspective image by parameter sharing, and then the feature maps of each perspective are fused by parameter separation to obtain the corresponding fused feature map.

[0108] In one or more embodiments, the fused feature maps of the three perspectives are subjected to pyramid pooling and average pooling respectively to obtain semantic vectors, and then the corresponding scores are output through a linear mapping layer.

[0109] In one or more embodiments, ResNet101 is used as a visual backbone network in a CLIP (Contrastive Language-Image Pre-training) pre-training model.

[0110] In some preferred embodiments, the calculation formula of the perspective fusion feature map of multiple scales is:

[0111]

[0112] Where W iRepresents the dynamic weight of fine-grained aggregation, with dimensions h and w are the height and width of the feature map, n is the number of viewing angles, and F in represents the feature map of the i-th scale and the n-th perspective, f (1×1) represents a two-dimensional convolution with a convolution kernel size of 1×1, W i (j) represents the weight of fine-grained aggregation of feature maps at the i-th scale and j-th perspective, F i It is the feature map of the i-th scale after fusion of multiple perspectives.

[0113] Embodiment 2:

[0114] On the basis of Example 1, Figure 2 As shown, a bidirectional feature pyramid network is used to fuse dry tea feature maps of different scales extracted from the dry tea perspective image, leaf bottom feature maps of different scales extracted from the leaf bottom perspective image, and soup color feature maps of different scales extracted from the soup color perspective image, and in the top-down fusion process, a convolutional block attention module is used to adjust the fused feature maps.

[0115] In some preferred embodiments, for each level of feature maps downsampled in the image at each viewpoint, bottom-up fusion is achieved by upsampling the feature maps of the adjacent lower layer to a size that matches the feature maps of the current layer, and then summing these feature maps using learnable normalized weights, followed by deep fusion through a 3×3 depthwise separable convolution and a 1×1 convolution; conversely, top-down fusion involves pooling and downsampling the feature maps of the adjacent upper layer to match the size of the current layer, followed by operations similar to the bottom-up fusion process, and the calculation formula is

[0116]

[0117] In the formula, and They represent the bottom-up and top-down fused feature maps of the i-th layer, UP and DOWN represent upsampling and pooling downsampling operations to ensure the consistency of the size of the feature map, PC 1×1 Represents a point-by-point convolution with a convolution kernel size of 1×1, DC 3×3 represents a depth-wise separable convolution with a kernel size of 3×3 and parameters β1, β2, β3, β j is a learnable weight used to adjust the contribution of the fused feature map, φ is the ReLU activation function, which ensures that the weight remains non-negative, ∈ is a small constant included to prevent division by zero during weight normalization, and σ represents the swish activation function, which is usually used in neural networks to enhance the nonlinear representation ability of the network.

[0118] In a more preferred embodiment, a convolutional block attention module is used in the top-down fusion process to adjust the fused feature map, thereby generating a more refined semantic feature representation, wherein the calculation formula of the convolutional block attention module is:

[0119]

[0120] In the formula, Represents the top-down fusion feature map of the i-th layer, Ac is the channel attention, As is the spatial attention, A is the global attention combined with the channel attention and the spatial attention, f (5) represents a one-dimensional convolution with a convolution kernel length of 5, f (7×7) Represents a two-dimensional convolution with a convolution kernel size of 7×7, AvgPool c Represents global average pooling along the height and width directions, AvgPool s Represents average pooling along the channel dimension, MaxPool s represents the maximum pooling along the channel dimension, σ s is the sigmoid activation function.

[0121] Embodiment 3:

[0122] Based on the above embodiments, Figure 2 As shown, the evaluation method also includes the following steps:

[0123] Obtaining the review elements of dry tea, leaf bottom, and tea soup color in the tea sensory evaluation method, and encoding the review elements to obtain a first semantic vector;

[0124] The following processing is performed on the dry tea fusion feature map, the leaf bottom fusion feature map, and the tea color fusion feature map of the multiple scales:

[0125] The larger scale fusion feature map of each perspective is used as the key and value, and the first semantic vector is used as the query vector. Cross-modal attention calculation is performed to obtain the prior visual semantic features of each perspective.

[0126] The smaller-scale fused feature maps of each perspective are pooled to obtain the second semantic vector of each perspective;

[0127] The prior visual semantic features and the corresponding second semantic vectors of each perspective are spliced ​​respectively, and after passing through a linear mapping layer, the dry tea score, leaf bottom score, and tea soup color score of the tea are output.

[0128] In some embodiments, the review element is encoded using the Transformer language model in CLIP.

[0129] Taking the dry tea perspective as an example, in some embodiments, after obtaining the dry tea fusion feature map of five scales, the dry tea fusion feature map of the first four scales with larger scales is used as the key and value, and the five review elements of shape, tenderness, color, wholeness and purity are used as the query vector after encoding, and cross-modal attention is performed to obtain the prior visual semantic features from the dry tea perspective; the dry tea fusion feature map of the last layer with the smallest scale is subjected to spatial pyramid pooling and average pooling to obtain the second semantic vector from the dry tea perspective; finally, the prior visual semantic features and the second semantic vector from the dry tea perspective are spliced, and the score of the dry tea is output after a linear mapping layer. Similarly, in some embodiments, based on the leaf base fusion feature map of multiple scales, the soup color fusion feature map, and the review elements of the leaf base and soup color, cross-modal attention can be used to calculate and output the scores of the leaf base and soup color respectively.

[0130] In some preferred embodiments, the calculation formula for cross-modal attention processing is:

[0131]

[0132] In the formula, Q pe It is a sequence representation of multiple review elements in the current perspective. The language description of the review element is encoded using the language transformer in CLIP to obtain a dimension belonging to The query vector is , where L is the number of elements examined in the current perspective and D is the dimension of the encoded semantic vector. and Respectively represent the feature map of the i-th scale in the CLIP visual encoder as the key and value vectors, ∥·∥ and T represent the binary norm operation and transposition of the matrix respectively. The cosine similarity is used for the cross-modal attention operation, and the ReLU activation function is used for activation and then element-by-element square to obtain a more general attention representation. This method can directly set the negatively correlated attention weights to zero, avoiding the problem that the softmax activation will reset the negatively correlated weights to greater than zero, and achieve the purpose of adaptively filtering irrelevant information. Then, the results of cross-modal attention at all scales are aggregated to obtain the prior visual semantic feature H of the semantic enhancement of the censored element. sp .

[0133] In some embodiments, for review elements of the same perspective, the following processing is performed:

[0134] Using the prior visual semantic features as node information;

[0135] Perform self-attention calculation on the first semantic vector to obtain a weighted adjacency matrix;

[0136] A graph attention network is used to learn the associations between the review elements.

[0137] In this embodiment, a graph attention network (GAT) is used to learn the correlation between various review elements under the same perspective, so as to further enhance the model's ability to understand the visual attributes of tea.

[0138] In some preferred embodiments, the enhancement of the prior visual semantic features is calculated using the following formula:

[0139]

[0140] and Represents the query and key encoding vectors of the language description of the i-th and j-th review elements in the current perspective. By doing self-attention, we can get the weighted adjacency matrix, β ij is the weight value of position i,j in the adjacency matrix. ij Represents the normalized attention weights of the neighboring nodes at positions i and j in the graph attention network. Represents the prior visual semantic features of the semantic enhancement of the i-th prior review element in the current perspective, which is regarded as node information. is a shared linear transformation of each node, which converts the input features into high-level features in order to obtain sufficient expressive power. ⊕ represents a concatenation operation, which concatenates the feature vectors of two neighboring nodes and then passes them through a single-layer feedforward neural network. And apply the LeakyReLU nonlinear activation function to get the attention weight, and use softmax normalization. Finally, use α ij and β ij The product of is used as the aggregation weight of the neighbor node feature information, and the nonlinear activation function SiLU is used to activate the aggregated feature representation to obtain the final prior visual semantic features of the semantic enhancement of the prior review elements.

[0141] Embodiment 4:

[0142] Based on the above embodiments, Figure 2 As shown, the method further comprises the following steps:

[0143] Pooling the perspective fusion feature maps of the multiple scales to obtain a third semantic vector;

[0144] The third semantic vector is concatenated with the prior visual semantic features of the dry tea, the tea leaves, and the tea soup color to obtain a global visual semantic feature;

[0145] The aroma score, flavor score, and tea type identification result of the tea are output based on the global visual semantic features.

[0146] Embodiment 5:

[0147] Based on the above embodiments, Figure 2 As shown, a loss function is introduced in the process of training the tea sensory evaluation model. The loss function L total for:

[0148]

[0149] In the formula, l cls is the loss function for tea classification, l aux is the regression loss function of the weighted total score, l i is the regression loss function of the i-th scoring dimension, α, β, and γ are hyperparameters and their sum is 1, l sl is the smooth L1 loss, ρ is the hyperparameter for adjusting the shape of the function, θ is the hyperparameter for adjusting the shape of the exponential distribution function, and λ i is the dynamic weight, w i is the rate of decrease of the loss of the ith task, t is the iteration index, T is the temperature coefficient used to control the weighted softness of the task, K is the number of tasks, and y i is the true value of the label, Y is the difference between the predicted value and the true value of the label, and μ is the factor for adjusting the focusing strength.

[0150] In this embodiment, by setting the loss function, the closer the regression loss value of the sample is to 0, the smaller the weight will become, and vice versa, which directly amplifies the difference between the two, allowing the network model to adaptively mine difficult samples during the optimization process. At the same time, if the loss of a task decreases faster, the weight value assigned to the task will become smaller, thereby slowing down its learning optimization speed. On the contrary, if the loss decreases more slowly, the learning optimization speed will be accelerated. This dynamic adjustment of the learning weights of each task during the optimization process is intended to achieve balanced optimization of each task and prevent some tasks from being over-optimized and other tasks from being under-optimized.

[0151] Embodiment 6:

[0152] Based on the above embodiment, a multi-view quantitative evaluation device for tea leaves includes:

[0153] A collection unit, used to collect images from the perspective of dry tea, images from the perspective of leaf bottom, and images from the perspective of tea soup color of the tea to be reviewed;

[0154] A tea sensory evaluation unit, comprising a trained tea sensory evaluation model, wherein the trained tea sensory evaluation model is used to output a dry tea score, a leaf bottom score, a soup color score, an aroma score, a taste score, and a tea type identification result based on the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image;

[0155] Wherein, the trained tea sensory evaluation model includes:

[0156] A feature extraction module is used to extract dry tea feature maps of multiple scales from the dry tea perspective image, extract leaf bottom feature maps of multiple scales from the leaf bottom perspective image, and extract soup color feature maps of multiple scales from the soup color perspective image;

[0157] A feature fusion module is used to fuse dry tea feature maps of multiple scales to obtain dry tea fusion feature maps of multiple scales, fuse leaf bottom feature maps of multiple scales to obtain leaf bottom fusion feature maps of multiple scales, and fuse soup color feature maps to obtain soup color fusion feature maps of multiple scales;

[0158] The semantic enhancement module is used to use the larger-scale dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map as the key and value of the corresponding perspective, encode the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method, and obtain the first semantic vector as the query vector of the corresponding perspective, perform cross-modal attention calculation to obtain the prior visual semantic features of each perspective; obtain the second semantic vector of each perspective after pooling the smaller-scale dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map; and respectively splice the prior visual semantic features and the corresponding second semantic vector of each perspective, and output the dry tea score, leaf bottom score, and soup color score of the tea;

[0159] The perspective fusion module is used to splice the dry tea fusion feature map, leaf bottom fusion feature map and soup color fusion feature map of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, pool the perspective fusion feature maps of multiple scales to obtain a third semantic vector, splice the third semantic vector and the prior visual semantic features of each perspective to obtain a global visual semantic feature, and output the aroma score, taste score and tea identification result of the tea based on the global visual semantic feature.

[0160] In this embodiment, the constructed tea sensory evaluation model effectively integrates visual inputs from three perspectives, enhances the model's ability to learn and predict quality attributes based on visual data from multiple perspectives, achieves a more detailed and comprehensive evaluation, and the model is able to capture subtle visual cues related to sensory attributes such as aroma and taste, thereby improving the accuracy and reliability of tea quality management.

[0161] Embodiment 7:

[0162] Based on the above embodiments, a storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a multi-view quantitative evaluation method for tea shown in any of the above embodiments.

[0163] In this embodiment, the storage medium is a computer-readable storage medium. If the tea multi-view quantitative evaluation device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0164] Embodiment 8:

[0165] In this embodiment, the performance of the adopted tea sensory evaluation model Tea MvMtNet is tested.

[0166] 1) Data collection:

[0167] The tea samples were collected by the Key Laboratory of Refined Sichuan Tea of ​​Sichuan Agricultural University between April 2021 and July 2022. A total of 806 tea samples were collected from 18 tea-producing provinces (municipalities and autonomous regions) in China, including 346 black teas, 245 yellow teas and 215 green teas. Among them, 703 tea samples were used for model training, and the remaining 103 tea samples (including 27 green teas, 30 yellow teas and 46 black teas) were used for model performance testing. All tea samples were stored at 4°C. The geographical distribution and species distribution of tea samples are as follows: Figure 3 shown.

[0168] 2) Construct training set:

[0169] For the 703 tea samples used for model training, a panel of five experts from the key laboratory conducted sensory scoring to obtain a tea review dataset. Among them, the expert panel scored according to the Chinese national standard GB / T 23776-2018. The five dimensions of the scoring include dry tea, soup color, taste, aroma and leaf bottom. Each item is scored on a 10-point scale, and the score of each tea sample is the average of the expert scores. Specifically, the dry tea appearance of the dry tea samples is first scored. About 3 grams of tea samples are placed in a white porcelain cup, 150 ml of boiling water is added, the lid is covered, black tea and yellow tea are soaked for 5 minutes, and green tea is soaked for 4 minutes. After soaking, the tea soup is immediately poured into a white porcelain bowl to evaluate the soup color. The leaf bottom in the porcelain cup is used to evaluate the quality of the aroma and leaf bottom. Figure 3 Shown are the sensory scores assigned by experts to some representative tea samples.

[0170] Figure 4 The score distribution and probability density of all tea samples in five dimensions and total score are shown. The score range is 5 to 10 points, and the scores of most tea samples are concentrated around 9 points. The quality scores show a normal distribution, indicating that the tea samples in this study are representative and cover a wide range of tea types. This balanced representation enhances the generalization ability of the model at different quality levels.

[0171] A Canon EOS 80D (Canon, Japan) camera was used to collect images of dry tea, leaf bottom, and soup color of tea samples to obtain a visual image dataset. Specifically, about 5 grams of dry tea was randomly taken from each tea sample and evenly spread in a 7.5 cm × 7.5 cm sample pool. The tea soup was poured into a white porcelain bowl and photographed immediately. After soaking, the leaf bottom was transferred from the porcelain cup to a 7.5 × 7.5 cm sample pool for photography. All images were taken under consistent lighting conditions to ensure comparability. The final generated image is an uncompressed sRGB color image with a pixel size of 1984 × 1984. Figure 3 Perspective images of some representative tea samples are shown.

[0172] Finally, the visual image dataset and the tea review dataset together constitute the training set of the model.

[0173] 3) Training model:

[0174] ResNet101 is used as the visual backbone network in CLIP, and its pre-trained parameters are frozen throughout the training process. For preprocessing of training data, all images are scaled to 512×512 pixels. For images with different heights and widths, they are resized to 512×512 using gray border padding. Data augmentation is performed by random rotation of 90, 180, and 270 degrees, as well as vertical flipping and vertical flipping combined with rotation. In the experiments, the Adam optimizer is used, the training batch size is 7, and the network is trained for 500 cycles.

[0175] 4) Performance testing:

[0176] In this embodiment, 103 tea samples were tested, and the image inputs included images from the perspective of dry tea, images from the perspective of leaf bottom, and images from the perspective of tea soup. The performance of Tea MvMtNet was compared with seven common deep learning algorithms. In the test, all seven comparison models were pre-trained on the ImageNet dataset. For the prediction of the score for each perspective, global pooling is used to process the feature map of each perspective, and then the corresponding score is output through a separate multi-layer perceptron (MLP). For the prediction of aroma and taste scores and the identification of tea types, the feature vectors after global pooling of each perspective are spliced, and three independent MLPs are used to output the scores of aroma, taste, and tea types.

[0177] Table 1:

[0178]

[0179]

[0180] In Table 1, the MAE% (mean absolute error percentage) calculation formula is MAE% = (1 / n) ∑ | predicted value - true value | × 100%. The experimental results are shown in Table 1. Tea MvMtNet outperforms other models in all indicators, showing its advantage in integrating multi-view input in comprehensive tea quality assessment. It achieved an accuracy of 98.11% in tea type recognition, exceeding the best performing comparison model by 0.94%. For the quantitative scoring indicators of the five dimensions - dry tea, soup color, aroma, taste and leaf bottom - Tea MvMtNet's MAE% are 19.60%, 28.40%, 26.48%, 31.67% and 17.20%, respectively, which are 0.54%, 0.01%, 0.07%, 0.26% and 0.44% higher than the best performing comparison model, respectively. In addition, the model's MAE% on the weighted total score is 17.23%, which is 2.63% lower than the closest competitor. This equates to an error of 0.1723 on a 10-point total score, which is comparable to the typical error range of many senior assessors. These improvements in accuracy and MAE% across all dimensions demonstrate the effectiveness of the multi-view, multi-task approach adopted by Tea MvMtNet, indicating its ability to serve as a practical tool in tea quality management.

[0181] In order to clearly illustrate the superior performance in quantitative scoring, a visual comparison is made between the predicted values ​​and true values ​​of Tea MvMtNet and the other two models, as shown in the figure below: Figure 5As shown in the figures. In these figures, the diagonal lines represent perfect predictions, and the proximity of the data points to these lines indicates the accuracy of the model. The results show that Tea MvMtNet generally outperforms other models, especially in areas where the score distribution is sparse, highlighting its robustness and precision. For dry tea, soup color, aroma, taste, and leaf bottom, TeaMvMtNet shows consistently high accuracy. In particular, in the three dimensions with larger weights, dry tea, aroma, and taste, Tea MvMtNet shows a significant advantage. Superior performance in these key areas contributes to the effectiveness of the weighted total score.

[0182] In order to further illustrate the performance of Tea MvMtNet, one representative tea sample was selected from each of black tea, yellow tea, and green tea, representing the flavor-balanced type, soup color defect type, and taste defect type tea samples, respectively. Figure 6 Radar charts of the evaluation scores of three representative samples are shown. In these figures, the blue areas represent predicted values ​​and the red areas represent true values. The degree of overlap of these areas reflects the accuracy of the model. The high overlap observed in all five dimensions indicates that Tea MvMtNet has the ability to predict multiple quality attributes with high accuracy. Specifically, the representative black tea performs more balanced in the five flavor factors, while the representative yellow tea performs well in dry tea and taste. In contrast, the representative green tea shows some weaknesses in taste. Although these tea samples have different flavor characteristics, Tea MvMtNet can accurately predict them.

[0183] Furthermore, in addition to the quantitative quality assessment, the performance of Tea MvMtNet in tea classification was evaluated using the confusion matrix. Figure 6 As shown in the table, these matrices compare the classification accuracy of Tea MvMtNet with the other two models. All models showed high accuracy in identifying black tea, with few misclassifications. However, identifying yellow tea was more challenging, with some yellow teas being misclassified as green teas and vice versa. This misclassification is understandable given the visual similarities between these tea types. Notably, Tea MvMtNet showed a slight improvement in accurately distinguishing between yellow and green teas, which may be attributed to its advanced feature extraction and classification capabilities. Although all models had difficulty distinguishing between similar tea types, the enhanced performance of Tea MvMtNet suggests its potential for more accurate classification, an advantage that is due to the multi-view and multi-task learning approach, which enables it to effectively utilize diverse data inputs.

[0184] The terms "first", "second", "third", etc. (e.g., first semantic vector, second semantic vector, third semantic vector, etc.) used in the present invention are only used to distinguish the corresponding components for the sake of clarity of description, and are not intended to limit any order or emphasize importance, etc. In addition, the term "connected" used in the present invention may be directly connected or indirectly connected via other components unless otherwise specified.

[0185] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-view quantitative evaluation method for tea, characterized in that: The following steps are involved: Collect images from the perspective of dry tea leaves, leaf bottoms, and tea soup color of the tea to be reviewed; The trained tea sensory evaluation model extracts and fuses the dry tea perspective image, the leaf bottom perspective image, and the tea soup color perspective image respectively to obtain multiple scales of dry tea fusion feature maps, leaf bottom fusion feature maps, and tea soup color fusion feature maps, and outputs the dry tea score, leaf bottom score, and tea soup color score of the tea based on the dry tea fusion feature map, leaf bottom fusion feature map, and tea soup color fusion feature map respectively; The trained tea sensory evaluation model splices dry tea fusion feature maps, leaf bottom fusion feature maps and soup color fusion feature maps of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, and outputs the aroma score, taste score and tea identification result of the tea based on the perspective fusion feature maps of multiple scales and the dry tea fusion feature maps, leaf bottom fusion feature maps and soup color fusion feature maps; The training method of the tea sensory evaluation model comprises the following steps: Obtaining several kinds of tea for model training, labeling the tea based on the tea sensory evaluation method, and obtaining a tea review data set; Collecting dry tea perspective images, leaf bottom perspective images, and tea soup color perspective images of several types of tea as a visual image dataset, and obtaining a training set based on the visual image dataset and the tea review dataset; The tea sensory evaluation model is trained using the training set to obtain a trained tea sensory evaluation model.

2. A multi-view quantitative evaluation method for tea according to claim 1, characterized in that: A bidirectional feature pyramid network is used to fuse dry tea feature maps of different scales extracted from the dry tea perspective image, leaf bottom feature maps of different scales extracted from the leaf bottom perspective image, and soup color feature maps of different scales extracted from the soup color perspective image. In the top-down fusion process, a convolutional block attention module is used to adjust the fused feature maps.

3. A multi-view quantitative evaluation method for tea according to claim 2, characterized in that: The calculation formula of the convolutional block attention module is: In the formula, F i df Represents the top-down fusion feature map of the i-th layer, A c is the channel attention, A s is spatial attention, A is the global attention that combines channel attention and spatial attention, and f (5) represents a one-dimensional convolution with a convolution kernel length of 5, f (7×7) Represents a two-dimensional convolution with a convolution kernel size of 7×7, AvgPool c Represents global average pooling along the height and width directions, AvgPool s Represents average pooling along the channel dimension, MaxPool s represents the maximum pooling along the channel dimension, σ s is the sigmoid activation function.

4. The multi-view quantitative evaluation method for tea according to claim 1, characterized in that: The following steps are also included: Acquire the review elements of dry tea, leaf bottom, and tea soup color in the tea sensory evaluation method, and encode the review elements to obtain a first semantic vector; The following processing is performed on the dry tea fusion feature map, the leaf bottom fusion feature map, and the tea color fusion feature map of the multiple scales: The larger scale fusion feature map of each perspective is used as the key and value, and the first semantic vector is used as the query vector. Cross-modal attention calculation is performed to obtain the prior visual semantic features of each perspective. The smaller-scale fused feature maps of each perspective are pooled to obtain the second semantic vector of each perspective; The prior visual semantic features and the corresponding second semantic vectors of each perspective are spliced ​​respectively, and after passing through a linear mapping layer, the dry tea score, leaf bottom score, and tea soup color score of the tea are output.

5. A multi-view quantitative evaluation method for tea according to claim 4, characterized in that: For review elements of the same perspective, the following processing is performed: Using the prior visual semantic features as node information; Perform self-attention calculation on the first semantic vector to obtain a weighted adjacency matrix; A graph attention network is used to learn the associations between the review elements.

6. A multi-view quantitative evaluation method for tea according to claim 4 or 5, characterized in that: The following steps are also included: Pooling the perspective fusion feature maps of the multiple scales to obtain a third semantic vector; The third semantic vector is concatenated with the prior visual semantic features of the dry tea, the tea leaves, and the tea soup color to obtain a global visual semantic feature; The aroma score, flavor score, and tea type identification result of the tea are output based on the global visual semantic features.

7. The multi-view quantitative evaluation method for tea according to claim 1, characterized in that: The calculation formula of the perspective fusion feature map of multiple scales is: Where W i Represents the dynamic weight of fine-grained aggregation, with dimensions h and w are the height and width of the feature map, n is the number of viewing angles, and F in represents the feature map of the i-th scale and the n-th perspective, f (1×1) represents a two-dimensional convolution with a convolution kernel size of 1×1, W i (j) represents the weight of fine-grained aggregation of feature maps at the i-th scale and j-th perspective, F i It is the feature map of the i-th scale after fusion of multiple perspectives.

8. The multi-view quantitative evaluation method for tea according to claim 1, characterized in that: In the process of training the tea sensory evaluation model, a loss function is introduced. The loss function L total for: In the formula, l cls is the loss function for tea classification, l aux is the regression loss function of the weighted total score, l i is the regression loss function of the i-th scoring dimension, α, β, and γ are hyperparameters and their sum is 1, l sl is the smooth L1 loss, ρ is the hyperparameter for adjusting the shape of the function, θ is the hyperparameter for adjusting the shape of the exponential distribution function, and λ i is the dynamic weight, w i is the rate of decrease of the loss of the ith task, t is the iteration index, T is the temperature coefficient used to control the weighted softness of the task, K is the number of tasks, and y i is the true value of the label, Y is the difference between the predicted value and the true value of the label, and μ is the factor for adjusting the focusing strength.

9. A multi-view quantitative evaluation device for tea, characterized in that: include: A collection unit, used to collect images from the perspective of dry tea, images from the perspective of leaf bottom, and images from the perspective of tea soup color of the tea to be reviewed; A tea sensory evaluation unit, comprising a trained tea sensory evaluation model, wherein the trained tea sensory evaluation model is used to output a dry tea score, a leaf bottom score, a soup color score, an aroma score, a taste score, and a tea type identification result based on the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image; Wherein, the trained tea sensory evaluation model includes: A feature extraction module is used to extract dry tea feature maps of multiple scales from the dry tea perspective image, extract leaf bottom feature maps of multiple scales from the leaf bottom perspective image, and extract soup color feature maps of multiple scales from the soup color perspective image; A feature fusion module is used to fuse dry tea feature maps of multiple scales to obtain dry tea fusion feature maps of multiple scales, fuse leaf bottom feature maps of multiple scales to obtain leaf bottom fusion feature maps of multiple scales, and fuse soup color feature maps to obtain soup color fusion feature maps of multiple scales; The semantic enhancement module is used to use the larger-scale dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map as the key and value of the corresponding perspective, encode the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method, and obtain the first semantic vector as the query vector of the corresponding perspective, perform cross-modal attention calculation to obtain the prior visual semantic features of each perspective; obtain the second semantic vector of each perspective after pooling the smaller-scale dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map; and respectively splice the prior visual semantic features and the corresponding second semantic vector of each perspective, and output the dry tea score, leaf bottom score, and soup color score of the tea; The perspective fusion module is used to splice the dry tea fusion feature map, leaf bottom fusion feature map and soup color fusion feature map of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, pool the perspective fusion feature maps of multiple scales to obtain a third semantic vector, splice the third semantic vector and the prior visual semantic features of each perspective to obtain a global visual semantic feature, and output the aroma score, taste score and tea identification result of the tea based on the global visual semantic feature.

10. A storage medium, characterized in that: It comprises a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a multi-view quantitative evaluation method for tea leaves as claimed in any one of claims 1 to 8.

Images