A multi-perspective quantitative evaluation method, device and storage medium for tea

Abstract

A multi-perspective quantitative evaluation method, device and storage medium for tea. The method includes the following steps: collecting dry tea perspective images, leaf bottom perspective images, and soup color perspective images of the tea to be evaluated; the trained tea sensory evaluation model extracts and fuses the dry tea perspective images, leaf bottom perspective images, and soup color perspective images respectively, to obtain fused feature maps of multiple scales from different perspectives, and outputs scores of the dry tea, leaf bottom, and soup color of the tea based on each fused feature map; also, fuse the fused feature maps of different perspectives of the same scale along the channel dimension to obtain perspective fused feature maps of multiple scales, and based on the perspective fused feature maps of multiple scales and the fused feature maps of each perspective, output the aroma score, taste score, and tea type identification result of the tea. Through multi-scale fusion and multi-perspective fusion, the present invention can more comprehensively and accurately reflect the overall quality of tea, and effectively promote the modernization and standardization of the tea industry.

Description

Technical Field

[0001] The present invention relates to the technical field of tea quality assessment, and in particular, to a multi-perspective quantitative evaluation method, device and storage medium for tea. Background Art

[0002] The sensory evaluation of tea is a crucial link in the tea industry, directly determining the market positioning of tea, the experience of consumers and the market recognition. According to the Chinese national standard GB / T 23776-2018, the traditional tea sensory evaluation method is that professional tea tasters conduct sensory evaluation of tea from five main aspects: dry tea, soup color, aroma, taste and leaf bottom based on their long-term accumulated sensory experience. However, the traditional evaluation method highly depends on the experience of tea tasters, making the evaluation results often have strong subjectivity and are easily affected by factors such as the state, experience difference of tea tasters and environmental factors, resulting in instability and inconsistency of the evaluation results.

[0003] In order to improve the consistency and stability of tea evaluation results, instrumental analysis methods such as gas chromatography-mass spectrometry, high performance liquid chromatography, electronic tongue and electronic nose are introduced into the tea evaluation process. The instrumental analysis results can provide certain scientific basis for tea quality. However, the high detection cost, long detection cycle, high experimental environment requirements and complex detection process make it difficult to meet the rapidity and large-scale evaluation requirements of tea evaluation, and the application scenarios are limited.

[0004] With the development of artificial intelligence technology, algorithms based on machine learning and deep learning are gradually applied to tea quality management in order to reduce costs and improve efficiency. Patent CN118351369A discloses a machine vision-based intelligent identification and grading method for tea. This grading method only focuses on the characteristics of dry tea, tea soup and leaf bottom from a perspective, without considering the impact of aroma and taste on the overall quality of tea. Patent CN118797413A extracts the shape, color and texture characteristics of tea by using an electronic eye, collects taste signals by using an electronic tongue, collects aroma signals by using an electronic nose, and combines near-infrared spectral information to comprehensively evaluate the grade of Gongfu black tea. This method is based on multiple instrumental analysis methods, not only with a long detection time, high cost and cumbersome process, but also the output evaluation results can only qualitatively divide the grade of black tea, and it is difficult to quantitatively evaluate the complex characteristics and features of the same grade of tea in the five main dimensions of dry tea, soup color, aroma, taste and leaf bottom, and the mutual influence between each dimension cannot be well captured and reflected, resulting in poor accuracy of tea sensory evaluation. Summary of the Invention

[0005] An object of the present invention is to provide a multi - perspective quantitative evaluation method for tea. The constructed tea sensory evaluation model is based on the input of three visual feature information of dry tea, soup color, and leaf bottom. Through multi - scale fusion processing and multi - perspective fusion processing, it can quantitatively capture the characteristics of tea in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom and identify the type of tea, comprehensively and accurately reflect the overall quality of tea, thereby effectively promoting the modernization and standardization of the tea industry.

[0006] The above object of the present invention is achieved by the following technical solutions:

[0007] A multi - perspective quantitative evaluation method for tea includes the following steps:

[0008] Collect the dry - tea perspective images, leaf - bottom perspective images, and soup - color perspective images of the tea to be evaluated;

[0009] The trained tea sensory evaluation model respectively extracts and fuses the dry - tea perspective images, leaf - bottom perspective images, and soup - color perspective images to obtain dry - tea fusion feature maps, leaf - bottom fusion feature maps, and soup - color fusion feature maps at multiple scales, and respectively outputs the dry - tea score, leaf - bottom score, and soup - color score of the tea based on the dry - tea fusion feature maps, leaf - bottom fusion feature maps, and soup - color fusion feature maps;

[0010] The trained tea sensory evaluation model splices the dry - tea fusion feature maps, leaf - bottom fusion feature maps, and soup - color fusion feature maps at the same scale along the channel dimension to obtain perspective fusion feature maps at multiple scales, and based on the perspective fusion feature maps at multiple scales, as well as the dry - tea fusion feature maps, leaf - bottom fusion feature maps, and soup - color fusion feature maps, outputs the aroma score, taste score, and tea type identification result of the tea;

[0011] Among them, the training method of the tea sensory evaluation model includes the following steps:

[0012] Obtain several kinds of tea for model training, label the tea based on the tea sensory evaluation method to obtain a tea evaluation data set;

[0013] Collect the dry - tea perspective images, leaf - bottom perspective images, and soup - color perspective images of several kinds of tea as a visual image data set, and obtain a training set based on the visual image data set and the tea evaluation data set;

[0014] Use the training set to train the tea sensory evaluation model to obtain the trained tea sensory evaluation model.

[0015] In this technical solution, it is necessary to collect the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image of the tea to be reviewed as the input of the tea sensory evaluation model. Among them, the dry tea perspective image is an image directly collected from the dry tea sample. After soaking the dry tea sample, the soup is poured out to collect the soup color perspective image, and the remaining leaf bottom is used to collect the leaf bottom perspective image. In some embodiments, the three perspective images are all taken by a camera under consistent lighting conditions.

[0016] In this technical solution, the trained tea sensory evaluation model is used to process the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image. Specifically, a visual backbone network is used to extract the dry tea feature maps of different scales from the dry tea perspective image, and then multi-scale feature fusion is performed on the dry tea feature maps of different scales to obtain dry tea fusion feature maps of multiple scales. Finally, based on the dry tea fusion feature maps of multiple scales, the score of the dry tea is output.

[0017] Similarly, a visual backbone network is used to extract the leaf bottom feature maps of different scales from the leaf bottom perspective image, and then multi-scale feature fusion is performed on the leaf bottom feature maps of different scales to obtain leaf bottom fusion feature maps of multiple scales. Finally, based on the leaf bottom fusion feature maps of multiple scales, the score of the leaf bottom is output. A visual backbone network is used to extract the soup color feature maps of different scales from the soup color perspective image, and then multi-scale feature fusion is performed on the soup color feature maps of different scales to obtain soup color fusion feature maps of multiple scales. Finally, based on the soup color fusion feature maps of multiple scales, the score of the soup color is output.

[0018] In this technical solution, by using multi-scale feature fusion, it is possible to more effectively utilize the visual information of the three perspectives to capture the global and local detailed features of the tea.

[0019] In one or more embodiments, ResNet101 is used as the visual backbone network in the CLIP (Contrastive Language-Image Pre-training) pre-trained model.

[0020] In this technical solution, the trained tea sensory evaluation model concatenates the fusion feature maps of three perspectives of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales. In some embodiments, the perspective fusion feature maps of different scales generate the weights for the per-pixel weighted fusion of the perspective fusion feature maps of each scale after pooling. After normalization along the channel dimension, dynamic weights are used for fine-grained weighted aggregation to obtain the perspective fusion feature maps with multi-scale fusion. Subsequently, the perspective fusion feature maps are concatenated with the feature maps of the dry tea fusion feature map, the bottom leaf fusion feature map, and the soup color fusion feature map, and the aroma score, taste score, and tea type identification result of the tea are output. By concatenating the fusion feature maps of different perspectives, the model's ability to represent the multi-dimensional quality characteristics of tea can be effectively enhanced. Taking the tea as a whole, based on the characteristics of the dry tea, the bottom leaf, and the soup color, the aroma and taste of the tea are predicted, and the type of tea is comprehensively judged in combination with five main dimensions.

[0021] In this technical solution, when training the tea sensory evaluation model, the training set mainly includes two parts: the tea evaluation data set and the visual image data set. Among them, for the tea evaluation data set, experts score the five dimensions of dry tea, soup color, aroma, taste, and bottom leaf based on the Chinese national standard GB / T 23776-2018, and the calculation formula for the overall score of each type of tea is:

[0022] Overall score = 0.25 × dry tea + 0.1 × soup color + 0.25 × aroma + 0.3 × taste + 0.1 × bottom leaf

[0023] The acquisition methods of the dry tea perspective image, the bottom leaf perspective image, and the soup color perspective image in the visual image data set are the same as those of the tea to be evaluated. In some embodiments, three-perspective images of other types of tea are also acquired in the same way as the test set for model training.

[0024] In this technical solution, during the model training process, the visual image data set is used as the input of the model. By using the loss function and adjusting the parameters of the model, the expert evaluation data set is used to evaluate the tea score output by the model, monitor various performance indicators of the model, and finally obtain the trained tea sensory evaluation model.

[0025] In this technical solution, by inputting the feature information from three perspectives of dry tea, soup color, and leaf bottom through multi-scale fusion, it is possible to effectively utilize the visual information from the three perspectives to capture the global and local detailed features of tea. At the same time, based on the feature fusion from the three perspectives, the representation ability of the model for the multi-dimensional quality features of tea can be enhanced. Furthermore, by only inputting the visual features from the three perspectives of dry tea, soup color, and leaf bottom, it is possible to quantitatively capture the features of tea in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom and identify the type of tea, more comprehensively and accurately reflecting the overall quality of tea. Moreover, it can significantly shorten the detection time, reduce the detection cost, simplify the detection process, and is conducive to efficiently evaluating the quality of a large number of teas, promoting the modernization and standardization of the tea industry.

[0026] Furthermore, traditional visual models usually abstract high-level semantic features by gradually downsampling the feature maps as the network depth increases. Although this method effectively abstracts the global semantic information, it often leads to the loss of local detailed information, especially for the case where the local detailed information in the tea leaf image occupies only a small number of pixels. Therefore, to solve this problem, it is necessary to use multi-scale feature maps to enrich the semantic representation so that the model can simultaneously focus on the global and local details.

[0027] As a preferred implementation manner of multi-scale fusion in the present invention, a bidirectional feature pyramid network is used to fuse the dry tea feature maps of different scales extracted from the dry tea perspective image, the leaf bottom feature maps of different scales extracted from the leaf bottom perspective image, and the soup color feature maps of different scales extracted from the soup color perspective image, and in the top-down fusion process, a convolutional block attention module is used to adjust the fused feature maps.

[0028] In this technical solution, a bidirectional feature pyramid network is used to perform multi-scale feature map fusion, thereby enhancing the semantic representation of different scales. Specifically, for each feature layer level downsampled in the same perspective, such as the dry tea perspective image, bottom-up fusion is achieved by upsampling the feature map of the adjacent lower layer to the size matching the current layer feature map, and then these feature maps are summed using learnable normalization weights, and then deep fusion is performed through a 3×3 depthwise separable convolution and a 1×1 convolution; conversely, top-down fusion involves pooling and downsampling the feature maps of the adjacent upper layer to match the size of the current layer, and then performing operations similar to the bottom-up fusion process, and its calculation formula is

[0029]

[0030] In the formula, and respectively represent the feature maps fused from bottom to top and from top to bottom in the $i$-th layer. UP and DOWN represent the upsampling and pooling downsampling operations respectively to ensure the consistency of the size of the feature maps, PC 1×1 represents a pointwise convolution with a kernel size of 1×1, DC 3×3 represents a depthwise separable convolution with a kernel size of 3×3, and the parameters $\beta_1$, $\beta_2$, $\beta_3$, $\beta$ j are learnable weights used to adjust the contribution of the fused feature maps. $\varphi$ is the ReLU activation function to ensure that the weights remain non-negative. $\epsilon$ is a small constant included to prevent division by zero when normalizing the weights. $\sigma$ represents the swish activation function, which is commonly used in neural networks to enhance the network's non-linear representation ability.

[0031] In this technical solution, during the top-down fusion process, a Convolutional Block Attention Module (CBAM) is also introduced. The convolutional block attention module can sequentially apply channel and spatial attention to capture the attention distribution in the global feature map. By using the convolutional block attention module in the top-down fusion process to adjust the fused feature map, a more refined semantic feature representation is generated.

[0032] In some preferred embodiments, the calculation formula of the convolutional block attention module is:

[0033]

[0034] In the formula, represents the feature map fused from top to bottom in the $i$-th layer, $A$ c is the channel attention, $A$ s is the spatial attention, $A$ is the global attention combined by the channel attention and the spatial attention, $f$ (5) represents a one-dimensional convolution with a kernel length of 5, which is used to generate the channel attention weights, $f$ (7×7) represents a two-dimensional convolution with a kernel size of 7×7, which is used to generate the spatial attention weights, AvgPool c represents global average pooling along the height and width directions, which is used to capture the global information of each channel, AvgPool s represents average pooling along the channel dimension, MaxPool s represents max pooling along the channel dimension, $\sigma$ s is the sigmoid activation function, which is used to compress the global attention weights obtained by element-wise multiplying the channel and spatial attention weights into a gating value between 0 and 1. The symbol $\oplus$ represents the concatenation operation, that is, combining the feature maps obtained by two different pooling modes along the channel dimension. The symbol $\odot$ represents element-wise multiplication.

[0035] In this technical solution, by improving the convolutional block attention module, not only the number of parameters is reduced and the parallelism of calculations is increased, enhancing the efficiency of the model, but also the output amount of information in the global feature map is adjusted, strengthening the correlation of image features in the channel and spatial dimensions, enriching the fine-grained semantic feature representation, and thus improving the generalization performance of the network.

[0036] As a preferred implementation manner of the tea sensory evaluation model in the present invention, prior review elements and cross-modal attention mechanisms are further introduced to further enhance the model's understanding ability of the visual attributes of tea, so as to more deeply explore the flavor characteristics of tea.

[0037] Specifically, it includes the following steps:

[0038] Obtain the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method, and obtain the first semantic vector after encoding the review elements;

[0039] Perform the following processing on the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps of multiple scales:

[0040] Use the larger-scale fusion feature maps of each perspective as keys and values, and the first semantic vector as the query vector, and perform cross-modal attention calculations respectively to obtain the prior visual semantic features of each perspective;

[0041] Pool the smaller-scale fusion feature maps of each perspective to obtain the second semantic vector of each perspective;

[0042] Concatenate the prior visual semantic features and the corresponding second semantic vectors of each perspective respectively, and pass through a linear mapping layer to output the dry tea score, leaf bottom score, and soup color score of the tea.

[0043] In this technical solution, review elements of three perspectives are obtained based on the tea sensory evaluation method. Among them, the review elements of dry tea include shape, tenderness, color, integrity, and cleanliness; the review elements of leaf bottom include tenderness, color, brightness, and evenness; the review elements of soup color include color type and chroma, brightness, and clarity. After obtaining the review elements of the three perspectives, the review elements are encoded to obtain the first semantic vector, which is used as the query vector in the cross-modal attention mechanism.

[0044] In this technical solution, multiple-scale dry tea fusion feature maps, multiple-scale bottom leaf fusion feature maps, and multiple-scale soup color fusion feature maps are obtained through attention-enhanced multi-scale fusion processing. For the fusion feature maps with larger scales in the fusion feature maps of each perspective, they are used as keys and values, and cross-modal attention calculation is performed in combination with the corresponding first semantic vector to obtain the prior visual semantic features of the current perspective. The fusion feature maps with larger scales have more detailed information. Through cross-modal attention calculation, it is possible to better imitate the review logic of experts and understand visual features from the dimensions of multiple review elements corresponding to the current perspective. At the same time, for the fusion feature maps with smaller scales in the fusion feature maps of each perspective, they have more global information, and a second semantic vector is obtained after pooling. Finally, the prior visual semantic features and the second semantic features of each perspective are concatenated, passed through a linear mapping layer, and the quality evaluation scores corresponding to the perspectives are output.

[0045] In this technical solution, by using the review elements corresponding to each perspective as query vectors and performing cross-modal attention calculation in combination with the fusion feature maps with larger scales, it is possible to endow each perspective with visual feature representations corresponding to the prior semantic enhancement provided by multiple review elements, enabling the model to analyze the visual features of tea from the perspectives of the review elements of each perspective in a way closer to that of expert reviews, effectively improving the model's ability to understand the visual attributes of tea.

[0046] Furthermore, for the review elements of the same perspective, the following processing is performed:

[0047] Use the prior visual semantic features as node information;

[0048] Perform self-attention calculation on the first semantic vector to obtain a weighted adjacency matrix;

[0049] Use a graph attention network to learn the associations between the review elements.

[0050] In this technical solution, a graph attention network (Graph Attention Network, GAT) is used to learn the correlations between the review elements of the same perspective to further enhance the model's ability to understand the visual attributes of tea. Specifically, taking the dry tea perspective as an example, the prior visual semantic features corresponding to the dry tea perspective are used as node information, and self-attention calculation is performed on the five review elements of the encoded shape, tenderness, color, integrity, and cleanliness, thereby generating a weighted adjacency matrix representing the associations between the review elements. Subsequently, the graph attention network is used to learn the relationships between the semantic information of each review element to further enhance the prior visual semantic features.

[0051] Furthermore, the following steps are also included:

[0052] After pooling the perspective fusion feature maps of the multiple scales, a third semantic vector is obtained;

[0053] The third semantic vector and the prior visual semantic features of the dry tea, leaf bottom, and soup color are concatenated to obtain the global visual semantic feature;

[0054] Based on the global visual semantic feature, the aroma score, taste score, and tea type identification result of the tea are output.

[0055] In this technical solution, after obtaining the perspective fusion feature maps of multiple scales, pooling is performed, such as spatial pyramid pooling and average pooling, to obtain the third semantic vector. Then, the third semantic vector is concatenated with the prior visual semantic features from the dry tea perspective, the prior visual semantic features from the leaf bottom perspective, and the prior visual semantic features from the soup color perspective to obtain the complete global perspective semantic feature. Finally, the aroma score and taste score are output through a linear mapping layer, and the tea type identification result is output through a linear mapping layer and a softmax layer.

[0056] Further, the calculation formula for the perspective fusion feature maps of the multiple scales is:

[0057]

[0058] In the formula, W i represents the dynamic weight of fine-grained aggregation, with a dimension of h and w are respectively the height and width of the feature map, n is the number of perspectives, F in represents the feature map of the i-th scale and the n-th perspective, f (1×1) represents a two-dimensional convolution with a convolution kernel size of 1×1, W i (j) represents the weight of the fine-grained aggregation of the feature map of the i-th scale and the j-th perspective, which is sliced from W i along the channel dimension. F i is the feature map of the i-th scale after fusing multiple perspectives. ⊕ and ⊙ respectively represent the concatenation operation and the element-wise product operation.

[0059] In this technical solution, a fine-grained dynamic fusion mechanism is proposed. Feature maps of the same scale size from different perspectives are concatenated along the channel dimension, and then two layers of convolutions with a convolution kernel size of 1×1 are used to generate the weights for pixel-wise weighted fusion of the multi-perspective feature maps, and the softmax function is used for normalization along the channel dimension. Finally, these dynamic weights are used for fine-grained weighted aggregation to obtain the multi-scale feature maps after multi-perspective fusion.

[0060] In this technical solution, through the above dynamic fusion mechanism, dynamic weight aggregation can be performed on pixel-level multi-perspective features, effectively enhancing the multi-granularity and multi-semantics of visual features and improving the generalization performance.

[0061] As a preferred embodiment of the loss function in the present invention, a loss function is introduced during the training of the tea sensory evaluation model. The loss function L total for:

[0062]

[0063] In this technical solution, considering the need to simultaneously predict the tea type and the scores of the five dimensions, and the uneven distribution of the quality evaluation score data, a multi-task attention network (MTAN) is used to perform multi-task balance. At the same time, a multi-task balanced focal regression loss function is proposed to solve the problem of category imbalance in classification tasks.

[0064] Where, l cls is the loss function for tea classification, which is the standard cross entropy loss function. aux and l i The regression loss function used to predict the score is calculated in the same way, where l aux It is a regression loss function of the weighted total score, which is used to provide the network model with information on the importance of each scoring dimension to assist the optimization process; i is the regression loss function of the i-th scoring dimension. α, β, and γ are the hyperparameters that balance the three loss functions and their sum is 1. sl is the smooth L1 loss, where ρ is the hyperparameter that adjusts the shape of the function. θ is the hyperparameter that adjusts the shape of the exponential distribution function. i is the true value of the label, Y is the difference between the predicted value and the true value of the label, and μ is the factor for adjusting the focusing strength.

[0065] With this setting, the closer the regression loss value of the sample is to 0, the smaller the weight will become, and vice versa. This directly amplifies the difference between the two, allowing the network model to adaptively mine difficult samples during the optimization process.

[0066] In order to balance multiple tasks, a dynamic weight λ is assigned to the loss value of each regression task in the time step of the training process i , w in Eq. i (·) represents the rate of decrease of the loss of the i-th task, and t is the iteration index. In experiments, the rate of decrease of the task loss is expressed as the ratio of the loss value of the next iteration to the loss value of the previous iteration. In the formula, T is the temperature coefficient, which controls the softness of the task weighting. A larger T results in a more uniform distribution across tasks. Finally, after softmax normalization, it is multiplied by K to ensure that ∑kλi(t) = K, where K is the number of tasks.

[0067] With this setting, if the loss of a certain task drops faster, the weight value assigned to this task will become smaller, thus slowing down its learning and optimization speed. On the contrary, if the loss drops slower, the learning and optimization speed will increase. This dynamic adjustment of the learning weights of each task during the optimization process aims to achieve balanced optimization of each task, preventing some tasks from being over-optimized while others are under-optimized.

[0068] Another object of the present invention is to provide a multi-perspective quantitative evaluation device for tea, which processes the input dry tea, soup color, and leaf bottom images of tea based on the trained tea sensory evaluation model, outputs the scores of tea in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom, and identifies the type of tea.

[0069] The above object of the present invention is achieved by the following technical solutions:

[0070] A multi-perspective quantitative evaluation device for tea, comprising:

[0071] An acquisition unit for acquiring dry tea perspective images, leaf bottom perspective images, and soup color perspective images of the tea to be evaluated;

[0072] A tea sensory evaluation unit, including a trained tea sensory evaluation model, and the trained tea sensory evaluation model is used to output dry tea scores, leaf bottom scores, soup color scores, aroma scores, taste scores, and tea type identification results based on the dry tea perspective images, leaf bottom perspective images, and soup color perspective images;

[0073] Wherein, the trained tea sensory evaluation model includes:

[0074] A feature extraction module for extracting dry tea feature maps of multiple scales of dry tea perspective images, extracting leaf bottom feature maps of multiple scales of leaf bottom perspective images, and extracting soup color feature maps of multiple scales of soup color perspective images;

[0075] A feature fusion module for fusing dry tea feature maps of multiple scales to obtain dry tea fusion feature maps of multiple scales, fusing leaf bottom feature maps of multiple scales to obtain leaf bottom fusion feature maps of multiple scales, and fusing soup color feature maps to obtain soup color fusion feature maps of multiple scales;

[0076] A semantic enhancement module, which is used to use the dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map with a larger scale as the key and value of the corresponding perspective, and use the first semantic vector obtained by encoding the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method as the query vector of the corresponding perspective, and perform cross-modal attention calculation to obtain the prior visual semantic features of each perspective; obtain the second semantic vector of each perspective after pooling the dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map with a smaller size; and respectively splice the prior visual semantic features of each perspective and the corresponding second semantic vector, and output the dry tea score, leaf bottom score, and soup color score of the tea.

[0077] A perspective fusion module, which is used to splice the dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map with the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, pool the perspective fusion feature maps of multiple scales to obtain a third semantic vector, splice the third semantic vector and the prior visual semantic features of each perspective to obtain a global visual semantic feature, and output the aroma score, taste score, and tea type identification result of the tea based on the global visual semantic feature.

[0078] The present invention also provides a storage medium, including a stored computer program, wherein when the computer program runs, it controls the device where the storage medium is located to execute any one of the foregoing tea multi-perspective quantitative evaluation methods.

[0079] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0080] 1. By inputting the feature information of three perspectives of dry tea, soup color, and leaf bottom through multi-scale fusion, the present invention can effectively utilize the visual information of the three perspectives to capture the global and local detail features of the tea. At the same time, based on the feature fusion of the three perspectives, the representation ability of the model for the multi-dimensional quality features of the tea can be enhanced. Furthermore, only by inputting the visual features of the three perspectives of dry tea, soup color, and leaf bottom, the features of the tea in five dimensions of dry tea, soup color, aroma, taste, and leaf bottom can be quantitatively captured and the type of the tea can be identified, more comprehensively and accurately reflecting the overall quality of the tea.

[0081] 2. The present invention uses a bidirectional feature pyramid network to fuse feature maps of different scales under different perspectives, and introduces a convolutional block attention module to adjust the fused feature maps during the top-down fusion process, so as to generate a more refined semantic feature representation, and at the same time enable the model to pay attention to both the global and local details.

[0082] 3. By using the review elements corresponding to each perspective as query vectors and combining a larger-scale fused feature map for cross-modal attention calculation, the present invention can endow each perspective with a visual feature representation corresponding to the prior semantic enhancement provided by multiple review elements, enabling the model to analyze the visual features of tea from the perspective of the review elements of each perspective in a manner closer to that of expert review, effectively improving the model's understanding ability of the visual attributes of tea;

[0083] 4. By using a graph attention network to learn the correlation between the review elements under the same perspective, the present invention can further enhance the model's understanding ability of the visual attributes of tea;

[0084] 5. Through a dynamic fusion mechanism, the present invention can perform dynamic weight aggregation on multi-perspective features at the pixel level, effectively enhancing the multi-granularity and multi-semantics of visual features and improving the generalization performance;

[0085] 6. By designing a multi-task balanced focus regression loss function, the network model can adaptively mine difficult samples during the optimization process, and the dynamic adjustment of the learning weights of each task during the optimization process aims to achieve balanced optimization of each task, preventing some tasks from being over-optimized while other tasks are under-optimized;

[0086] 7. The present invention can significantly shorten the detection time, reduce the detection cost, simplify the detection process, facilitate the efficient evaluation of the quality of a large number of teas, and promote the modernization and standardization of the tea industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of this application, and do not limit the embodiments of the present invention. In the drawings:

[0088] Figure 1 is a flowchart of the multi-perspective quantitative evaluation method in a specific embodiment of the present invention;

[0089] Figure 2 is a schematic diagram of the model framework of the trained tea sensory evaluation model in a specific embodiment of the present invention;

[0090] Figure 3 shows the information of tea samples and the scores and images of representative tea samples in the training set in a specific embodiment of the present invention, where (a) the number of black tea, yellow tea, and green tea samples; (b) the sensory scores of the representative tea samples by the expert panel in five dimensions; (c) the representative images of the tea samples;

[0091] Figure 4 shows the sensory score distribution of all tea samples in a specific embodiment of the present invention, including five dimensions and the total score;

[0092] Figure 5 Shows the comparison of the accuracy of the flavor factor scores of Tea MvMtNet with two other representative models in specific embodiments of the present invention;

[0093] Figure 6 Shows the comparison between the predicted values and the true values of the evaluation scores of Tea MvMtNet for three representative tea samples in specific embodiments of the present invention;

[0094] Figure 7 Shows the comparison of the classification accuracy of Tea MvMtNet and two other models in tea type recognition evaluated by a confusion matrix in specific embodiments of the present invention. Detailed implementation manners

[0095] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with embodiments and drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0096] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "front", "rear", "left", "right", "upper", "lower", "vertical", "horizontal", "high", "low", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the protection scope of the present invention.

[0097] Embodiment 1:

[0098] As Figure 1 and Figure 2 shown, a multi-view quantitative evaluation method for tea includes the following steps:

[0099] Collect the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image of the tea to be evaluated;

[0100] The trained tea sensory evaluation model respectively extracts and fuses the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image to obtain dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps at multiple scales, and respectively outputs the dry tea score, the leaf bottom score, and the soup color score of the tea based on the dry tea fusion feature maps, the leaf bottom fusion feature maps, and the soup color fusion feature maps;

[0101] The trained tea sensory evaluation model concatenates the dry tea fusion feature maps, bottom leaf fusion feature maps, and soup color fusion feature maps of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales. Based on the perspective fusion feature maps of multiple scales, as well as the dry tea fusion feature maps, bottom leaf fusion feature maps, and soup color fusion feature maps, the aroma score, taste score, and tea category identification result of the tea are output;

[0102] Among them, the training method of the tea sensory evaluation model includes the following steps:

[0103] Obtain several kinds of teas for model training, and label the teas based on the tea sensory evaluation method to obtain a tea evaluation data set;

[0104] Collect the dry tea perspective images, bottom leaf perspective images, and soup color perspective images of several kinds of teas as the visual image data set, and obtain a training set based on the visual image data set and the tea evaluation data set;

[0105] Use the training set to train the tea sensory evaluation model to obtain the trained tea sensory evaluation model.

[0106] In some embodiments, the three perspective images are scaled proportionally to the same size. For images with different heights and widths, they can be adjusted to the same size by filling with gray borders, and then the pixel values are normalized for the visual backbone network to extract feature maps from different perspectives. In some preferred embodiments, the three perspective images are all adjusted to 512×512 pixels.

[0107] In one or more embodiments, advanced semantic features are extracted from the normalized dry tea perspective images, bottom leaf perspective images, and soup color perspective images in a parameter-sharing manner, and then the feature maps of each perspective are fused in a parameter-separating manner to obtain the corresponding fusion feature maps.

[0108] In one or more embodiments, the fusion feature maps of the three perspectives are respectively subjected to pyramid pooling and average pooling to obtain semantic vectors, and then the corresponding scores are output through a linear mapping layer.

[0109] In one or more embodiments, ResNet101 is used as the visual backbone network in the CLIP (Contrastive Language-Image Pre-training) pre-trained model.

[0110] In some preferred embodiments, the calculation formula for the perspective fusion feature maps of multiple scales is:

[0111]

[0112] In the formula, W iDenote the dynamic weights for fine-grained aggregation, with the dimension of h and w are the height and width of the feature map respectively, n is the number of perspectives, and F in denotes the feature map of the i-th scale and the n-th perspective, and f (1×1) denotes a two-dimensional convolution with a kernel size of 1×1, and W i (j) denotes the weights for the fine-grained aggregation of the feature map of the i-th scale and the j-th perspective, and F i is the feature map of the i-th scale after fusing multiple perspectives.

[0113] Example 2:

[0114] Based on Example 1, as Figure 2 shown, a bidirectional feature pyramid network is used to fuse the dry tea feature maps of different scales extracted from the dry tea perspective image, the leaf bottom feature maps of different scales extracted from the leaf bottom perspective image, and the soup color feature maps of different scales extracted from the soup color perspective image. During the top-down fusion process, a convolutional block attention module is used to adjust the fused feature maps.

[0115] In some preferred embodiments, for each feature map level after downsampling in the images of each perspective, bottom-up fusion is achieved by upsampling the feature maps of the adjacent lower layer to the size matching the current layer's feature map, then summing these feature maps using learnable normalization weights, and then performing depth fusion through a 3×3 depthwise separable convolution and a 1×1 convolution; conversely, top-down fusion involves pooling and downsampling the feature maps of the adjacent upper layer to match the size of the current layer, and then performing operations similar to the bottom-up fusion process. Its calculation formula is

[0116]

[0117] In the formula, and respectively denote the feature maps of bottom-up and top-down fusion of the i-th layer. UP and DOWN respectively represent the upsampling and pooling downsampling operations to ensure the consistency of the feature map size. PC 1×1 denotes a pointwise convolution with a kernel size of 1×1, and DC 3×3 denotes a depthwise separable convolution with a kernel size of 3×3. The parameters β1, β2, β3, and β j are learnable weights used to adjust the contribution of the fused feature maps. φ is the ReLU activation function to ensure that the weights remain non-negative. ∈ is a small constant included to prevent division by zero when normalizing the weights. σ represents the swish activation function, which is usually used in neural networks to enhance the network's non-linear representation ability.

[0118] In a more preferred embodiment, by using a convolutional block attention module in the top-down fusion process to adjust the fused feature map, a finer semantic feature representation is generated. Wherein, the calculation formula of the convolutional block attention module is:

[0119]

[0120] In the formula, represents the feature map of the i-th layer of top-down fusion, Ac is the channel attention, As is the spatial attention, A is the global attention combining the channel attention and the spatial attention, and f (5) represents a one-dimensional convolution with a convolutional kernel length of 5, and f (7×7) represents a two-dimensional convolution with a convolutional kernel size of 7×7, AvgPool c represents global average pooling along the height and width directions, and AvgPool s represents average pooling along the channel dimension, MaxPool s represents max pooling along the channel dimension, and σ s is the sigmoid activation function.

[0121] Embodiment 3:

[0122] Based on the above embodiment, as Figure 2 shown, the evaluation method further includes the following steps:

[0123] Obtain the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method, and obtain the first semantic vector after encoding the review elements;

[0124] Perform the following processing on the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps of multiple scales:

[0125] Use the larger-scale fusion feature maps of each perspective as keys and values, and the first semantic vector as the query vector, and perform cross-modal attention calculations respectively to obtain the prior visual semantic features of each perspective;

[0126] Pool the smaller-scale fusion feature maps of each perspective to obtain the second semantic vector of each perspective;

[0127] Concatenate the prior visual semantic features of each perspective and the corresponding second semantic vector respectively, and pass through a linear mapping layer to output the dry tea score, leaf bottom score, and soup color score of the tea.

[0128] In some embodiments, the Transformer language model in CLIP is used to encode the review elements.

[0129] Taking the perspective of dry tea as an example, in some embodiments, after obtaining the dry tea fusion feature maps of five scales, the first four dry tea fusion feature maps with larger scales are used as keys and values, and the five review elements of shape, tenderness, color, integrity, and cleanliness are used as query vectors after encoding, and cross-modal attention is performed to obtain the prior visual semantic features from the dry tea perspective; the dry tea fusion feature map of the last layer with the smallest scale is subjected to spatial pyramid pooling and average pooling to obtain the second semantic vector from the dry tea perspective; finally, the prior visual semantic features and the second semantic vector from the dry tea perspective are concatenated, and after passing through a linear mapping layer, the score of the dry tea is output. Similarly, in some embodiments, the scores of the leaf bottom and the soup color can also be calculated and output respectively by using cross-modal attention based on the leaf bottom fusion feature maps, the soup color fusion feature maps, and the review elements of the leaf bottom and the soup color at multiple scales.

[0130] In some preferred embodiments, the calculation formula for cross-modal attention processing is:

[0131]

[0132] In the formula, Q pe is the sequence representation of multiple review elements in the current perspective. After encoding the language description of the review elements with the language transformer in CLIP, a query vector with a dimension belonging to is obtained. Here, L is the number of review elements in the current perspective, and D is the dimension of the encoded semantic vector. and respectively represent the feature maps of the i-th scale in the CLIP visual encoder as key and value vectors. ∥·∥ and T respectively represent the matrix two-norm operation and transpose. For the operation of cross-modal attention, cosine similarity is used, and the ReLU activation function is used to activate and then square element-wise to obtain a more general attention representation. This method can directly set the negative correlation attention weights to zero, avoiding the problem that the softmax activation will set the negative correlation weights to greater than zero, and achieving the purpose of adaptively filtering irrelevant information. Then, the results of cross-modal attention for all scales are aggregated to obtain the prior visual semantic feature H sp .

[0133] In some embodiments, for the review elements of the same perspective, the following processing is performed:

[0134] Use the prior visual semantic features as node information;

[0135] Perform self-attention calculation on the first semantic vector to obtain a weighted adjacency matrix;

[0136] Use a graph attention network to learn the associations between the review elements.

[0137] In this embodiment, the Graph Attention Network (GAT) is used to learn the correlation between various review elements from the same perspective, so as to further enhance the model's ability to understand the visual attributes of tea leaves.

[0138] In some preferred embodiments, the enhancement of the prior visual semantic features is calculated using the following formula:

[0139]

[0140] and represent the language description query and key encoding vectors of the i-th and j-th review elements in the current perspective. By performing self-attention, a weighted adjacency matrix can be obtained, where β ij is the weight value at the i,j position in the adjacency matrix. α ij represents the normalized attention weight of the neighbor nodes at the i and j positions in the graph attention network. represents the prior visual semantic feature of the semantic enhancement of the i-th prior review element in the current perspective, which is regarded as node information. is the shared linear transformation of each node, which is used to convert the input features into high-level features in order to obtain sufficient expressive power. ⊕ represents the concatenation operation, which concatenates the feature vectors of two neighbor nodes and then passes through a single-layer feed-forward neural network and applies the LeakyReLU non-linear activation function to obtain the attention weights, which are normalized using softmax. Finally, the product of α ij and β ij is used as the aggregation weight of the neighbor node feature information, and the non-linear activation function SiLU is used to activate the aggregated feature representation to obtain the final prior visual semantic feature of the prior review element semantic enhancement.

[0141] Embodiment 4:

[0142] Based on the above embodiment, as Figure 2 shown, the method further includes the following steps:

[0143] Pool the perspective fusion feature maps of multiple scales to obtain a third semantic vector;

[0144] Concatenate the third semantic vector and the prior visual semantic features of the dry tea, leaf bottom, and soup color to obtain the global visual semantic feature; [[ID=4 (2]]

[0145] Output the aroma score, taste score, and tea type identification result of the tea based on the global visual semantic feature.

[0146] Embodiment 5:

[0147] Based on the above embodiments, as Figure 2 shown, a loss function is introduced during the process of training the tea sensory evaluation model. The loss function L total is as follows:

[0148]

[0149] In the formula, l cls is the loss function for tea classification, l aux is the regression loss function for the weighted total score, l i is the regression loss function for the i-th scoring dimension. α, β, and γ are hyperparameters and the sum of the three is 1. l sl is the smooth L1 loss, ρ is the hyperparameter for adjusting the function shape, θ is the hyperparameter for adjusting the shape of the exponential distribution function, λ i is the dynamic weight, w i is the descent rate of the loss for the i-th task, t is the iteration index, T is the temperature coefficient for controlling the task weighting softness, K is the number of tasks, y i is the label true value, Y is the difference between the predicted value and the label true value, and μ is the factor for adjusting the focusing strength.

[0150] In this embodiment, by setting this loss function, when the regression loss value of the sample is closer to 0, the weight will become smaller, and vice versa, which directly amplifies the difference between the two, enabling the network model to adaptively mine difficult samples during the optimization process. At the same time, if the loss of a certain task decreases faster, the weight value assigned to this task will become smaller, thereby slowing down its learning and optimization speed. On the contrary, if the loss decreases slower, the learning and optimization speed will increase. This dynamic adjustment of the learning weights of each task during the optimization process aims to achieve balanced optimization of each task, preventing some tasks from being over-optimized while other tasks are under-optimized.

[0151] Embodiment 6:

[0152] Based on the above embodiments, a multi-perspective quantitative evaluation device for tea includes:

[0153] An acquisition unit for acquiring the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image of the tea to be evaluated;

[0154] A tea sensory evaluation unit including a trained tea sensory evaluation model, and the trained tea sensory evaluation model is used to output the dry tea score, the leaf bottom score, the soup color score, the aroma score, the taste score, and the tea category identification result based on the dry tea perspective image, the leaf bottom perspective image, and the soup color perspective image;

[0155] Among them, the trained tea sensory evaluation model includes:

[0156] A feature extraction module, configured to extract dry tea feature maps of multiple scales from dry tea perspective images, extract leaf bottom feature maps of multiple scales from leaf bottom perspective images, and extract soup color feature maps of multiple scales from soup color perspective images;

[0157] A feature fusion module, configured to fuse dry tea feature maps of multiple scales to obtain dry tea fusion feature maps of multiple scales, fuse leaf bottom feature maps of multiple scales to obtain leaf bottom fusion feature maps of multiple scales, and fuse soup color feature maps to obtain soup color fusion feature maps of multiple scales;

[0158] A semantic enhancement module, configured to use the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps with larger scales as keys and values for the corresponding perspectives, use the first semantic vectors obtained by encoding the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method as query vectors for the corresponding perspectives, perform cross-modal attention calculation to obtain prior visual semantic features for each perspective; use the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps with smaller scales to obtain second semantic vectors for each perspective after pooling; and respectively concatenate the prior visual semantic features and the corresponding second semantic vectors for each perspective, and output the dry tea score, leaf bottom score, and soup color score of the tea;

[0159] A perspective fusion module, configured to concatenate the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps of the same scale along the channel dimension to obtain perspective fusion feature maps of multiple scales, pool the perspective fusion feature maps of multiple scales to obtain a third semantic vector, concatenate the third semantic vector and the prior visual semantic features of each perspective to obtain a global visual semantic feature, and output the aroma score, taste score, and tea type identification result of the tea based on the global visual semantic feature.

[0160] In this embodiment, the constructed tea sensory evaluation model effectively integrates the visual inputs of the three perspectives, enhances the model's ability to learn and predict quality attributes based on visual data from multiple perspectives, realizes a more detailed and comprehensive evaluation, and the model can capture subtle visual cues related to sensory attributes such as aroma and taste, thereby improving the accuracy and reliability of tea quality management.

[0161] Embodiment 7:

[0162] Based on the above embodiment, a storage medium includes a stored computer program, wherein when the computer program runs, it controls the device where the storage medium is located to execute a tea multi-perspective quantitative evaluation method shown in any one of the foregoing embodiments.

[0163] In this embodiment, the above storage medium is a computer-readable storage medium. When the multi-perspective quantitative evaluation device for tea is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above embodiment methods of the present invention, it can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0164] Embodiment 8:

[0165] In this embodiment, the performance of the tea sensory evaluation model Tea MvMtNet adopted is tested.

[0166] 1) Data collection:

[0167] The tea samples were collected by the Key Laboratory of Refined Sichuan Tea of Sichuan Agricultural University from April 2021 to July 2022. A total of 806 tea samples from 18 tea-producing provinces (municipalities directly under the Central Government, autonomous regions) in China were collected, including 346 black teas, 245 yellow teas, and 215 green teas. Among them, 703 tea samples were used for model training, and the remaining 103 tea samples (including 27 green teas, 30 yellow teas, and 46 black teas) were used for model performance testing. All tea samples were stored at 4°C. The geographical distribution and variety distribution of the tea samples are as Figure 3 shown.

[0168] 2) Construction of the training set:

[0169] For the 703 tea samples used for model training, an expert panel composed of five experts from the key laboratory conducted sensory scoring to obtain a tea review dataset. Among them, the expert panel scored according to the Chinese national standard GB / T 23776-2018. The five dimensions of the scoring include dry tea, soup color, taste, aroma and leaf bottom. Each item is scored on a 10-point scale, and the score of each tea sample is the average of the expert scores. Specifically, the dry tea appearance of the dry tea samples is first scored. About 3 grams of tea sample is placed in a white porcelain cup, 150 ml of boiling water is added, and the lid is covered. Black tea and yellow tea are soaked for 5 minutes, and green tea is soaked for 4 minutes. After soaking, the tea soup is immediately poured into a white porcelain bowl to evaluate the soup color. The leaf bottom in the porcelain cup is used to evaluate the quality of the aroma and leaf bottom. Figure 3 Shown are the sensory scores assigned by experts to some representative tea samples.

[0170] Figure 4 The distribution and probability density of the scores for all tea samples across the five dimensions and the total score are shown. Scores range from 5 to 10, with most tea samples scoring around 9. The quality scores exhibit a normal distribution, indicating that the tea samples in this study are representative and cover a wide range of tea types. This balanced representation enhances the model's generalization across different quality levels.

[0171] A Canon EOS 80D (Canon, Japan) camera was used to capture images of the dry tea leaves, tea leaves, and tea soup to create a visual image dataset. Specifically, approximately 5 grams of dry tea leaves were randomly sampled from each tea sample and evenly spread in a 7.5 cm × 7.5 cm sample cell. The tea soup was poured into a white porcelain bowl and immediately photographed. After steeping, the tea leaves were transferred from the porcelain cup to a 7.5 × 7.5 cm sample cell and photographed. All images were captured under consistent lighting conditions to ensure comparability. The resulting images are uncompressed sRGB color images with a pixel size of 1984 × 1984. Figure 3 Perspective images of some representative tea samples are shown.

[0172] Ultimately, the visual image dataset and the tea review dataset together constitute the training set of the model.

[0173] 3) Training the model:

[0174] In CLIP, ResNet101 was used as the visual backbone network, and its pre-trained parameters were frozen throughout training. For training data preprocessing, all images were uniformly scaled to 512×512 pixels. Images with varying heights and widths were resized to 512×512 using gray border padding. Data augmentation was performed using random rotations of 90, 180, and 270 degrees, as well as vertical flips and vertical flips combined with rotations. In the experiments, the Adam optimizer was used with a training batch size of 7, and the network was trained for 500 epochs.

[0175] 4) Performance test:

[0176] In this embodiment, 103 tea samples were used for testing. The image inputs included dry tea perspective images, leaf bottom perspective images, and soup color perspective images. The performance of Tea MvMtNet was compared with that of seven common deep learning algorithms. In the test, all seven comparison models were pre-trained on the ImageNet dataset. For the prediction of each perspective score, global pooling was used to process the feature maps of each perspective, and then the corresponding scores were output through a separate multi-layer perceptron (MLP). For the prediction of aroma and taste scores and the identification of tea categories, the feature vectors after global pooling of each perspective were concatenated, and three independent MLPs were used to output the scores of aroma, taste, and tea categories.

[0177] Table 1:

[0178]

[0179]

[0180] In Table 1, the formula for calculating MAE% (mean absolute error percentage) is MAE% = (1 / n)∑|predicted value - true value|×100%. The test results are shown in Table 1. Tea MvMtNet is superior to other models in all metrics, showing its advantage in integrating multi-perspective inputs in comprehensive tea quality assessment. It achieved an accuracy of 98.11% in tea category identification, exceeding the best-performing comparison model by 0.94%. For the five-dimensional quantitative scoring metrics - dry tea, soup color, aroma, taste, and leaf bottom - the MAE% of Tea MvMtNet were 19.60%, 28.40%, 26.48%, 31.67%, and 17.20% respectively, which were improved by 0.54%, 0.01%, 0.07%, 0.26%, and 0.44% respectively compared with the best-performing comparison model. In addition, the MAE% of this model in the weighted total score was 17.23%, which was 2.63% lower than that of the closest competitor. In the total score on a 10-point scale, this corresponds to an error of 0.1723, which is comparable to the typical error range of many senior assessors. These improvements in accuracy and MAE% in each dimension indicate the effectiveness of the multi-perspective and multi-task method adopted by Tea MvMtNet, showing its ability to be a practical tool in tea quality management.

[0181] To clearly illustrate the superior performance in quantitative scoring, a visual comparison of the predicted values and true values of Tea MvMtNet with those of two other models was made, as Figure 5As shown. In these figures, the diagonal lines represent perfect predictions, and the degree of proximity of the data points to these lines indicates the accuracy of the model. The results show that Tea MvMtNet is generally superior to other models, especially in regions where the scoring distribution is sparse, highlighting its robustness and precision. For dry tea, soup color, aroma, taste, and leaf bottom, TeaMvMtNet shows consistent high accuracy. Especially in the three dimensions of dry tea, aroma, and taste with larger weights, Tea MvMtNet demonstrates significant advantages. The excellent performance in these key areas contributes to the effectiveness of the weighted total score.

[0182] To further illustrate the performance of Tea MvMtNet, a representative tea sample was selected from black tea, yellow tea, and green tea respectively, representing a tea sample with a balanced flavor, a soup color defect type, and a taste defect type. Figure 6 The radar charts showing the evaluation scores of the three representative samples are presented. In these charts, the blue area represents the predicted value, and the red area represents the true value. The degree of overlap of these areas reflects the accuracy of the model. The high overlap observed in all five dimensions indicates that Tea MvMtNet has the ability to predict multiple quality attributes with high precision. Specifically, the representative black tea shows a more balanced performance in the five flavor factors, while the representative yellow tea performs excellently in dry tea and taste. In contrast, the representative green tea shows some weaknesses in taste. Despite the different flavor characteristics of these tea samples, Tea MvMtNet can accurately predict them.

[0183] Furthermore, in addition to quantitative quality assessment, the performance of Tea MvMtNet in tea type recognition was also evaluated through a confusion matrix. As Figure 6 shown, these matrices compare the classification accuracy of Tea MvMtNet with that of two other models. All models show a high accuracy rate in identifying black tea, with few misclassifications. However, identifying yellow tea is more challenging, and some yellow teas are misclassified as green teas, and vice versa. This misclassification is understandable because these tea types are visually similar. Notably, Tea MvMtNet shows a slight improvement in accurately distinguishing yellow tea and green tea, which may be attributed to its advanced feature extraction and classification capabilities. Although all models encounter difficulties in distinguishing similar tea types, the enhanced performance of Tea MvMtNet indicates its potential for more accurate classification. This advantage benefits from the multi-perspective and multi-task learning method, enabling it to effectively utilize diverse data inputs.

[0184] In the present invention, the terms "first", "second", "third", etc. (such as the first semantic vector, the second semantic vector, the third semantic vector, etc.) are only used to distinguish the corresponding components for the sake of clear description, and are not intended to limit any order or emphasize importance, etc. In addition, the term "connection" used in the present invention, without special explanation, may be directly connected or indirectly connected via other components.

[0185] The specific embodiments described above have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi - perspective quantitative evaluation method for tea, characterized in that, It includes the following steps: Collect the dry tea perspective images, leaf bottom perspective images, and soup color perspective images of the tea leaves to be reviewed; The trained tea sensory evaluation model extracts and fuses the dry tea perspective images, leaf bottom perspective images, and soup color perspective images respectively, to obtain dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps at multiple scales, and outputs the dry tea score, leaf bottom score, and soup color score of the tea leaves respectively based on the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps; The trained tea sensory evaluation model concatenates the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps at the same scale along the channel dimension to obtain perspective fusion feature maps at multiple scales, and outputs the aroma score, taste score, and tea type identification result of the tea leaves based on the perspective fusion feature maps at multiple scales, as well as the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps; Among them, the training method of the tea sensory evaluation model includes the following steps: Obtain several kinds of tea leaves for model training, and label the tea leaves based on the tea sensory evaluation method to obtain a tea review data set; Collect the dry tea perspective images, leaf bottom perspective images, and soup color perspective images of several kinds of tea leaves as a visual image data set, and obtain a training set based on the visual image data set and the tea review data set; Use the training set to train the tea sensory evaluation model to obtain the trained tea sensory evaluation model.

2. The multi - perspective quantitative evaluation method for tea leaves according to claim 1, wherein Adopt a bidirectional feature pyramid network to fuse the dry tea feature maps at different scales extracted from the dry tea perspective images, the leaf bottom feature maps at different scales extracted from the leaf bottom perspective images, and the soup color feature maps at different scales extracted from the soup color perspective images, and in the top-down fusion process, use a convolutional block attention module to adjust the fused feature maps.

3. A multi-perspective quantitative evaluation method for tea leaves according to claim 2, characterized in that The calculation formula of the convolutional block attention module is: Where, F i df represents the feature map fused from top to bottom of the i-th layer, A c is the channel attention, A s is the spatial attention, A is the global attention combined by the channel attention and the spatial attention, f (5) represents a one-dimensional convolution with a convolution kernel length of 5, f (7×7) represents a two-dimensional convolution with a convolution kernel size of 7×7, AvgPool c represents the global average pooling along the height and width directions, AvgPool s represents the average pooling along the channel dimension, MaxPool s represents the maximum pooling along the channel dimension, σ s is the sigmoid activation function.

4. A multi - perspective quantitative evaluation method for tea leaves according to claim 1, characterized in that, It also includes the following steps: Obtain the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method, and encode the review elements to obtain the first semantic vector; Perform the following processing on the dry tea fusion feature maps, leaf bottom fusion feature maps, and soup color fusion feature maps at multiple scales: Use the larger-scale fusion feature maps of each perspective as keys and values, and the first semantic vector as the query vector, and perform cross-modal attention calculation respectively to obtain the prior visual semantic features of each perspective; Pool the smaller-scale fusion feature maps of each perspective to obtain the second semantic vector of each perspective; Concatenate the prior visual semantic features of each perspective and the corresponding second semantic vectors respectively, and pass through a linear mapping layer to output the dry tea score, leaf bottom score, and soup color score of the tea leaves.

5. A multi-perspective quantitative evaluation method for tea according to claim 4, characterized in that, For the review elements of the same perspective, perform the following processing: Use the prior visual semantic features as node information; Perform self-attention calculation on the first semantic vector to obtain a weighted adjacency matrix; Adopt a graph attention network to learn the association between the review elements.

6. A multi - perspective quantitative evaluation method for tea leaves according to claim 4 or 5, characterized in that, It also includes the following steps: Pool the perspective fusion feature maps at multiple scales to obtain a third semantic vector; Concatenate the third semantic vector and the prior visual semantic features of dry tea, leaf bottom, and soup color to obtain the global visual semantic feature; Output the aroma score, taste score, and tea type identification result of the tea based on the global visual semantic features.

7. A multi - perspective quantitative evaluation method for tea leaves according to claim 1, characterized in that, The calculation formula for the perspective fusion feature maps of multiple scales is as follows: Where, W i represents the dynamic weight of fine-grained aggregation, with the dimension of h and w are the height and width of the feature map respectively, n is the number of perspectives, and F in represents the feature map of the i-th scale and the n-th perspective, and f (1×1) represents a two-dimensional convolution with a kernel size of 1×1, and W i (j) represents the weight of fine-grained aggregation of the feature map of the i-th scale and the j-th perspective, and F i is the feature map of the i-th scale after fusing multiple perspectives.

8. A multi - perspective quantitative evaluation method for tea leaves according to claim 1, characterized in that, Introduce a loss function during the process of training the tea sensory evaluation model, and the loss function L total is as follows: where \(l\) cls is the loss function for tea classification, \(l\) aux is the regression loss function for the weighted total score, \(l\) i is the regression loss function for the \(i\)-th scoring dimension, \(\alpha\), \(\beta\), \(\gamma\) are hyperparameters and the sum of the three is 1, \(l\) sl is the smooth L1 loss, \(\rho\) is the hyperparameter that adjusts the shape of the function, \(\theta\) is the hyperparameter that adjusts the shape of the exponential distribution function, \(\lambda\) i is the dynamic weight, \(w\) i is the rate of decrease of the loss for the \(i\)-th task, \(t\) is the iteration index, \(T\) is the temperature coefficient used to control the task weighting softness, \(K\) is the number of tasks, \(y\) i is the label ground truth, \(Y\) is the difference between the predicted value and the label ground truth, and \(\mu\) is the factor that adjusts the focusing strength.

9. A multi - perspective quantitative evaluation device for tea, characterized in that, Including: A collection unit for collecting the dry tea perspective image, leaf bottom perspective image, and soup color perspective image of the tea to be evaluated. A tea sensory evaluation unit, including a trained tea sensory evaluation model, which is used to output the dry tea score, leaf bottom score, soup color score, aroma score, taste score, and tea type identification result based on the dry tea perspective image, leaf bottom perspective image, and soup color perspective image. Among them, the trained tea sensory evaluation model includes: A feature extraction module for extracting multiple scales of dry tea feature maps of the dry tea perspective image, extracting multiple scales of leaf bottom feature maps of the leaf bottom perspective image, and extracting multiple scales of soup color feature maps of the soup color perspective image. A feature fusion module for fusing multiple scales of dry tea feature maps to obtain multiple scales of dry tea fusion feature maps, fusing multiple scales of leaf bottom feature maps to obtain multiple scales of leaf bottom fusion feature maps, and fusing soup color feature maps to obtain multiple scales of soup color fusion feature maps. A semantic enhancement module for using the dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map with a larger scale as the key and value of the corresponding perspective, using the first semantic vector obtained by encoding the review elements of dry tea, leaf bottom, and soup color in the tea sensory evaluation method as the query vector of the corresponding perspective, performing cross-modal attention calculation to obtain the prior visual semantic features of each perspective; obtaining the second semantic vector of each perspective after pooling the dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map with a smaller size; and respectively splicing the prior visual semantic features of each perspective and the corresponding second semantic vector to output the dry tea score, leaf bottom score, and soup color score of the tea. A perspective fusion module for splicing the dry tea fusion feature map, leaf bottom fusion feature map, and soup color fusion feature map of the same scale along the channel dimension to obtain multiple scales of perspective fusion feature maps, pooling the multiple scales of perspective fusion feature maps to obtain a third semantic vector, splicing the third semantic vector and the prior visual semantic features of each perspective to obtain the global visual semantic features, and outputting the aroma score, taste score, and tea type identification result of the tea based on the global visual semantic features.

10. A storage medium, characterized in that, Including a stored computer program, wherein, when the computer program runs, it controls the device where the storage medium is located to execute a tea multi-perspective quantitative evaluation method according to any one of claims 1 to 8.

Images