A mulberry leaf tea fermentation degree discrimination method and system based on image intelligent recognition

Abstract

The application discloses a mulberry leaf tea fermentation degree discrimination method and system based on image intelligent recognition, and the method comprises the following steps: collecting images of standard fermentation degree mulberry leaf tea before and after fermentation, images in a fermentation process and tea soup images, and simultaneously collecting corresponding near-infrared spectrum data of the images; pre-processing the collected images of standard fermentation degree mulberry leaf tea before and after fermentation, images in a fermentation process and tea soup images and the corresponding near-infrared spectrum data; extracting external contour features of the tea leaf images and extracting image RGB values by an Otsu threshold segmentation algorithm, and extracting discrimination information of the near-infrared spectrum data to obtain a vector matrix; respectively constructing a mulberry leaf tea fermentation degree initial discriminator and a deep fusion discriminator; inputting real-time collected images of standard fermentation degree mulberry leaf tea and image corresponding near-infrared spectrum data into the mulberry leaf tea fermentation degree initial discriminator and the deep fusion discriminator for judgment; and realizing judgment of the mulberry leaf tea fermentation degree based on image intelligent recognition.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically, it relates to a method and system for determining the fermentation degree of mulberry leaf tea based on intelligent image recognition. Background Technology

[0002] Mulberry leaves, a plant used both as food and medicine, have been a traditional Chinese medicine since ancient times. Rich in flavonoids, amino acids, polysaccharides, alkaloids, and other active ingredients, mulberry leaves are used not only for silkworm rearing but also to make mulberry leaf tea. Mulberry leaf tea is believed to have effects such as lowering blood sugar, lowering blood pressure, and anti-aging. As a substitute tea, mulberry leaf tea differs from traditional tea. The polyphenol content in mulberry leaves is significantly lower than in traditional tea. Through fermentation, polyphenols are converted into beneficial components such as theaflavins and thearubigins, giving black tea its unique flavor and color. Therefore, the fermentation degree of mulberry leaves is not obvious, the redness of the leaf veins is lower than in traditional tea, and the tea soup is not as bright red. In the production process, the degree of fermentation of mulberry leaf tea is an important indicator, directly affecting its taste, aroma, color, and the release of nutrients. To assess the quality of mulberry leaf tea, it is necessary to determine its degree of fermentation.

[0003] Chinese invention patent application CN202510301535.0 discloses a machine learning-based method for identifying the fermentation degree of oolong tea. This method preprocesses characteristic component factors obtained from oolong tea to construct training and testing sets for distinguishing different fermentation types. It then employs a machine learning approach combining statistics and ensemble learning to accurately identify the different fermentation types of oolong tea.

[0004] The drawback of existing patents is that the fermentation degree of the mulberry leaf tea is determined by sampling and testing with artificial chemical reagents, but it is difficult to accurately determine the degree of fermentation manually. Secondly, the determination of the fermentation degree of the finished mulberry leaf tea cannot provide feedback on abnormal conditions or abnormal steps in the fermentation process. Summary of the Invention

[0005] To address the problem that the fermentation degree of mulberry leaf tea is difficult to determine manually, this invention provides a method and system for judging the fermentation degree of mulberry leaf tea based on image intelligent recognition.

[0006] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition, comprising the following steps:

[0008] S1. Collect images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup at the standard fermentation degree, and simultaneously collect the corresponding near-infrared spectral data of the images;

[0009] S2. Preprocess the collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data at the standard fermentation degree.

[0010] S3. Extract the external contour features of the tea image and extract the RGB values ​​of the image using the Otsu threshold segmentation algorithm. Use the fuzzy weighted discrimination information extraction method to extract discrimination information from the near-infrared spectral data to obtain a vector matrix.

[0011] S4. Construct an initial fermentation degree judge and a deep fusion judge for mulberry leaf tea, respectively;

[0012] S5. Input the real-time collected near-infrared spectral data into the mulberry leaf tea fermentation degree initial judgment device for judgment. If the judgment result is lower than the preset mulberry leaf tea fermentation degree, proceed to step S6. If the judgment result is greater than or equal to the preset mulberry leaf tea fermentation degree, proceed to the next round of mulberry leaf tea fermentation degree discrimination process.

[0013] S6. Input the real-time collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data of the standard fermentation degree into the deep fusion judgment device for judgment; if the judgment result is lower than the preset mulberry leaf tea fermentation degree, proceed to step S7; if the judgment result is greater than or equal to the preset mulberry leaf tea fermentation degree, proceed to the next round of mulberry leaf tea fermentation degree judgment.

[0014] S7. If the judgment result in step S6 is lower than the preset fermentation degree of mulberry leaf tea, then perform image indexing on the images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data of the standard fermentation degree mulberry leaf tea to determine the process steps in which the fermentation process caused the failure.

[0015] Furthermore, the collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, and tea infusion images were preprocessed using Gaussian filtering for image denoising. Detailed steps for Gaussian filtering image denoising preprocessing are as follows:

[0016] First, distinguish between images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup to determine the current processing step of mulberry leaf tea. Then, differentiate the images based on the camera codes or locations carried during image transmission.

[0017] Based on different processing steps of mulberry leaf tea, different pre-trained two-dimensional Gaussian functions are matched to filter the background image in the mulberry leaf tea image;

[0018] If the images are of mulberry leaf tea before and after fermentation, or during the fermentation process, then the background image in the mulberry leaf tea image will be filtered directly.

[0019] If the image is of tea infusion, the tea leaves in the mulberry leaf tea image are filtered as background to obtain a tea color image. The threshold or neighborhood for background filtering varies depending on the object being filtered.

[0020] Furthermore, the near-infrared spectral data of mulberry leaf tea collected in step S2 are processed by multivariate scattering correction. The detailed steps of multivariate scattering correction are as follows:

[0021] Calculate the mean of all spectral data as the baseline spectrum;

[0022] Perform linear regression on each measured spectrum;

[0023] Calculate the corrected spectrum. This is the reference spectrum.

[0024] Use the obtained a i and b i Correcting the original spectrum:

[0025]

[0026] Where a i and b i The calculation method is as follows:

[0027] a. Calculate the mean: Calculate the mean for each measured spectrum X. i mean and the mean of the reference spectrum

[0028] b. Calculate the covariance between the measured spectrum and the reference spectrum:

[0029]

[0030] c. Calculate the variance of the reference spectrum:

[0031]

[0032] d. Calculate the regression coefficient a i and b i :

[0033]

[0034] Furthermore, the detailed steps in S3 for extracting the external contour features of the tea leaf image using the Otsu thresholding algorithm are as follows:

[0035] First, calculate the maximum inter-class variance between the foreground target and the background image as the segmentation threshold; the formula for calculating the inter-class variance is:

[0036]

[0037] Where g is the inter-class variance, The average gray level of the image. The average gray level, The proportion of the target pixels to the total number of pixels in the image. The threshold T, which is the ratio of background pixels to the total number of pixels in the entire image, is obtained by traversing the data to find the threshold T that maximizes the inter-class variance g. This threshold T is then used as the segmentation threshold for the image.

[0038] Then, the image is segmented according to the segmentation threshold to obtain the extracted outer contour features of the tea image.

[0039] Furthermore, in step S3, the RGB values ​​of the image are extracted from the tea color image using an image segmentation algorithm based on color thresholding;

[0040] S301. First, input the preprocessed tea soup image into the program and normalize the grayscale values ​​of each channel of the tea tree color image.

[0041] S302. Convert the tea image from the RGB color space to the HSV color space.

[0042] S303. Set the segmentation thresholds for H, S, and V channels through preset experimental analysis.

[0043] S304. Then, determine whether each pixel in the H, S, and V channels of the image in the HSV color space is within the set threshold range (the upper and lower boundaries need to be set according to the scene). If the result is yes, retain the pixel value at the corresponding position in the input image; otherwise, set it to 0.

[0044] S305. The tea and tea color images in the tea soup image are segmented through the steps S301-S304 above.

[0045] Furthermore, the detailed steps for extracting discrimination information from near-infrared spectral data using a fuzzy weighted discrimination information extraction method to obtain the vector matrix are as follows:

[0046] The spectral data after multivariate scattering correction is defined as sample set D;

[0047] Secondly, define the fuzzy class scatter matrix and the fuzzy class scatter matrix of the sample set;

[0048] Then define the membership function u. ij and cluster center function v i ;

[0049] Substituting the membership function and cluster center function into the fuzzy class intra-class scatter matrix and the fuzzy class inter-class scatter matrix, the objective function of FLDA is derived.

[0050] Then perform eigenvalue decomposition to obtain its eigenvalues ​​and eigenvectors;

[0051] Performing fuzzy linear discriminant analysis (FLDA) yields the eigenvectors corresponding to the largest total number of categories minus one eigenvalue, which form the transformation vector matrix W of FLDA. FLDA .

[0052] Furthermore, the near-infrared spectral data is extracted using a fuzzy weighted discrimination information extraction method to obtain a vector matrix. Then, the vector matrix is ​​normalized to obtain a range value, which is used as the near-infrared spectral data discrimination threshold of the initial judge of the fermentation degree of mulberry leaf tea. This threshold is usually the near-infrared spectrum of 1325.27 to 1889.49 nm.

[0053] The deep fusion discriminator combines the segmentation threshold of the extracted tea leaf image's outer contour features, the threshold set for each pixel in the H, S, and V channels of the image in the HSV color space based on the image's RGB values, and the near-infrared spectral data discrimination threshold of the initial tea leaf fermentation degree discriminator. This data is then input into an extreme learning machine for training to obtain multiple discrimination thresholds for the deep fusion discriminator. Finally, it comprehensively judges whether the images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea infusion images, and near-infrared spectral data at the standard fermentation degree meet the corresponding thresholds for preset fermentation degree conversion.

[0054] The training process of an extreme learning machine is mainly divided into two stages: random feature mapping and linear parameter solving.

[0055] Phase 1: First, given the number of neurons in the hidden layer, randomly generate the connection weights w between the input layer and the hidden layer, as well as the biases b of the hidden layer neurons. Then, select a nonlinear mapping as the activation function to calculate the hidden layer output matrix H.

[0056] The second stage: To obtain the minimum training error, the Extreme Learning Machine (ELM) solves for β by minimizing the approximate squared difference between the network output Hβ (β being the weights between the hidden and output layers) and the expected output L. The final result is... H * Let H be the Moore-Penrose generalized inverse matrix. These are the weights between the hidden layer and the output layer obtained after training.

[0057] A fermentation degree discrimination system for mulberry leaf tea based on image intelligent recognition includes a mulberry leaf tea related data acquisition module, a near-infrared spectral data acquisition module, an image preprocessing module, a near-infrared spectral data preprocessing module, a tea leaf image external contour feature extraction module, an image RGB value extraction module, a near-infrared spectral data identification information extraction module, a mulberry leaf tea fermentation degree initial judgment module, a deep fusion judgment module, and a data storage module.

[0058] The mulberry leaf tea related data acquisition module is used to collect images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup of mulberry leaf tea with standard fermentation degree;

[0059] Near-infrared spectral data acquisition module, used to acquire near-infrared spectral data corresponding to images related to mulberry leaf tea;

[0060] The image preprocessing module is connected to the mulberry leaf tea related data acquisition module. It preprocesses the acquired images of mulberry leaf tea before and after fermentation, images during the fermentation process, and tea soup images by using Gaussian filtering.

[0061] The near-infrared spectral data preprocessing module is connected to the near-infrared spectral data acquisition module and preprocesses the near-infrared spectral data through multivariate scattering correction.

[0062] The tea image external contour feature extraction module is connected to the image preprocessing module and extracts the external contour features of the tea image through the Otsu threshold segmentation algorithm;

[0063] The image RGB value extraction module is connected to the image preprocessing module and uses a color threshold-based image segmentation algorithm to extract the tea color image from the tea soup image.

[0064] The near-infrared spectral data identification information extraction module uses a fuzzy weighted identification information extraction method to extract identification information from near-infrared spectral data, resulting in a vector matrix composed of feature values.

[0065] The initial fermentation degree judgment module of mulberry leaf tea is connected to the near-infrared spectral data identification information extraction module. By normalizing the extracted vector matrix, a range value is obtained. This range value is used as the near-infrared spectral data discrimination threshold of the initial fermentation degree judgment module of mulberry leaf tea to make an initial judgment on the fermentation degree of mulberry leaf tea.

[0066] The deep fusion judgment module communicates with the mulberry leaf tea fermentation degree initial judgment module. It combines the segmentation threshold of the extracted tea leaf image's outer contour features, the threshold set for each pixel in the H, S, and V channels of the image in the HSV color space based on the image's RGB values, and the near-infrared spectral data discrimination threshold of the mulberry leaf tea fermentation degree initial judgment module. Then, it inputs the data into an extreme learning machine for training to obtain multiple discrimination thresholds for the deep fusion judgment module. Finally, it comprehensively judges whether the images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and near-infrared spectral data of standard fermentation degree mulberry leaf tea meet the corresponding thresholds for preset fermentation degree conversion.

[0067] The data storage module is used to store the near-infrared spectral data discrimination threshold of the initial fermentation degree judge of mulberry leaf tea, the various discrimination thresholds of the deep fusion judge, and the historical mulberry leaf tea related data images and near-infrared spectral data collected by the entire image-based intelligent recognition mulberry leaf tea fermentation degree discrimination system.

[0068] Compared with the prior art, the present invention has the following advantages:

[0069] By effectively acquiring images and tea infusion images during the mulberry leaf tea processing, as well as the corresponding near-infrared spectral data, and then performing data processing steps such as data preprocessing, feature extraction, image RGB value extraction, and near-infrared spectral data identification information extraction, an initial fermentation degree judge and a deep fusion judge for mulberry leaf tea are constructed. Finally, the initial fermentation degree judge is used to determine the fermentation degree of mulberry leaf tea by inputting real-time acquired near-infrared spectral data, while the deep fusion judge is used to determine the fermentation degree of mulberry leaf tea by inputting real-time acquired images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea infusion images, and the corresponding near-infrared spectral data of these images. This achieves the determination of mulberry leaf tea fermentation degree based on image intelligent recognition, and also enables real-time identification of abnormal conditions or abnormal steps in the mulberry leaf tea fermentation process based on images during the mulberry leaf tea processing. Attached Figure Description

[0070] Figure 1 This is an overall flowchart of a method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition in an embodiment of the present invention;

[0071] Figure 2 This is an overall structural block diagram of a mulberry leaf tea fermentation degree discrimination system based on image intelligent recognition in an embodiment of the present invention.

[0072] The markings in the diagram are as follows: 10 - Mulberry leaf tea related data acquisition module, 20 - Near-infrared spectral data acquisition module, 30 - Image preprocessing module, 40 - Near-infrared spectral data preprocessing module, 50 - Tea leaf image external contour feature extraction module, 60 - Image RGB value extraction module, 70 - Near-infrared spectral data identification information extraction module, 80 - Mulberry leaf tea fermentation degree initial judgment module, 90 - Deep fusion judgment module, 100 - Data storage module. Detailed Implementation

[0073] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0074] like Figure 1 As shown, this embodiment provides a method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition, including the following steps:

[0075] S1. Collect images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup at the standard fermentation degree, and simultaneously collect the corresponding near-infrared spectral data of the images;

[0076] S2. Preprocess the collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data at the standard fermentation degree.

[0077] S3. Extract the external contour features of the tea image and extract the RGB values ​​of the image using the Otsu threshold segmentation algorithm. Use the fuzzy weighted discrimination information extraction method to extract discrimination information from the near-infrared spectral data to obtain a vector matrix.

[0078] S4. Construct an initial fermentation degree judge and a deep fusion judge for mulberry leaf tea, respectively;

[0079] S5. Input the real-time acquired near-infrared spectral data into the initial fermentation degree judge of mulberry leaf tea for judgment. If the judgment result is lower than the preset fermentation degree of mulberry leaf tea, proceed to step S6. If the judgment result is greater than or equal to the preset fermentation degree of mulberry leaf tea, proceed to the next round of mulberry leaf tea fermentation degree discrimination process. The initial fermentation degree judge of mulberry leaf tea is conducive to judging the fermentation degree of mulberry leaf tea relatively accurately with a small amount of data. Then, for the near-infrared spectral data of mulberry leaf tea with a lower than the preset fermentation degree of mulberry leaf tea, the images before and after fermentation, the images during the fermentation process, and the tea soup images, along with the corresponding near-infrared spectral data of the images, are input into the deep fusion judge for judgment through correlation indexing. This reduces the amount of data for the first judgment and the complexity of the analysis and judgment.

[0080] S6. Input the real-time collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data of the standard fermentation degree into the deep fusion judgment device for judgment; if the judgment result is lower than the preset mulberry leaf tea fermentation degree, proceed to step S7; if the judgment result is greater than or equal to the preset mulberry leaf tea fermentation degree, proceed to the next round of mulberry leaf tea fermentation degree judgment.

[0081] S7. If the judgment result in step S6 is lower than the preset fermentation degree of mulberry leaf tea, then perform image indexing on the images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data of the standard fermentation degree mulberry leaf tea to determine the process steps in which the fermentation process caused the failure.

[0082] Images of mulberry leaf tea before and after fermentation, during the fermentation process, and infusion were collected and pre-processed using Gaussian filtering for image denoising. Detailed steps for Gaussian filtering image denoising preprocessing are as follows:

[0083] First, distinguish between images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup to determine the current processing step of mulberry leaf tea. Then, differentiate the images based on the camera codes or locations carried during image transmission.

[0084] Based on different processing steps of mulberry leaf tea, different pre-trained two-dimensional Gaussian functions are matched to filter the background image in the mulberry leaf tea image;

[0085] Two-dimensional Gaussian function:

[0086]

[0087] Where (x, y) represents the coordinates of the center point. Indicates variance.

[0088] The value of each pixel in the neighborhood can be determined by the following formula:

[0089]

[0090] The formula for calculating variance D is:

[0091]

[0092] S i,j This represents a neighborhood of size (2k+1)×(2k+1) around the center point (i,j).

[0093] If the images are of mulberry leaf tea before and after fermentation, or during the fermentation process, then the background image in the mulberry leaf tea image will be filtered directly.

[0094] If the image is of tea infusion, the tea leaves in the mulberry leaf tea image are filtered as background to obtain a tea color image. The threshold or neighborhood for background filtering varies depending on the object being filtered.

[0095] The near-infrared spectral data of mulberry leaf tea collected in step S2 were processed by multivariate scattering correction. The detailed steps of multivariate scattering correction processing are as follows:

[0096] Calculate the mean of all spectral data as the baseline spectrum;

[0097]

[0098] Perform linear regression on each measured spectrum;

[0099] For each measured spectrum X i A linear model is fitted using the least squares method:

[0100]

[0101] Calculate the corrected spectrum. This is the reference spectrum.

[0102] Use the obtained a i and b i Correcting the original spectrum:

[0103]

[0104] Note: where a i and b i The calculation method is as follows:

[0105] a. Calculate the mean: Calculate the mean for each measured spectrum X. i mean and the mean of the reference spectrum

[0106] b. Calculate the covariance between the measured spectrum and the reference spectrum:

[0107]

[0108] c. Calculate the variance of the reference spectrum:

[0109]

[0110] d. Calculate the regression coefficient a i and b i :

[0111]

[0112] Detailed steps for extracting the external contour features of tea leaf images using the Otsu thresholding algorithm in S3:

[0113] First, find the maximum inter-class variance between the foreground target and the background image as the segmentation threshold;

[0114] For an image I(x,y), let T be the segmentation threshold between the foreground and background, and let y be the proportion of pixels belonging to the foreground (target) to the entire image. The average gray level of the image is denoted as The proportion of background pixels to the total number of pixels in the entire image is: Its average gray level is The overall average gray level of the image is denoted as... The inter-class variance of the image is denoted as g.

[0115] Assuming the background of the image is dark and the image size is M×N, let N0 be the number of pixels with a gray value less than a threshold T, and N1 be the number of pixels with a gray value greater than the threshold T. Then:

[0116]

[0117] The formula for calculating the variance between classes is as follows: Substituting φ into the formula for the variance between classes g, we get the following formula:

[0118]

[0119] Where g is the inter-class variance, The average gray level of the image. The average gray level, The proportion of the target pixels to the total number of pixels in the image. The threshold T, which is the ratio of background pixels to the total number of pixels in the entire image, is obtained by traversing the data to find the threshold T that maximizes the inter-class variance g. This threshold T is then used as the segmentation threshold for the image.

[0120] Then, the image is segmented according to the segmentation threshold to obtain the extracted outer contour features of the tea image.

[0121] In step S3, the RGB values ​​of the image are extracted from the tea color image using an image segmentation algorithm based on color thresholding.

[0122] S301. First, input the preprocessed tea soup image into the program and normalize the grayscale values ​​of each channel of the tea tree color image.

[0123] S302. Convert the tea image from the RGB color space to the HSV color space.

[0124] S303. Set the segmentation thresholds for H, S, and V channels through preset experimental analysis.

[0125] S304. Then, determine whether each pixel in the H, S, and V channels of the image in the HSV color space is within the set threshold range (the upper and lower boundaries need to be set according to the scene). If the result is yes, retain the pixel value at the corresponding position in the input image; otherwise, set it to 0.

[0126] S305. The tea and tea color images in the tea soup image are segmented through the steps S301-S304 above.

[0127] The detailed steps for extracting discrimination information from near-infrared spectral data using a fuzzy weighted discrimination information extraction method to obtain the vector matrix are as follows:

[0128] The spectral data after multivariate scattering correction is defined as sample set D;

[0129] Secondly, define the fuzzy class scatter matrix and the fuzzy class scatter matrix of the sample set;

[0130] Then define the membership function u. ij and cluster center function v i ;

[0131] Substituting the membership function and cluster center function into the fuzzy class intra-class scatter matrix and the fuzzy class inter-class scatter matrix, the objective function of FLDA is derived.

[0132] Then perform eigenvalue decomposition to obtain its eigenvalues ​​and eigenvectors;

[0133] Performing fuzzy linear discriminant analysis (FLDA) yields the eigenvectors corresponding to the largest total number of categories minus one eigenvalue, which form the transformation vector matrix W of FLDA. FLDA .

[0134] Given a sample set D = [x1, x2, ..., xd] of size n×d n [ ], where n represents the number of samples, d represents the sample dimension, and the sample set D can be divided into c categories. Fuzzy within-class scatter matrix S fb And the fuzzy class scatter matrix S fw The definition is as follows:

[0135]

[0136]

[0137] Membership function u ij :

[0138]

[0139] Cluster center function v i :

[0140]

[0141] FLDA's objective function:

[0142]

[0143] Calculations show that:

[0144] .

[0145] The data after FLDA transformation can be obtained using the transformation vector matrix of FLDA.

[0146] The near-infrared spectral data is extracted using a fuzzy weighted discrimination information extraction method to obtain a vector matrix. The vector matrix is ​​then normalized to obtain a range value, which is used as the discrimination threshold of the near-infrared spectral data for the initial judgment of the fermentation degree of mulberry leaf tea. This threshold is usually the near-infrared spectrum of 1325.27 to 1889.49 nm.

[0147] The deep fusion discriminator combines the segmentation threshold of the extracted tea leaf image's outer contour features, the threshold set for each pixel in the H, S, and V channels of the image in the HSV color space based on the image's RGB values, and the near-infrared spectral data discrimination threshold of the initial tea leaf fermentation degree discriminator. This data is then input into an extreme learning machine for training to obtain multiple discrimination thresholds for the deep fusion discriminator. Finally, it comprehensively judges whether the images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea infusion images, and near-infrared spectral data at the standard fermentation degree meet the corresponding thresholds for preset fermentation degree conversion.

[0148] The training process of an extreme learning machine is mainly divided into two stages: random feature mapping and linear parameter solving.

[0149] Phase 1: First, given the number of neurons in the hidden layer, randomly generate the connection weights w between the input layer and the hidden layer, as well as the biases b of the hidden layer neurons. Then, select a nonlinear mapping as the activation function to calculate the hidden layer output matrix H.

[0150] The second stage: To obtain the minimum training error, the Extreme Learning Machine (ELM) solves for β by minimizing the approximate squared difference between the network output Hβ (β being the weights between the hidden and output layers) and the expected output L. The final result is... H * Let H be the Moore-Penrose generalized inverse matrix. These are the weights between the hidden layer and the output layer obtained after training.

[0151] like Figure 2 As shown, a system for judging the fermentation degree of mulberry leaf tea based on image intelligent recognition includes a mulberry leaf tea related data acquisition module 10, a near-infrared spectral data acquisition module 20, an image preprocessing module 30, a near-infrared spectral data preprocessing module 40, a tea leaf image external contour feature extraction module 50, an image RGB value extraction module 60, a near-infrared spectral data identification information extraction module 70, a mulberry leaf tea fermentation degree initial judgment module 80, a deep fusion judgment module 90, and a data storage module 100.

[0152] The mulberry leaf tea related data acquisition module 10 is used to collect images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup of mulberry leaf tea with standard fermentation degree;

[0153] Near-infrared spectral data acquisition module 20 is used to acquire near-infrared spectral data corresponding to images of mulberry leaf tea;

[0154] The image preprocessing module 30 is connected to the mulberry leaf tea related data acquisition module 10. It preprocesses the acquired images of mulberry leaf tea before and after fermentation, images during the fermentation process, and tea soup images by using Gaussian filtering.

[0155] The near-infrared spectral data preprocessing module 40 is communicatively connected to the near-infrared spectral data acquisition module 20, and preprocesses the near-infrared spectral data through multivariate scattering correction;

[0156] The tea image external contour feature extraction module 50 is communicatively connected to the image preprocessing module 30, and extracts the external contour features of the tea image through the Otsu threshold segmentation algorithm;

[0157] The image RGB value extraction module 60 is communicatively connected to the image preprocessing module 30, and uses an image segmentation algorithm based on color threshold to extract the tea color image from the tea soup image;

[0158] The near-infrared spectral data identification information extraction module 70 uses a fuzzy weighted identification information extraction method to extract identification information from the near-infrared spectral data, and obtains a vector matrix composed of feature values;

[0159] The mulberry leaf tea fermentation degree initial judgment module 80 is communicatively connected to the near-infrared spectral data identification information extraction module 70. By normalizing the extracted vector matrix, a range value is obtained. This range value is used as the near-infrared spectral data discrimination threshold of the mulberry leaf tea fermentation degree initial judgment device to make an initial judgment on the fermentation degree of mulberry leaf tea.

[0160] The deep fusion judgment module 90 is communicatively connected to the mulberry leaf tea fermentation degree initial judgment module 80. It combines the segmentation threshold of the extracted tea leaf image's outer contour features, the threshold set for each pixel in the H, S, and V channels of the image under the HSV color space based on the image's RGB values, and the near-infrared spectral data discrimination threshold of the mulberry leaf tea fermentation degree initial judgment device. Then, it inputs the data into an extreme learning machine for training to obtain multiple discrimination thresholds for the deep fusion judgment device. Finally, it comprehensively judges whether the images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and near-infrared spectral data of the standard fermentation degree mulberry leaf tea meet the corresponding thresholds for the preset fermentation degree conversion.

[0161] The data storage module 100 is used to store the near-infrared spectral data discrimination threshold of the initial fermentation degree judge of mulberry leaf tea, the various discrimination thresholds of the deep fusion judge, and the historical mulberry leaf tea related data images and near-infrared spectral data collected by the entire image-based intelligent recognition mulberry leaf tea fermentation degree discrimination system.

[0162] Compared with the prior art, the present invention has the following advantages:

[0163] By effectively acquiring images and tea infusion images during the mulberry leaf tea processing, as well as the corresponding near-infrared spectral data, and then performing data processing steps such as data preprocessing, feature extraction, image RGB value extraction, and near-infrared spectral data identification information extraction, an initial fermentation degree judge and a deep fusion judge for mulberry leaf tea are constructed. Finally, the initial fermentation degree judge is used to determine the fermentation degree of mulberry leaf tea by inputting real-time acquired near-infrared spectral data, while the deep fusion judge is used to determine the fermentation degree of mulberry leaf tea by inputting real-time acquired images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea infusion images, and the corresponding near-infrared spectral data of these images. This achieves the determination of mulberry leaf tea fermentation degree based on image intelligent recognition, and also enables real-time identification of abnormal conditions or abnormal steps in the mulberry leaf tea fermentation process based on images during the mulberry leaf tea processing.

[0164] The foregoing has provided a detailed description of a method and system for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition, as provided in this application. The specific embodiments are described only to aid in understanding the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition, characterized in that, Including the following steps: S1. Collect images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup at the standard fermentation degree, and simultaneously collect the corresponding near-infrared spectral data of the images; S2. Preprocess the collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data at the standard fermentation degree. S3. Extract the external contour features of the tea image and extract the RGB values ​​of the image using the Otsu threshold segmentation algorithm. Then, use the fuzzy weighted discrimination information extraction method to extract discrimination information from the near-infrared spectral data to obtain a vector matrix. S4. Construct an initial fermentation degree judge and a deep fusion judge for mulberry leaf tea, respectively; S5. Input the real-time collected near-infrared spectral data into the mulberry leaf tea fermentation degree initial judgment device for judgment. If the judgment result is lower than the preset mulberry leaf tea fermentation degree, proceed to step S6. If the judgment result is greater than or equal to the preset mulberry leaf tea fermentation degree, proceed to the next round of mulberry leaf tea fermentation degree discrimination process. S6. Input the real-time collected images of mulberry leaf tea before and after fermentation, images during the fermentation process, tea soup images, and corresponding near-infrared spectral data of the standard fermentation degree into the deep fusion judgment device for judgment; if the judgment result is lower than the preset mulberry leaf tea fermentation degree, proceed to step S7; if the judgment result is greater than or equal to the preset mulberry leaf tea fermentation degree, proceed to the next round of mulberry leaf tea fermentation degree judgment. S7. If the judgment result in step S6 is lower than the preset fermentation degree of mulberry leaf tea, then perform image indexing on the images before and after fermentation of mulberry leaf tea with standard fermentation degree, the images during the fermentation process, the tea soup images, and the corresponding near-infrared spectral data to determine the process steps in which the fermentation process malfunctions. The near-infrared spectral data was extracted using a fuzzy weighted discrimination information extraction method to obtain a vector matrix. Then, the vector matrix was normalized to obtain a range value, which was used as the discrimination threshold of the near-infrared spectral data for the initial judgment of the fermentation degree of mulberry leaf tea. The deep fusion discriminator combines the segmentation threshold of the extracted tea image's outer contour features, the threshold set for each pixel in the H, S, and V channels of the image under the HSV color space based on the image's RGB values, and the discrimination threshold of the near-infrared spectral data from the initial tea fermentation degree discriminator. Then, it inputs the data into an extreme learning machine for training to obtain multiple discrimination thresholds for the deep fusion discriminator. The judgment is made by comprehensively analyzing the images of mulberry leaf tea before and after fermentation, the images during the fermentation process, the tea soup images, and the near-infrared spectral data to determine whether they meet the corresponding thresholds for the preset fermentation degree conversion.

2. The method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition according to claim 1, characterized in that, Images of mulberry leaf tea before and after fermentation, during the fermentation process, and infusion were collected and pre-processed using Gaussian filtering for image denoising. Detailed steps for Gaussian filtering image denoising preprocessing are as follows: First, distinguish between images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup to obtain the current processing step of mulberry leaf tea. Based on different processing steps of mulberry leaf tea, different pre-trained two-dimensional Gaussian functions are matched to filter the background image in the mulberry leaf tea image; If the images are of mulberry leaf tea before and after fermentation, or during the fermentation process, then the background image in the mulberry leaf tea image will be filtered directly. If it is a tea infusion image, the tea leaves in the mulberry leaf tea image are filtered as background to obtain a tea infusion color image.

3. The method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition according to claim 2, characterized in that, The near-infrared spectral data of mulberry leaf tea collected in step S2 were processed by multivariate scattering correction. The detailed steps of multivariate scattering correction processing are as follows: Calculate the mean of all spectral data as the baseline spectrum; Perform linear regression on each measured spectrum; Calculate the corrected spectrum.

4. The method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition according to claim 3, characterized in that, Detailed steps for extracting the outer contour features of tea leaf images using the Otsu thresholding algorithm in S3: First, calculate the maximum inter-class variance between the foreground target and the background image as the segmentation threshold; the formula for calculating the inter-class variance is: Where g is the inter-class variance, The average gray level of the image. The average gray level, The proportion of the target pixels to the total number of pixels in the image. The threshold T, which is the ratio of background pixels to the total number of pixels in the entire image, is obtained by traversing the data to find the threshold T with the largest inter-class variance g, and is used as the image segmentation threshold.

5. The method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition according to claim 4, characterized in that, In step S3, the RGB values ​​of the image are extracted from the tea color image using an image segmentation algorithm based on color thresholding. S301. First, input the preprocessed tea soup image into the gray value normalization processing program to normalize the gray value of each channel of the tea soup image. S302. Convert the tea infusion image from the RGB color space to the HSV color space; S303. Set the segmentation thresholds for H, S, and V channels through preset experimental analysis; S304. Then, by determining whether each pixel in the H, S, and V channels of the image in the HSV color space is within the set threshold range, if the result is yes, the pixel value at the corresponding position in the input image is retained; otherwise, it is set to 0. S305. The tea and tea color images in the tea soup image are segmented through the steps S301-S304 above.

6. The method for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition according to claim 5, characterized in that, The detailed steps for extracting discrimination information from near-infrared spectral data using a fuzzy weighted discrimination information extraction method to obtain the vector matrix are as follows: The spectral data after multivariate scattering correction is defined as sample set D; Secondly, define the fuzzy class scatter matrix and the fuzzy class scatter matrix of the sample set; Then define the membership function and the cluster center function; Substituting the membership function and cluster center function into the fuzzy class intra-class scatter matrix and the fuzzy class inter-class scatter matrix, the objective function of FLDA is derived. Then perform eigenvalue decomposition to obtain its eigenvalues ​​and eigenvectors; Perform fuzzy linear discriminant analysis to obtain the transformation vector matrix of FLDA, which is the eigenvector corresponding to the largest total number of categories minus one eigenvalue.

7. A system for determining the fermentation degree of mulberry leaf tea based on image intelligent recognition, characterized in that, It includes a mulberry leaf tea related data acquisition module (10), a near-infrared spectral data acquisition module (20), an image preprocessing module (30), a near-infrared spectral data preprocessing module (40), a tea image external contour feature extraction module (50), an image RGB value extraction module (60), a near-infrared spectral data identification information extraction module (70), a mulberry leaf tea fermentation degree initial judgment module (80), a deep fusion judgment module (90), and a data storage module (100); The mulberry leaf tea related data acquisition module (10) is used to collect images of mulberry leaf tea before and after fermentation, images during the fermentation process, and images of tea soup of standard fermentation degree mulberry leaf tea; Near-infrared spectral data acquisition module (20) is used to acquire near-infrared spectral data corresponding to images of mulberry leaf tea; The image preprocessing module (30) is connected to the mulberry leaf tea related data acquisition module (10) and preprocesses the images of mulberry leaf tea before and after fermentation, images during the fermentation process and tea soup images of the standard fermentation degree through Gaussian filtering; The near-infrared spectral data preprocessing module (40) is connected to the near-infrared spectral data acquisition module (20) and performs preprocessing on the near-infrared spectral data through multivariate scattering correction; The tea image external contour feature extraction module (50) is connected to the image preprocessing module (30) and extracts the tea image external contour features through the Otsu threshold segmentation algorithm; The image RGB value extraction module (60) is connected to the image preprocessing module (30) and uses an image segmentation algorithm based on color threshold to extract the tea color image of the tea soup image; The near-infrared spectral data identification information extraction module (70) uses a fuzzy weighted identification information extraction method to extract identification information from the near-infrared spectral data, and obtains a vector matrix composed of feature values; The initial judgment module (80) of mulberry leaf tea fermentation degree is connected to the near-infrared spectral data identification information extraction module (70). By normalizing the extracted vector matrix, a range value is obtained. This range value is used as the near-infrared spectral data discrimination threshold of the initial judgment module of mulberry leaf tea fermentation degree to make an initial judgment on the fermentation degree of mulberry leaf tea. The deep fusion judgment module (90) is connected to the mulberry leaf tea fermentation degree initial judgment module (80) to combine the segmentation threshold of the extracted tea image external contour features, the threshold set for each pixel in the H, S, and V channels of the image under the HSV color space for judging the RGB values ​​of the image, and the near-infrared spectral data discrimination threshold of the mulberry leaf tea fermentation degree initial judgment module; then input it into the extreme learning machine for training to obtain multiple discrimination thresholds of the deep fusion judgment module; The system comprehensively judges whether the images before and after fermentation, the images during the fermentation process, the tea soup images, and the near-infrared spectral data of mulberry leaf tea with standard fermentation degree meet the corresponding thresholds for preset fermentation degree conversion; the data storage module (100) is used to store the near-infrared spectral data discrimination threshold of the initial fermentation degree judger of mulberry leaf tea, the various discrimination thresholds of the deep fusion judger, and the historical mulberry leaf tea related data images and near-infrared spectral data collected by the entire image-based intelligent recognition mulberry leaf tea fermentation degree discrimination system.

Images