An image-based tea leaf stir-frying quality monitoring method and system

Abstract

The present application belongs to the field of image technology, and particularly relates to a tea leaf stir-frying quality monitoring method and system based on images, which comprises the following steps: collecting tea leaf stir-frying time sequence hyperspectral images and extracting real-time images; calculating two characteristic quantities of each pixel, i.e. a spectral channel reflectivity ratio representing the conversion of chlorophyll to de-magnesium chlorophyll and a local anisotropy descriptor of water loss shrinkage based on the eigenvalue of a structure tensor; adjusting the cumulative distribution function of the second characteristic quantity with the statistical mean of the first characteristic quantity, determining the asymmetric correlation quantitative boundary of the two by combining the derivative, constructing a weighted two-dimensional joint histogram, and making the neighborhood scale negatively correlated with the second characteristic quantity of the center pixel, and making the weight fusion space distance and the included angle between the gradient direction; mapping the reference histogram into a low-dimensional standard manifold through local linear embedding, calculating the local density of the real-time projection point as the stability confidence, and combining the weighted geodesic distance and the weighted moving rate of the historical and real-time projection points to comprehensively determine the tea leaf quality after adjusting the weight.

Description

Technical Field

[0001] This invention relates to the field of image technology, and in particular to an image-based method and system for monitoring the quality of tea leaves during stir-frying. Background Technology

[0002] The quality control of tea leaves during the stir-frying process relies heavily on the tea master's personal experience, primarily through visual observation of color changes, tactile perception of moisture loss, and olfactory assessment of aroma development to determine the progression and termination of the stir-frying process. This manual sensory evaluation method is not only highly subjective and subject to individual differences, leading to inconsistent quality between different batches of tea, but also fails to meet the demands of modern tea industry's large-scale, standardized, and automated production. During the stir-frying process, the tea leaves are under continuous high-temperature agitation, resulting in complex internal changes. These changes include macroscopic and microscopic shrinkage caused by moisture loss, as well as the degradation and synthesis of deeper chemical substances, such as the conversion of chlorophyll to pheophytin.

[0003] Currently, industrial visual monitoring can only detect changes in the RGB color and shape of the surface, and cannot penetrate and quantitatively describe the internal chemical composition. However, hyperspectral imaging technology, with its "image and spectrum integration" advantage, can simultaneously acquire the spatial texture structure of the target and the molecular spectral information of the internal substances. Monitoring methods based on hyperspectral technology often extract feature bands to calculate conventional statistical indicators or construct image histograms, which are then used as inputs for classification models.

[0004] Chinese patent document CN114878567B discloses an online detection device and method for the appearance quality of tea leaves. The method collects tea leaf images, extracts RGB, HSV, and Lab color space features, and establishes a gray-level histogram to calculate the second moment, gray-level mean, consistency, and entropy as texture features. Combined with physical features obtained by infrared detection, the method uses principal component analysis for dimensionality reduction and inputs the data into a neural network to predict moisture content and bud-leaf ratio.

[0005] The existing technology uses a fixed grayscale histogram for texture feature extraction, independently quantifies color and texture features, performs linear dimensionality reduction through principal component analysis, and finally performs classification prediction based on static feature vectors. This feature extraction and quantization method ignores the asymmetric correlation between the macroscopic stage of chemical changes during tea stir-frying and the distribution of microstructural features. The fixed-size statistical window cannot adapt to the dynamic changes in local texture scale caused by tea withering due to water loss. The linear principal component analysis dimensionality reduction cannot capture the nonlinear manifold evolution law of time series images in high-dimensional feature space. The fragmented processing of chemical transformation and morphological shrinkage leads to feature expression distortion. Fixed neighborhood statistics miss the spatial structure information of local texture. Linear dimensionality reduction and static judgment cannot track the dynamic evolution trend of tea quality along the manifold trajectory. Ultimately, this type of method has insufficient accuracy in monitoring the tea quality status in actual industrial stir-frying environments, and there is lag and error in the determination of the stir-frying endpoint, making it difficult to meet the requirements of high-precision and real-time automated tea production. Summary of the Invention

[0006] To address the technical problems of insufficient accuracy in monitoring the quality status of tea leaves and the lag and error in determining the end point of stir-frying in existing technologies, this invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides an image-based method for monitoring the quality of tea leaves during stir-frying, comprising:

[0008] S1. Obtain the time series hyperspectral image of the stir-frying process, take the current frame as the real-time hyperspectral image, and calculate the first and second feature quantities of each pixel in the hyperspectral image; calculate the mean of the first feature quantity of each pixel as a macroscopic indicator representing the chemical change; transform the cumulative distribution function of the second feature quantity based on the indicator, and determine the quantization boundary of the second feature quantity according to the first and second derivatives of the transformed function; the quantization boundary of the first feature quantity is determined by the derivative of its own cumulative distribution function, thereby completing the asymmetric correlation quantization. S2. Construct a weighted two-dimensional joint histogram. For each central pixel, count the quantized feature pairs between it and each pixel in its neighborhood. The neighborhood scale is negatively correlated with the second feature value of the central pixel. Multiply the count value of each feature pair by a weighting coefficient with respect to the angle between the spatial distance and the gradient direction. Construct a low-dimensional standard manifold and mapping function from the reference histogram using local linear embedding. S3. Project the histogram corresponding to the time series hyperspectral image onto the manifold and calculate the stability confidence. Divide the projection length of the movement vector from the historical projection point to the real-time projection point in the tangent direction of the target state point by the corresponding time interval to obtain the weighted movement rate. Based on the confidence, assign weights to the real-time observation distance and the predicted distance. Combine the weighted geodesic distance between the real-time projection point, the historical projection point and the target state point, and the weighted movement rate to comprehensively determine the tea quality.

[0009] This invention reflects the comprehensive change patterns of tea leaves during the stir-frying process by combining spectral features representing chemical transformation with structural features representing dehydration and shrinkage. It utilizes macroscopic chemical indicators to adjust feature transformations and completes asymmetric correlation quantification, detecting details of data distribution and strengthening the correlation between physicochemical changes. When constructing a two-dimensional joint histogram, it adjusts the statistical neighborhood scale based on pixel feature differences and incorporates spatial and gradient direction weights, improving the precision and anti-interference ability of feature representation. It maps high-dimensional histogram features to a low-dimensional manifold space, calculates confidence levels through local density, and optimizes the weight parameters for observation and prediction in real time by combining the rate of change and multidimensional distance information, tracking state change trends and improving the accuracy of tea quality monitoring and the reliability of evaluation results.

[0010] Preferably, the first feature is a specific spectral channel reflectance ratio representing the conversion of chlorophyll to pheophytin, including: extracting the real value of the first spectral reflectance of the current pixel in the 680nm wavelength channel and the real value of the second spectral reflectance in the 550nm wavelength channel, and dividing the real value of the first spectral reflectance by the real value of the second spectral reflectance to obtain the quotient as the specific spectral channel reflectance ratio.

[0011] The above-mentioned technical solution, through this division operation, not only reduces some of the systematic errors of background lighting, but also enhances the sensitivity to the microscopic transformation of chemical components, enabling it to accurately reflect the physicochemical change trend of chlorophyll being degraded by heat and transformed into pheophytin during the tea-frying process.

[0012] Preferably, the second feature is a local anisotropic descriptor representing the degree of water loss and shrinkage calculated based on the eigenvalues ​​of the structural tensor, including: obtaining the maximum and minimum eigenvalues ​​of the structural tensor within the local neighborhood window of the current pixel; dividing the difference between the maximum and minimum eigenvalues ​​by the sum of the maximum and minimum eigenvalues, and using the result as the local anisotropic descriptor.

[0013] The descriptor in the above technical solution can quantitatively assess the degree of shrinkage of tea leaves during the stir-frying and dehydration process. Its value can accurately represent the change pattern of tea leaf morphology from a smooth surface to a highly wrinkled state.

[0014] Preferably, obtaining the maximum and minimum eigenvalues ​​of the structure tensor within the local neighborhood window of the current pixel includes: selecting a near-infrared channel with a center wavelength of 750nm as a grayscale image, calculating the gradient components of the grayscale image in the horizontal and vertical directions; constructing a 3×3 pixel local neighborhood window centered on the current pixel, using a Gaussian smoothing kernel to perform weighted filtering on the gradient product to construct a structure tensor matrix; solving the characteristic equation of the structure tensor matrix to obtain the maximum and minimum eigenvalues.

[0015] The aforementioned technical solution can accurately locate the inflection point of the distribution state change, allowing the quantization boundary to be set at the low-density valley of the feature distribution. This enables high-frequency clustered homogeneous features to be grouped into the same quantization interval, achieving accurate quantization division.

[0016] Preferably, the quantization boundary of the first feature is determined by the derivative of its own cumulative distribution function, thereby completing asymmetric correlation quantization, including: sorting the first feature values ​​of all pixels, fitting a smooth probability density function and integrating to obtain the smoothed cumulative distribution function of the first feature; the quantization boundary of the first feature is determined by points that simultaneously satisfy the following conditions: the second derivative of the smoothed cumulative distribution function of the first feature with respect to the first feature is equal to zero, and the first derivative of the smoothed cumulative distribution function with respect to the first feature is less than a preset derivative judgment threshold; according to the determined quantization interval boundary, the first feature is mapped to discrete quantization levels to complete quantization.

[0017] The window adjustment mechanism described above allows the algorithm to use a larger statistical window in flat areas of the tea leaves, thus exhibiting higher noise resistance. Conversely, a smaller statistical window is used in highly wrinkled, densely detailed areas, which helps to better preserve edge features.

[0018] Preferably, the neighborhood scale is negatively correlated with the second feature value of the center pixel, including: multiplying the adjustment coefficient by the value of the second feature value of the center pixel to obtain a first product; adding 1 to the first product to obtain a denominator value; and dividing the basic neighborhood radius by the denominator value to obtain the result as the radius of the neighborhood scale.

[0019] Preferably, the weighting coefficients for the angle between spatial distance and gradient direction include: calculating the cosine of the angle between the feature gradient directions of the center pixel and neighboring pixels, and taking the absolute value of the cosine; dividing the spatial distance between the center pixel and neighboring pixels by a preset spatial distance normalization constant, taking the negative value as the exponent term, and calculating the exponential function value with the natural logarithm as the base and the exponent term as the exponent.

[0020] Preferably, the calculation of stability confidence includes: calculating the distance between the real-time projection point and each reference projection point; counting the number of reference projection points whose distance is less than or equal to a preset distance threshold; dividing the number by the total number of reference projection points, and using the result as the local density, which is the stability confidence.

[0021] Preferably, the normalized local density is used to obtain the confidence weight coefficient; the weighted moving rate is multiplied by the time interval to obtain the actual displacement; the weighted geodesic distance between the historical projection point and the target state point is subtracted from the actual displacement, and the absolute value of the difference is taken as the actual predicted distance; the weighted geodesic distance between the real-time projection point and the target state point is dimensionless to obtain the first distance term, and the actual predicted distance is dimensionless to obtain the predicted distance term; the confidence weight coefficient is subtracted from 1 to obtain the prediction weight coefficient; the comprehensive evaluation score is calculated, including: multiplying the confidence weight coefficient by the first distance term to obtain the first value, and multiplying the prediction weight coefficient by the predicted distance term to obtain the second value; the sum of the first value and the second value is negative to obtain the total exponent term; the result with the natural logarithm as the base and the total exponent term as the exponent is calculated as the comprehensive evaluation score; the larger the comprehensive evaluation score, the closer the tea quality is to the target state.

[0022] This invention reflects the comprehensive change pattern of tea leaves during the stir-frying process by combining spectral features representing chemical transformation (chlorophyll degradation) with structural features representing water loss and shrinkage. By using macroscopic chemical indicators to adjust feature transformation and complete asymmetric correlation quantification, the details of data distribution are effectively detected, and the correlation between internal chemical changes and external physical morphological changes is strengthened.

[0023] Furthermore, when constructing the two-dimensional joint histogram, the algorithm innovatively dynamically adjusts the statistical neighborhood scale based on pixel feature differences and incorporates dual weights of spatial distance and gradient direction. This enables the algorithm to adapt to local texture scale changes caused by the withering of tea leaves due to water loss, significantly enhancing its noise and interference resistance in complex industrial environments while preserving details.

[0024] Simultaneously, the high-dimensional histogram features are mapped to the low-dimensional standard manifold space, and the confidence level of the observation data is obtained by calculating the local density. Combined with the weighted movement rate and multi-dimensional distance information of the tea leaves evolving along the manifold trajectory, the weight parameters of the current observation and historical prediction are optimized in real time, accurately tracking the trend of state change, and effectively overcoming the judgment lag and error brought about by the traditional static and linear models.

[0025] Secondly, the present invention provides an image-based tea stir-frying quality monitoring system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned image-based tea stir-frying quality monitoring method is implemented.

[0026] By adopting the above technical solution, a computer program for monitoring the quality of tea stir-frying based on images is generated and stored in a memory so that it can be loaded and executed by a processor. This allows for the creation of a terminal device based on the memory and processor, making it convenient to use. Attached Figure Description

[0027] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 This is a flowchart of an image-based method for monitoring the quality of tea leaves during stir-frying. Figure 2 This is a schematic diagram illustrating the change of characteristic quantities over time; Figure 3 This is a schematic diagram of the weight coefficient distribution. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] S1. Acquire time-series hyperspectral images and perform asymmetric correlation quantization.

[0031] Specifically, multiple consecutive frames of hyperspectral images of the tea-stirring process are acquired. The current frame of hyperspectral image is used as the real-time hyperspectral image, and the first and second feature quantities of each pixel are calculated. The mean of the first feature quantity of each pixel is calculated as a macroscopic indicator representing the chemical change. Based on the indicator, the cumulative distribution function of the second feature quantity is transformed, and the quantization boundary of the second feature quantity is determined according to the first and second derivatives of the transformed function. The quantization boundary of the first feature quantity is determined by the derivative of its own cumulative distribution function, thereby completing the asymmetric correlation quantization.

[0032] A hyperspectral camera was used to continuously acquire video streams of tea leaves being stir-fried, and the hyperspectral data cube was read. The current frame was set as a real-time hyperspectral image. The spectral reflectance matrices at wavelengths of 550 nm and 680 nm were extracted, and the ratio matrix between the two was calculated as the first feature. The first principal component image after dimensionality reduction by principal component analysis was extracted, and the horizontal and vertical gradients were calculated. The squares of the horizontal and vertical gradients, as well as the product of the horizontal and vertical gradients, were calculated, and smoothing filtering was performed to construct the structure tensor matrix.

[0033] Calculate two eigenvalues ​​of the structure tensor for each pixel. Obtain the local anisotropy descriptor based on the ratio of the difference between the larger and smaller eigenvalues ​​to the sum of the two eigenvalues, which serves as the second eigenvalue. Calculate the arithmetic mean of all pixel values ​​of the first eigenvalue for the entire image as the macroscopic index. Obtain the cumulative distribution function of the second eigenvalue. Use the value of the cumulative distribution function as the base and the macroscopic index as the exponent to perform a power operation to complete the transformation.

[0034] A cubic spline fit is performed on the transformed cumulative distribution function, and the first and second derivatives are calculated. The local extrema of the first derivative and the zero-crossing points of the second derivative are selected as the quantization boundaries of the second feature. The original cumulative distribution function of the first feature is smoothed using the same cubic spline fit method. The extrema of the first derivative and the zero-crossing points of the second derivative of the smoothed cumulative distribution function of the first feature are calculated as the quantization boundaries of the first feature. Based on the obtained boundaries, the first and second feature values ​​of all pixels are mapped to discrete integer level indices, respectively, to complete asymmetric correlation quantization.

[0035] In an optional embodiment, the first feature is the ratio of reflectance of a specific spectral channel in the conversion of chlorophyll to pheophytin. That is, the first feature is obtained by extracting the real value of the first spectral reflectance of the current pixel in the 680nm wavelength channel and the real value of the second spectral reflectance in the 550nm wavelength channel, and calculating the ratio of the real value of the first spectral reflectance to the real value of the second spectral reflectance as the ratio of reflectance of a specific spectral channel. The reflectance ratio is the first feature.

[0036] It should be noted that, in order to avoid the objective consequence of inconsistent surface moisture loss and deep chemical degradation rates caused by continuous high-temperature stirring of tea leaves during frying, which could lead to serious disconnection and miss of the target in the judgment of single visual color or static texture, this step calculates the first characteristic quantity representing internal chemical changes and the second characteristic quantity representing external water loss and shrinkage, and performs asymmetric correlation quantification operation.

[0037] The process of obtaining the first feature involves extracting data from the hyperspectral image in a specific band; performing radiometric calibration and black-and-white correction on the real-time hyperspectral image to reduce the effects of uneven illumination and dark current noise, so that the reflectance value is between 0 and 1.

[0038] It should be noted that the specific bands include the 680nm visible light band, which characterizes the chlorophyll absorption characteristics of tea leaves, and the 550nm green peak band, which serves as a reference for reflectance comparison. After acquiring raw data with a spectral resolution of not less than 5nm across the entire spectrum using a hyperspectral camera, the first characteristic quantity is constructed using the ratio of the spectral reflectance of the two specific bands mentioned above.

[0039] From the corrected hyperspectral data cube, for each pixel, the reflectance corresponding to the 680nm wavelength spectral channel is extracted as the first spectral reflectance real value, and the reflectance corresponding to the 550nm wavelength spectral channel is extracted as the second spectral reflectance real value.

[0040] In actual calculations, if the actual reflectance at 680nm is 0.12 and the actual reflectance at 550nm is 0.35, then the calculated value of the first characteristic quantity is 0.343.

[0041] As chlorophyll is degraded by heat and converted into pheophytin during the tea-frying process, the absorption at 680nm weakens, leading to an increase in reflectivity. This ratio will gradually increase. This division operation not only reduces some of the systematic errors of the background illumination, but also enhances the sensitivity to the microscopic transformation of chemical components.

[0042] In an optional embodiment, the second feature is a local anisotropic descriptor representing the degree of water loss and shrinkage calculated based on the eigenvalues ​​of the structural tensor, including: obtaining the maximum and minimum eigenvalues ​​of the structural tensor within the local neighborhood window of the current pixel; dividing the difference between the maximum and minimum eigenvalues ​​by the sum of the maximum and minimum eigenvalues, and using the result as the local anisotropic descriptor.

[0043] The extraction of the second feature quantity relies on the structural tensor analysis of the local gradient of the image, which is used to quantitatively evaluate the degree of shrinkage of tea leaves during the dehydration process of stir-frying. The near-infrared channel near the center wavelength of 750nm is selected as the grayscale image, and the gradient components of the grayscale image in the horizontal and vertical directions are calculated. A 3×3 pixel local neighborhood window is constructed with the current pixel as the center, and the gradient product is weighted and filtered using a Gaussian smoothing kernel to construct a 2×2 structural tensor matrix.

[0044] Furthermore, the characteristic equation of the structural tensor matrix is ​​solved to obtain the largest eigenvalue. with minimum eigenvalue For example, when a pixel is located on a smooth tea leaf surface, both feature values ​​are small and close. Assuming... , The calculated descriptor V is 0.022, indicating local isotropy; however, when the pixel is located at the edge of a wrinkle, Increase, assuming , At this point, V is 0.854. The descriptor V ranges from 0 to 1, representing the change in tea leaf morphology from smooth to highly wrinkled. For example... Figure 2 As shown, the changing trends of the first and second characteristic quantities are illustrated throughout the entire process of tea leaf stir-frying.

[0045] In an optional embodiment, the quantization boundary of the first feature is determined by the derivative of its own cumulative distribution function, thereby completing asymmetric correlation quantization, including: sorting the first feature values ​​of all pixels, fitting a smooth probability density function and integrating to obtain the smoothed cumulative distribution function of the first feature; the quantization interval boundary of the first feature is determined by points that simultaneously satisfy the following conditions: the second derivative of the smoothed cumulative distribution function of the first feature with respect to the first feature is equal to zero, and the first derivative of the smoothed cumulative distribution function with respect to the first feature is less than a preset derivative judgment threshold; according to the determined quantization interval boundary, the first feature is mapped to discrete quantization levels to complete quantization.

[0046] The division of the quantization interval aims to map the continuous first feature quantity to discrete quantization levels. The process includes locating the inflection point of the distribution state change by analyzing the feature distribution function, sorting the first feature quantity values ​​of all pixels, fitting a smooth probability density function using the Gaussian kernel density estimation method, and obtaining a smooth and continuous cumulative distribution function F by integral calculation.

[0047] The first derivative (probability density) and second derivative (rate of change of density) of F are calculated using the central difference algorithm. When determining the quantization boundary, the range of the first feature is traversed to extract the coordinates of the inflection point where the second derivative crosses the sign threshold. Simultaneously, a first derivative constraint is applied to this inflection point. A derivative determination threshold is assumed. If the second derivative is 0 and the first derivative is 0.12 (less than 0.15) at x=0.45, then 0.45 is used as a quantization boundary point.

[0048] This condition allows the quantization boundary to be set at the low-density valley of the feature distribution, which will group high-frequency clustered homogeneous features into the same quantization interval, thus achieving asymmetric correlation quantization.

[0049] It should be added that the acquisition of the above-mentioned derivative judgment threshold includes: collecting a large number of high-spectral test video streams of the entire life cycle of tea from different batches in a standard constant temperature stir-frying equipment in advance; extracting the original calculation data of the cumulative distribution function of the first feature quantity at the critical point of the transition from the smooth unfolding stage to the extreme withering and shrinking stage; obtaining the set of first derivatives in the critical point region through central difference discretization processing; using median filtering to remove abnormal jump peaks caused by the sensor's background thermal noise; calculating the lower quartile value of the remaining effective first derivative sequence as the set derivative judgment threshold; and solidifying it into the non-volatile memory of the lower-level microcontroller as a normalized comparison benchmark.

[0050] S2. Construct a weighted two-dimensional joint histogram and perform manifold mapping.

[0051] Specifically, a weighted two-dimensional joint histogram is constructed. For each central pixel, the quantized feature pairs formed by the pixel and its neighboring pixels are counted. The neighborhood scale is negatively correlated with the second feature value of the central pixel. The count value of each feature pair is multiplied by a weight coefficient, which is a function of the spatial distance and the gradient direction angle. The reference histogram is used as a high-dimensional vector, and a low-dimensional standard manifold and mapping function are constructed using local linear embedding.

[0052] Initialize a two-dimensional matrix of all zeros as a two-dimensional joint histogram matrix, with the number of rows and columns equal to the quantization levels of the first and second features, respectively. Traverse each center pixel in the image, subtract the product of the adjustment coefficient and the value of the second feature of the center pixel from the preset maximum window size, and round down to determine the side length of the square neighborhood window of the center pixel. Extract the quantized first feature of the center pixel and the quantized second feature of each neighboring pixel in the neighborhood to form a feature pair.

[0053] Calculate the Euclidean spatial distance from the adjacent pixel to the center pixel, and calculate the angle between the gradient vectors of the center pixel and the adjacent pixel; calculate the exponential function value of the negative square of the spatial distance, and multiply it by the absolute value of the cosine of the angle, as the weight coefficient of the adjacent pixel; accumulate the frequency of the feature pair at the corresponding coordinate position in the two-dimensional joint histogram with the above weight coefficient, and after completing the full image traversal, flatten the two-dimensional joint histogram into a one-dimensional high-dimensional vector.

[0054] Collect a series of hyperspectral images at various standard time points during the standard roasting process, repeat the above steps, and construct multiple reference histograms. Figure 1 The set of vectors is defined, the number of nearest neighbors and the target dimension after dimensionality reduction are set, and the set of reference histograms is input into the training module of the local linear embedding algorithm for training. By calculating the local reconstruction weight matrix and solving its eigenvectors, a low-dimensional standard manifold space is constructed, as well as a mapping function from the high-dimensional feature space to the low-dimensional manifold space.

[0055] In an optional embodiment, the neighborhood scale is negatively correlated with the second feature value of the center pixel, including: multiplying the adjustment coefficient by the value of the second feature value of the center pixel to obtain a first product; adding 1 to the first product to obtain a denominator value; and dividing the basic neighborhood radius by the denominator value to obtain the result as the radius of the neighborhood scale.

[0056] The neighborhood is constructed to dynamically adjust the spatial statistical range based on the local texture complexity of the tea leaves when calculating the two-dimensional joint histogram; the global control parameters of the algorithm are initialized, assuming the radius of the basic neighborhood. Set to 10 pixels, adjust coefficient k to 3, and the second feature value of the center pixel. As an input variable, its value is between 0 and 1.

[0057] In actual calculations, if the center pixel is located in a smooth area of ​​the tea leaf that has not shrunk, such as The calculated neighborhood scale R is 8.7. After rounding, a larger statistical window with a radius of 9 is used to detect more macroscopic smooth features. If the point is located in a highly wrinkled, textured, complex region, such as... The calculated R is 2.9, which is rounded to the nearest whole number and then a smaller statistical window with a radius of 3 is used. This window adjustment mechanism enables the device to have high noise resistance in flat areas and to better preserve edge features in areas with dense details.

[0058] Furthermore, the calibration method for the basic neighborhood radius and adjustment coefficient specifically includes: obtaining the nominal value of the physical epidermal texture span of a preset type of fresh leaf in a fully expanded state without water loss; combining the focal length of the optical lens of the hyperspectral camera and the physical size of the target pixel, mapping the nominal value of the physical epidermal texture span to a pixel scale span, and fixing the integer value greater than and closest to half of the pixel scale span as the basic neighborhood radius; simultaneously, using the hyperspectral camera to photograph a standard resolution test chart, extracting the pixel radius of the spot diffusion circle of the optical point spread function of the underlying system; setting the boundary value of the adjustment coefficient so that the denominator value of the basic neighborhood radius divided by the adjustment coefficient plus one is strictly greater than the pixel radius of the spot diffusion circle. Using this hardware physical limit as a constraint condition, the sub-pixel background noise caused by the underlying optical diffraction is avoided when using a small window in a locally extremely wrinkled area.

[0059] In an optional embodiment, the weighting coefficient is a function of the angle between the spatial distance and the gradient direction, including: calculating the cosine value of the angle between the feature gradient directions of the center pixel and the neighboring pixels, and taking the absolute value of the cosine value; dividing the spatial distance between the center pixel and the neighboring pixels by a preset spatial distance normalization constant, and taking the negative value as the exponent term, and calculating the exponential function value with the natural logarithm as the base and the exponent term as the exponent.

[0060] The weighting coefficients used in constructing the two-dimensional joint histogram are used to quantify the contribution of neighboring pixels to the feature distribution of the center pixel, integrating both spatial and structural constraints.

[0061] In the spatial dimension, the Euclidean distance d between neighboring pixels and the center pixel is calculated, for example using a spatial distance normalization constant of 3. The exponential decay function makes the spatial weight of points that are closer together increase.

[0062] Furthermore, the calibration method for the spatial distance normalization constant specifically includes: on a standard constant temperature frying platform, using a macro infrared thermal imager to synchronously acquire time-series data of the heat distribution on the surface of the benchmark tea leaves; extracting and locating the edge of the abrupt temperature gradient change caused by the local water loss and shrinkage of the tea leaves; fitting a Gaussian curve to the temperature decay distribution curve radiating outward from the edge, and extracting the standard deviation of the Gaussian distribution curve; converting the physical decay distance corresponding to the standard deviation into the benchmark pixel distance of the hyperspectral image, and solidifying the pixel distance value into the operation register of the underlying field-programmable gate array (FPGA) as the spatial distance normalization constant, so that the decay rate of the histogram spatial weight matches the real spatial diffusion range of the physical thermodynamics of the tea leaves and the moisture evaporation gradient.

[0063] In terms of structure, the feature gradient vectors of two pixels are obtained using the Sobel operator, and the angle between the vectors is calculated. The absolute value of the cosine is used as the structural similarity weight, giving higher weight to pixels with gradient directions parallel to each other. For example, if a neighboring point is d=2 from the center point, and the gradient angle is... If the value is °, then the spatial weight is 0.513, the structural weight is 0.866, and the overall weight coefficient W for this point is 0.444, thus strengthening the symbiotic relationship of local isomorphic features in histogram statistics. For example... Figure 3 As shown, the decay distribution of the weight coefficient with spatial distance is presented under different gradient direction angles. The weight coefficient is obtained by multiplying the absolute value of the cosine value with the value of the exponential function.

[0064] S3. Calculate the confidence level and moving rate, and determine the quality of the tea.

[0065] Specifically, the histograms corresponding to multiple consecutive hyperspectral images are projected onto a low-dimensional standard manifold. Based on the local density around the real-time projection point corresponding to the real-time hyperspectral image, the stability confidence is obtained. The projection length of the movement vector between the historical projection point generated in the previous frame and the real-time projection point is calculated in the tangent direction of the target state point. The ratio of the projection length to the corresponding time interval is used as the weighted movement rate with positive and negative directional attributes. Based on the confidence, the weights of the real-time observation distance and the predicted distance are assigned. Combining the weighted geodesic distance between the real-time projection point, the historical projection point and the target state point, as well as the weighted movement rate, the tea quality is comprehensively determined.

[0066] The one-dimensional histogram vectors of the flattened real-time hyperspectral images and historical hyperspectral images are projected onto a low-dimensional standard manifold to obtain the coordinates of the real-time projection points and historical projection points, respectively. The kernel density of all standard reference projection points in the manifold space is estimated using the Gaussian kernel function. The corresponding probability density value is calculated by inputting the coordinates of the real-time projection points and normalized to the interval of 0 to 1 using the Sigmoid function as the stability confidence.

[0067] Calculate the movement vector from the historical projection point to the real-time projection point, and the tangent direction vector from the real-time projection point to the target optimal cooking state point; calculate the projection length of the movement vector on the tangent direction vector, divide the projection length by the time difference between the historical time and the current real-time time to obtain the weighted movement rate, and construct the K-nearest neighbor connected graph of all projection points on the manifold space.

[0068] Calculate the weighted geodesic distance from the real-time projection point and historical projection point to the target state point on the connected graph, respectively; use the stability confidence as the weight coefficient of the real-time observation distance, and use the difference between 1 and the stability confidence as the prediction weight coefficient; subtract the product of the weighted movement rate and the time difference from the weighted geodesic distance from the historical projection point to the target state point to obtain the predicted distance.

[0069] The weighted geodesic distance from the real-time projection point to the target state point is used as the real-time observation distance. The real-time observation distance is multiplied by its weight coefficient and added to the predicted distance multiplied by its weight coefficient to obtain the quality judgment deviation score. The deviation score is compared with the preset quality threshold. When the score is less than the threshold, it is determined that the tea frying quality meets the target requirements. Otherwise, the control system is triggered to continue frying.

[0070] Furthermore, the calibration method for the preset quality threshold specifically includes: during the calibration stage of the standard frying process, a high-precision trace moisture analyzer is connected in parallel to the frying equipment to simultaneously acquire the dynamic change time series data of the dry basis moisture content of the tea; the hyperspectral time series data corresponding to the moment when the dry basis moisture content drops to the predetermined physical process target value is extracted; the hyperspectral time series data at this critical moment is projected onto a low-dimensional standard manifold space to calculate and obtain the benchmark deviation score set; in order to filter out the low-frequency disturbances introduced by the dark current drift of the underlying image sensor, the mathematical expectation value of the benchmark deviation score set is calculated, and the score tolerance corresponding to three times the system background noise variance is subtracted, and the final obtained physical baseline value is used as the preset quality threshold and burned into the non-volatile memory unit of the control motherboard.

[0071] In an optional embodiment, calculating the stability confidence score includes: calculating the distance between the real-time projection point and each reference projection point; counting the number of reference projection points whose distance is less than or equal to a preset distance threshold; dividing the number by the total number of reference projection points, and using the result as the local density, which is the stability confidence score.

[0072] It should be added that the acquisition of the above-mentioned preset distance threshold includes: collecting multiple batches of benchmark tea sample sequences that meet the optimal target roasting state in a standard workshop environment, mapping them as a whole to construct a reference projection point cloud model in a full-rank state in a low-dimensional standard manifold space, using the distance calculation logic unit of the underlying hardware microprocessor to traverse and extract the local Euclidean distance set between all adjacent reference projection points in the point cloud network, performing normal fitting on the physical distance set to construct the spatial distribution density function of the benchmark sample, taking the 95th percentile value of the statistical distribution curve as the absolute safe comparison boundary, and simultaneously superimposing the spatial discrete distortion tolerance calibrated by the optical acquisition equipment hardware at the factory for derating and conversion, and finally solidifying the obtained physical boundary limit value as the preset distance threshold and directly sending it to the register of the edge computing gateway.

[0073] Stability confidence is used to measure the stability and reliability of real-time acquired hyperspectral image features in a low-dimensional manifold space.

[0074] Obtain a standard point cloud model constructed by mapping a pre-established reference hyperspectral sequence to a low-dimensional manifold. For example, the model contains N=500 reference projection points representing different stir-frying states. Use the Euclidean distance algorithm to traverse and calculate the distance between the current real-time projection point and every reference projection point in the manifold space. Assuming the preset distance threshold is 0.15, if If the value is less than or equal to 0.15, the condition indicator function returns 1; otherwise, it returns 0. For example, after traversing and comparing, it is found that there are 45 reference projection points within the distance range of the real-time projection point that meet the condition. The sum of these points is 45, and the calculated local density is 0.09. This local density can represent the stability confidence of the current observation data. The higher the local density, the better the pattern of the current real-time features matches.

[0075] In an optional embodiment, the quality of tea is comprehensively determined by combining the weighted geodesic distance and weighted movement rate of the real-time projection point, historical projection point, and target state point, including: Normalize the local density to obtain the confidence weight coefficient; Multiply the weighted moving rate by the corresponding time interval to obtain the actual displacement; subtract the actual displacement from the weighted geodesic distance between the historical projection point and the target state point, and take the absolute value of the difference as the actual predicted distance. The first distance term is obtained by dimensionless processing of the weighted geodesic distance between the real-time projection point and the target state point, and the predicted distance term is obtained by uniformly dimensionless processing of the actual predicted distance. Subtract the confidence weight coefficient from 1 to obtain the prediction weight coefficient; The comprehensive evaluation score is calculated as follows: multiply the confidence weight coefficient by the first distance term to obtain the first value, multiply the prediction weight coefficient by the prediction distance term to obtain the second value; add the first value and the second value and take the negative value as the total index term; calculate the index result with the natural logarithm base as the base and the total index term as the index as the comprehensive evaluation score. The higher the overall evaluation score, the closer the tea quality is to the target state.

[0076] The calculation of the comprehensive evaluation score is a closed-loop decision-making process that integrates current observations and historical predictions; the confidence weight coefficient is obtained by normalizing the local density. Assuming Then predict the weighting coefficients The value is 0.3. Based on this, to ensure the consistency of dimensions in distance and velocity calculations, prediction calculations are performed on a physical scale. For example, the historical weighted geodesic distance is 35 units, and the actual displacement (the product of the weighted movement rate and the time interval) is 10 units, resulting in an actual predicted distance of 25 units. A uniform dimensionless processing is performed; for example, the first distance term after mapping... Predicting distance term =0.25, this compensation term represents the remaining distance of the current state based on historical inertia prediction.

[0077] Substituting into the evaluation formula, the index part is calculated as -0.25, and the comprehensive evaluation score is 0.7788. This mechanism tends to observe in real time under high confidence and relies on historical trajectory prediction under low confidence, which can smoothly monitor changes in tea quality under strong noise interference.

[0078] The ablation experiment was conducted using a self-constructed hyperspectral image dataset of spring tea stir-frying. The dataset contains 1500 hyperspectral data cube samples, covering all stir-frying stages from fresh leaves to complete drying and dehydration. 1200 samples were used to construct a reference hyperspectral sequence and a standard point cloud model in manifold space, while the remaining 300 samples served as a real-time test set to verify and evaluate accuracy. The baseline method used only basic spectral features and a fixed-window histogram for static threshold determination. Three variant schemes were then compared, incorporating a neighborhood and dual-weight constraint joint histogram module, and a local density confidence fusion comprehensive evaluation module. The evaluation metrics were the classification accuracy and root mean square error of continuous scoring for tea quality status assessment. The experimental results are as follows.

[0079] The baseline method achieved an accuracy of 81.5% and a root mean square error of 0.145. With only the addition of the neighborhood and dual-weight constraint joint histogram module, the evaluation accuracy is 86.4% and the root mean square error is 0.112. With only the addition of the local density confidence fusion comprehensive evaluation module, the evaluation accuracy is 88.2% and the root mean square error is 0.103. Simultaneously, by utilizing spatial statistical features and a closed-loop decision-making module, the evaluation accuracy was 94.6%, with a root mean square error of 0.072.

[0080] This invention also discloses an image-based tea stir-frying quality monitoring system, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement an image-based tea stir-frying quality monitoring method according to the present invention.

[0081] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0082] In the description of this specification, "multiple" or "several" means at least two, such as two, three or more, unless otherwise expressly and specifically defined.

[0083] While various embodiments of the invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of the invention.

Claims

1. An image-based method for monitoring the quality of tea leaves during stir-frying, characterized in that, Includes the following steps: S1. Acquire the time-series hyperspectral image of the stir-frying process, using the current frame as the real-time hyperspectral image, and calculate the first and second feature quantities of each pixel in the hyperspectral image; calculate the mean of the first feature quantity of each pixel as a macroscopic indicator representing chemical changes; transform the cumulative distribution function of the second feature quantity based on the indicator, and determine the quantization boundary of the second feature quantity according to the first and second derivatives of the transformed function; the quantization boundary of the first feature quantity is determined by the derivative of its own cumulative distribution function, thus completing asymmetric correlation quantization; S2. Construct a weighted two-dimensional joint histogram, and count the quantization feature pairs of each central pixel with each pixel in its neighborhood, with the neighborhood scale negatively correlated with the second feature quantity of the central pixel; multiply the count value of each feature pair by a weighting coefficient with respect to the spatial distance and the angle between the gradient direction; S3. The reference histogram is used to construct a low-dimensional standard manifold and mapping function using local linear embedding; S4. The histogram corresponding to the time series hyperspectral image is projected onto the manifold, and the stability confidence is calculated; The projection length of the movement vector from the historical projection point to the real-time projection point in the tangent direction of the target state point is divided by the corresponding time interval to obtain the weighted movement rate; Based on the confidence, the weights of the real-time observation distance and the predicted distance are assigned, and the tea quality is comprehensively judged by combining the weighted geodesic distance between the real-time projection point, the historical projection point and the target state point, and the weighted movement rate.

2. The method according to claim 1, characterized in that, The first characteristic quantity is a ratio of reflectance of a specific spectral channel representing the conversion of chlorophyll to pheophytin, including: Extract the real value of the first spectral reflectance of the current pixel in the 680nm wavelength channel and the real value of the second spectral reflectance in the 550nm wavelength channel, and divide the real value of the first spectral reflectance by the real value of the second spectral reflectance to obtain the quotient value as the reflectance ratio of the specific spectral channel.

3. The method according to claim 1, characterized in that, The second feature is a local anisotropic descriptor representing the degree of water loss and shrinkage, calculated based on the eigenvalues ​​of the structural tensor. It includes: obtaining the maximum and minimum eigenvalues ​​of the structural tensor within the local neighborhood window of the current pixel; dividing the difference between the maximum and minimum eigenvalues ​​by the sum of the maximum and minimum eigenvalues, and using the result as the local anisotropic descriptor.

4. The method according to claim 3, characterized in that, The step of obtaining the maximum and minimum eigenvalues ​​of the structure tensor within the local neighborhood window of the current pixel includes: selecting the near-infrared channel near the center wavelength of 750nm as the grayscale image, and calculating the gradient components of the grayscale image in the horizontal and vertical directions. A 3×3 pixel local neighborhood window is constructed with the current pixel as the center, and the gradient product is weighted and filtered using a Gaussian smoothing kernel to construct the structure tensor matrix. Solve the characteristic equation of the structure tensor matrix to obtain the maximum and minimum eigenvalues.

5. The method according to claim 1, characterized in that, The quantization boundary of the first feature is determined by the derivative of its own cumulative distribution function, thereby completing asymmetric correlation quantization, including: sorting the first feature values ​​of all pixels, fitting a smooth probability density function and integrating to obtain the smoothed cumulative distribution function of the first feature; The quantization boundary of the first feature quantity is determined by a point that simultaneously satisfies the following conditions: the second derivative of the smoothed cumulative distribution function of the first feature quantity with respect to the first feature quantity is equal to zero, and the first derivative of the smoothed cumulative distribution function with respect to the first feature quantity is less than a preset derivative judgment threshold; based on the determined quantization interval boundary, the first feature quantity is mapped to discrete quantization levels to complete the quantization.

6. The method according to claim 1, characterized in that, The neighborhood scale is negatively correlated with the second feature value of the center pixel, including: multiplying the adjustment coefficient by the value of the second feature value of the center pixel to obtain a first product; adding 1 to the first product to obtain a denominator value; and dividing the basic neighborhood radius by the denominator value to obtain the result as the radius of the neighborhood scale.

7. The method according to claim 1, characterized in that, The weighting coefficients regarding the angle between the spatial distance and the gradient direction include: Calculate the cosine of the angle between the feature gradient directions of the center pixel and its neighboring pixels, and take the absolute value of the cosine. Divide the spatial distance between the center pixel and its neighboring pixels by a preset spatial distance normalization constant, and take the negative value as the exponent term. Calculate the exponential function value with the natural logarithm as the base and the exponent term as the exponent.

8. The method according to claim 1, characterized in that, The calculation of stability confidence includes: calculating the distance between the real-time projection point and each reference projection point; counting the number of reference projection points whose distance is less than or equal to a preset distance threshold; dividing the number by the total number of reference projection points, and using the result as the local density, which is the stability confidence.

9. The method according to claim 1, characterized in that, The comprehensive assessment of tea quality includes: obtaining confidence weight coefficients by normalizing local density; multiplying the weighted movement rate by the time interval to obtain the actual displacement; subtracting the actual displacement from the weighted geodesic distance between the historical projection point and the target state point, and taking the absolute value of the difference as the actual predicted distance; dimensionlessly transforming the weighted geodesic distance between the real-time projection point and the target state point to obtain a first distance term, and dimensionlessly transforming the actual predicted distance to obtain a predicted distance term; subtracting the confidence weight coefficient from 1 to obtain a predicted weight coefficient; calculating the comprehensive evaluation score, including: multiplying the confidence weight coefficient by the first distance term to obtain a first value, and multiplying the predicted weight coefficient by the predicted distance term to obtain a second value; adding the first value and the second value and taking the negative of the sum as the total exponent term; calculating the result with the natural logarithm as the base and the total exponent term as the exponent as the comprehensive evaluation score; the larger the comprehensive evaluation score, the closer the tea quality is to the target state.

10. An image-based tea-stirring quality monitoring system, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement an image-based tea stir-frying quality monitoring method according to any one of claims 1-9.

Images