Image recognition-based loose tea fermentation degree determination method and system

Abstract

This invention discloses a method and system for measuring the fermentation degree of loose tea based on image recognition, belonging to the technical field of loose tea fermentation degree measurement. The system includes a multispectral imaging unit, used to simultaneously acquire visible light and near-infrared images at preset time intervals during the loose tea fermentation process, generating registered multispectral image pairs; a region self-segmentation unit, used to identify the initial fermentation region based on the brightness distribution of the loose tea pile in the visible light image, and then correct the boundary of the initial fermentation region based on the moisture content distribution in the near-infrared image, obtaining a dynamic fermentation partition map; a texture evolution mapping unit, used to extract the local binary pattern feature sequence of the visible light image and the gray-level co-occurrence matrix feature sequence of the near-infrared image, align the two types of feature sequences, input them into a temporal alignment network, and output the fermentation evolution vector of each sub-region; and a measurement result output unit, used to output the measurement results.

Description

Technical Field

[0001] This invention relates to the field of loose tea fermentation degree measurement technology, specifically to a method and system for measuring the fermentation degree of loose tea based on image recognition. Background Technology

[0002] Accurately determining the fermentation degree of loose-leaf tea is a core challenge in tea processing quality control. Existing methods for assessing the fermentation degree of loose-leaf tea largely rely on human experience, with technicians making subjective assessments of the fermentation process by observing color changes in the tea pile, smelling its aroma, or touching its temperature. Some solutions introduce a single type of industrial camera to capture images of loose-leaf tea during fermentation, using image processing algorithms to extract color or texture features, and then comparing these features with a pre-set fermentation model to output an approximate degree of fermentation. This approach, relying on manual methods or a single image modality, faces significant drawbacks in practical applications. Loose-leaf tea fermentation is a complex biochemical process involving pigment transformation, moisture migration, and heat and mass transfer; the fermentation progress in different regions within the pile often exhibits high asynchrony. Visible light images can only reflect color differences on the surface of the pile and cannot penetrate the pile to perceive internal moisture distribution and metabolic intensity information, resulting in insufficient ability to identify fermentation differences within the pile. Existing image processing workflows typically perform global feature extraction on the entire image, compressing the complex spatial state of uneven fermentation into a single statistic, losing local details and spatial relationships of the fermentation evolution in different sub-regions. Furthermore, fluctuations in initial moisture content and differences in pile morphology among different batches of raw materials can cause drift in the mapping relationship between image features and fermentation degree. A single modal feature sequence is insufficient to establish a robust representation of this nonlinear spatiotemporal coupling relationship, leading to a sharp amplification of errors in the measurement results during the later stages of fermentation. Regarding multimodal feature fusion and spatially structured modeling, existing technologies lack a mechanism for deeply fusing visible light texture evolution with near-infrared moisture distribution information in the temporal dimension, making it difficult to obtain a continuous and robust quantitative representation of the fermentation process in each independent sub-region. Simultaneously, existing methods fail to construct structured association models based on spatial adjacency between sub-regions, ignoring the fermentation evolution correlation caused by material exchange and microbial community transfer between adjacent regions during fermentation. This results in the inference of the global fermentation state relying solely on the simple splicing or averaging of features from each region, failing to accurately synthesize local information to invert the overall fermentation completion rate. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for measuring the fermentation degree of loose tea based on image recognition. By constructing a partitioning mechanism that coordinates visible light and near-infrared dual-mode images, and a nonlinear aggregation mechanism of multi-region evolution features based on spatial graph structure, the fermentation degree of loose tea can be accurately measured.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a loose tea fermentation degree measurement system based on image recognition. The system achieves real-time and quantitative determination of the fermentation degree of loose tea piles without human intervention by using multispectral imaging, region self-segmentation, texture evolution mapping, and graph convolution fermentation degree inversion.

[0005] The system is equipped with a multispectral imaging unit, used to synchronously acquire visible light and near-infrared images at preset time intervals during the fermentation of loose tea, generating registered multispectral image pairs. As a preferred embodiment of the invention, the multispectral imaging unit controls a multispectral camera to capture images at a fixed focal length and white balance parameter at the observation window position of the loose tea fermentation tank. Simultaneously, the visible light sensor and near-infrared sensor acquire visible light and near-infrared images under the same field of view, respectively. Then, affine transformation registration is performed using feature points at the edge of the loose tea pile as control points, so that each pixel in the visible light image forms a multispectral image pair with the corresponding spatially located pixel in the near-infrared image, thereby providing spatially strictly aligned image data for subsequent multimodal analysis. Preferably, before each capture, the exposure time of the visible light sensor is automatically adjusted according to the ambient light intensity, so that the average gray value of the visible light image is maintained within a preset gray range, thereby suppressing the interference of ambient light changes on image brightness and ensuring imaging consistency. Furthermore, the multispectral imaging unit is also equipped with an automatic dust removal device. Before each shot, it receives the humidity value from the humidity sensor inside the fermentation tank. If the humidity value is greater than the humidity threshold, it controls the compressed air nozzle to spray a pulsed airflow for 0.5 seconds onto the inner surface of the observation window glass. After the spraying ends, it delays for one second to allow the airflow to settle before triggering the multispectral camera to simultaneously acquire visible light and near-infrared images. This maintains the cleanliness of the observation window in a high-humidity environment, avoids image blurring or distortion caused by water mist, and improves the reliability of the acquired data.

[0006] The region self-segmentation unit receives multispectral image pairs, identifies the initial fermentation region based on the brightness distribution of loose tea piles in the visible light image, and then corrects the boundary of the initial fermentation region based on the moisture content distribution in the near-infrared image to obtain a dynamic fermentation zoning map. Preferably, this unit performs Gaussian filtering on the visible light image, calculates the local gray-level variance of each pixel, marks pixels with local gray-level variance greater than a first threshold as initial fermentation active pixels and connects them to form the initial fermentation region; simultaneously, it performs moisture inversion on the near-infrared image to obtain the relative moisture content value of each pixel, marks pixels with relative moisture content values ​​lower than a second threshold as moisture-limited pixels; removes the parts marked as moisture-limited pixels from the initial fermentation region, and then performs morphological closing operations to fill the internal holes to obtain the dynamic fermentation zoning map. Specifically, the moisture inversion on the near-infrared image is performed by calculating the relative moisture content value pixel by pixel based on a pre-calibrated exponential decay model of moisture content and near-infrared reflectivity. By using visible light texture and near-infrared moisture information for collaborative segmentation, low-activity areas in loose tea piles caused by water loss or differences in bulk density can be excluded from fermentation analysis, resulting in more consistent fermentation in the retained sub-regions. Preferably, after obtaining the dynamic fermentation zoning map, the region self-segmentation unit calculates the area of ​​each sub-region, merges sub-regions with areas smaller than the minimum area threshold into the adjacent sub-region with the largest area, re-performs connected component labeling, and assigns a unique region identifier to each sub-region. This eliminates the interference of isolated fragmented cells on subsequent feature extraction, resulting in stable and statistically significant sub-region divisions.

[0007] The texture evolution mapping unit extracts local binary pattern feature sequences from visible light images and gray-level co-occurrence matrix feature sequences from near-infrared images for each sub-region in the dynamic fermentation partition map. These two feature sequences are then aligned according to acquisition time and input into a pre-trained temporal alignment network, outputting a fermentation evolution vector for each sub-region. Preferably, the local binary pattern feature sequence is extracted as follows: for each pixel within each sub-region, a circular neighborhood with a radius of two is taken centered on that pixel. The gray-level values ​​of eight sampling points within the circular neighborhood are calculated to obtain an eight-bit binary code, which is then converted to a decimal number as the local binary pattern value of that pixel. A histogram is generated by statistically analyzing the local binary pattern values ​​of all pixels within the sub-region, and after normalization, it serves as the local binary pattern feature of that sub-region at the current time point. The local binary pattern features from all time points are arranged chronologically to obtain the local binary pattern feature sequence. This feature sequence can capture the coarsening and wrinkling of the micro-texture on the leaf surface during fermentation and its evolution over time. As a preferred embodiment of the present invention, the temporal alignment network is a bidirectional long short-term memory network based on an attention mechanism. It comprises two parallel feature encoding branches, which independently encode local binary pattern feature sequences and gray-level co-occurrence matrix feature sequences, respectively. An attention fusion layer then weights and sums the hidden states of the two branches to obtain the fused hidden state at each time point, which is used as the fermentation evolution vector. The attention fusion layer uses a self-attention mechanism to calculate the weights of the hidden states of the two branches. The weights are obtained by normalizing the concatenated vector of the hidden states of the two branches using a single-layer perceptron. During training, the network takes the local binary pattern feature sequences and gray-level co-occurrence matrix feature sequences of loose tea samples with known fermentation levels at multiple consecutive time points as input, and uses the manually labeled fermentation stage labels at each time point as the training objective. The network parameters are adjusted by minimizing the cross-entropy loss function. Through modal adaptive fusion, the network can automatically learn the association rules between visible light texture changes and near-infrared moisture migration, as well as their temporal lag or synchronization relationships, thereby generating a fermentation evolution vector with rich temporal semantics.

[0008] The fermentation degree inversion unit receives the fermentation evolution vectors of all sub-regions, constructs a fermentation evolution graph according to the spatial adjacency relationship of each sub-region in the dynamic fermentation partitioning graph, aggregates the evolution vectors of adjacent sub-regions using a graph convolutional layer, calculates the global fermentation characterization, and then regresses through a fully connected layer to obtain the current comprehensive fermentation degree of loose tea. Preferably, when constructing the fermentation evolution graph, each sub-region in the dynamic fermentation partitioning graph is mapped to a vertex in the graph. For any two sub-regions whose Euclidean distance is less than a preset neighborhood radius, an undirected edge is established between the corresponding vertices, and each vertex is assigned a feature vector composed of the fermentation evolution vector of that sub-region. The graph convolutional layer uses a Chebyshev graph convolution kernel to perform first-order neighbor aggregation on the feature vectors of each vertex, and obtains the higher-order features of each vertex after nonlinear activation. The higher-order features of all vertices are then spliced ​​together in spatial order to form the global fermentation characterization. During fully connected layer regression, the global fermentation characterization is input into three cascaded fully connected layers. The first fully connected layer outputs a 256-dimensional vector and connects to a batch normalization layer and a linear rectified activation function. The second fully connected layer outputs a 128-dimensional vector and connects to a dropout layer with a dropout rate of 0.5 and a linear rectified activation function. The third fully connected layer outputs a one-dimensional scalar, which is then compressed to the range of zero to one using a logistic function to obtain the overall fermentation degree of loose tea. By explicitly modeling the material and heat exchange between sub-regions using a graph structure, the influence of local fermentation outliers on the global determination can be effectively suppressed, allowing the overall fermentation degree to smoothly and accurately reflect the overall fermentation process.

[0009] The measurement result output unit outputs the overall fermentation degree of loose tea as the final measurement result. Preferably, this unit also compares the overall fermentation degree of loose tea with a preset fermentation completion threshold. If it is greater than or equal to the threshold, a fermentation termination signal is generated and sent to the temperature control actuator of the fermentation tank, thereby realizing automatic judgment and process control of the fermentation endpoint. If it is less than the threshold, the fermentation rate is calculated based on the difference between the current overall fermentation degree of loose tea and the historical overall fermentation degree output at the previous moment, and the fermentation rate is sent to the multispectral imaging unit to adjust the time interval for the next acquisition of multispectral image pairs. During the vigorous fermentation stage, the sampling period is automatically shortened to capture rapidly changing details, and during the slow fermentation stage, the sampling period is extended to reduce computational overhead.

[0010] Corresponding to the above system, the present invention also provides a method for determining the fermentation degree of loose tea based on image recognition. This method utilizes the multispectral imaging unit, region self-segmentation unit, texture evolution mapping unit, fermentation degree inversion unit, and measurement result output unit of the above system to achieve automated, multimodal fusion, and accurate determination of the fermentation degree during the fermentation process of loose tea.

[0011] The technical effects and advantages provided by the present invention in the above technical solution are as follows: By constructing a partitioning mechanism that coordinates visible light and near-infrared dual-mode images, the region self-segmentation unit first delineates the fermentation active region based on the brightness distribution of the visible light image, and then uses the relative moisture content value retrieved from the near-infrared image to remove moisture-limited regions and correct the region boundaries. This segmentation method, which is mutually constrained by visible light and near-infrared information, transforms the segmentation logic that relies solely on grayscale gradients or color thresholds into a dual-criteria driven by fermentation activity and moisture conditions. This ensures that the boundaries of each sub-region in the obtained dynamic fermentation partition map remain synchronized with the changes in biochemical gradients during the actual fermentation process. It avoids the problem of misclassifying regions that have not yet entered the fermentation state but have similar colors as fermentation regions, and also prevents inactive regions that have exhausted their moisture from continuing to be included in the fermentation evolution monitoring range. This provides a more representative regional base for subsequent texture feature extraction. By employing a nonlinear aggregation mechanism of multi-region evolutionary features based on a spatial graph structure, the fermentation degree inversion unit treats each sub-region as a graph vertex and constructs a fermentation evolution graph based on spatial adjacency relationships. Graph convolutional layers are used to aggregate the first-order neighbor features of each vertex, ensuring that the fermentation evolution vectors of each sub-region not only retain their own temporal texture evolution patterns but also incorporate the transmission influence of fermentation states from adjacent regions. This spatially structured aggregation method changes the processing approach that relies on global average pooling or direct feature concatenation. It allows the contribution of neighboring fermentation information to the fermentation degree of the central region to be nonlinearly weighted and combined using learnable convolutional kernels. When the fully connected layer regresses to obtain the overall fermentation degree of loose tea, the input global fermentation representation already embeds the spatial topological structure of the interrelationship between the fermentation progress of different sub-regions. This enables the overall fermentation degree to sensitively reflect the pulling or inhibiting effect of local fermentation-advanced or lagging regions on the overall fermentation completion degree through spatial adjacency relationships. Therefore, even when the non-uniformity of the pile significantly increases in the later stages of fermentation, it can still output measurement results consistent with the actual fermentation state. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 This is a schematic diagram of a loose tea fermentation degree measurement system based on image recognition; Figure 2 This is a flowchart of the multispectral imaging unit's image capture and registration process; Figure 3 This is a flowchart of the region self-segmentation process; Figure 4 This is a flowchart of the measurement result output and acquisition interval adjustment; Figure 5It is a curve showing the mapping relationship between the exposure time of the visible light sensor and the ambient light intensity; Figure 6 It is the calibration curve of the exponential decay model of near-infrared reflectance versus relative water content; Figure 7 It is the temporal variation of the comprehensive fermentation degree of loose tea and the logistic regression fitting curve; Figure 8 It is a curve showing the relationship between the absolute value of the fermentation rate and the time interval between sampling. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] See Figure 1 This invention provides an image recognition-based system for measuring the fermentation degree of loose tea, comprising: a multispectral imaging unit, used to synchronously acquire visible light and near-infrared images at preset time intervals during the fermentation process of loose tea, generating registered multispectral image pairs; a region self-segmentation unit, used to receive the multispectral image pairs, identify the initial fermentation region based on the brightness distribution of the loose tea pile in the visible light image, and then correct the boundary of the initial fermentation region based on the moisture content distribution in the near-infrared image, obtaining a dynamic fermentation partition map; and a texture evolution mapping unit, used to extract local second-order spectral data from the visible light image for each sub-region in the dynamic fermentation partition map. The system generates a fermentation evolution vector for each sub-region by aligning the two types of feature sequences according to the acquisition time and inputting them into a pre-trained temporal alignment network. The fermentation degree inversion unit receives the fermentation evolution vectors of all sub-regions, constructs a fermentation evolution map according to the spatial adjacency relationship of each sub-region in the dynamic fermentation partitioning map, aggregates the evolution vectors of adjacent sub-regions using graph convolutional layers, calculates the global fermentation characterization, and then regresses it through a fully connected layer to obtain the current comprehensive fermentation degree of the loose tea. The measurement result output unit outputs the comprehensive fermentation degree of the loose tea as the final measurement result.

[0016] In specific implementation, please refer to Figure 2The multispectral imaging unit controls the multispectral camera to capture images at the observation window of the loose tea fermentation tank. The focal length and white balance parameters of the multispectral camera remain constant throughout the imaging process. The multispectral imaging unit includes a synchronous trigger module. This module simultaneously sends trigger signals to both the visible light sensor and the near-infrared sensor during each image capture. The visible light sensor responds to the trigger signal and acquires a visible light image of the current field of view, while the near-infrared sensor responds to the trigger signal and acquires a near-infrared image of the same field of view. The visible light sensor and the near-infrared sensor are pre-aligned in space, ensuring a definite correspondence between the imaging position of the same point in the visible light image and its imaging position in the near-infrared image.

[0017] Optionally, before shooting, the multispectral imaging unit automatically adjusts the exposure time of the visible light sensor based on the ambient light intensity. The ambient light intensity is collected in real-time by an external ambient light sensor and converted into an ambient light intensity value. After reading the ambient light intensity value, the multispectral imaging unit determines the target exposure time of the visible light sensor according to a preset exposure time mapping relationship and sets the visible light sensor to the target exposure time. The exposure time mapping relationship is obtained through calibration, ensuring that the average pixel grayscale value of visible light images captured under multiple sets of different ambient light intensity conditions and corresponding exposure times falls within a preset grayscale range. The lower limit of the preset grayscale range is a first grayscale threshold, and the upper limit is a second grayscale threshold. An exposure time adjustment operation is performed before each shot. If the change in ambient light intensity compared to the previous shot is less than a preset change tolerance, the exposure time used in the previous shot remains unchanged.

[0018] After acquiring the visible light and near-infrared images, the multispectral imaging unit performs registration operations on the two images. Registration employs an affine transformation, using feature points at the edges of the loose tea pile as control points. Feature point extraction is as follows: in the visible light image, edge pixels of the loose tea pile are extracted using the Canny edge detection operator, and corner detection is performed on these edge pixels. The top-ranked corner points based on their response values ​​are selected as feature points in the visible light image. Similarly, in the near-infrared image, the same Canny edge detection operator is used to extract edge pixels of the loose tea pile, and corner detection is performed on these edge pixels. The top-ranked corner points based on their response values ​​are selected as feature points in the near-infrared image. Multiple control point pairs are obtained by matching the feature points from the visible light and near-infrared images based on their spatial arrangement consistency. An affine transformation matrix is ​​calculated based on these control point pairs. The near-infrared image is then resampled using the affine transformation matrix, aligning each pixel in the affine-transformed near-infrared image with the corresponding pixel in the visible light image, forming pixel-level registered multispectral image pairs. The registered visible light and near-infrared images are used as multispectral image pairs for subsequent units.

[0019] In some embodiments, the multispectral imaging unit is further equipped with an automatic dust removal device, which is connected to the multispectral camera and the humidity sensor inside the fermenter. Before each shot, the automatic dust removal device receives the humidity value output by the humidity sensor inside the fermenter and compares the humidity value with a preset humidity threshold. If the humidity value is greater than the humidity threshold, the automatic dust removal device controls a compressed air nozzle to spray a pulsed airflow onto the inner surface of the observation window glass. The duration of the pulsed airflow is 0.5 seconds. The compressed air nozzle is installed near the observation window and the spray direction is perpendicular to the inner surface of the observation window glass. After the spraying is completed, the automatic dust removal device starts a delay timer with a delay duration of one second, allowing the airflow to settle before shooting. After the delay timer reaches one second, the automatic dust removal device sends a permission signal to the multispectral camera. Only after receiving the permission signal does the multispectral camera perform synchronous triggering of the visible light sensor and near-infrared sensor for shooting. If the humidity value is less than or equal to the humidity threshold, the automatic dust removal device does not start spraying and directly sends a permission signal to the multispectral camera, allowing the multispectral camera to perform normal shooting.

[0020] See Figure 5 In the graph, the horizontal axis represents ambient light intensity in lux, ranging from approximately 500 to 5000; the vertical axis represents the visible light sensor exposure time in milliseconds (ms), ranging from approximately 1.5 ms to 36 ms. The blue dots represent calibration data points, and the black solid line represents the curve showing the mapping relationship between exposure time and ambient light intensity.

[0021] As the ambient light intensity increases, the exposure time of the visible light sensor decreases monotonically, and the curve shows an exponential decay. In the low ambient light intensity range (approximately 500 to 1500 lux), the exposure time decreases rapidly, dropping from a maximum of approximately 36 ms to approximately 15 ms. In the medium light intensity range (approximately 1500 to 3500 lux), the exposure time continues to decrease gradually, narrowing the range to approximately 5 ms to 15 ms. In the high light intensity range (approximately 3500 to 5000 lux), the exposure time tends to stabilize, gradually approaching a minimum value of approximately 2 ms.

[0022] The mapping curve reflects the mechanism by which the multispectral imaging unit in this embodiment automatically adjusts the exposure time of the visible light sensor based on ambient light intensity. By pre-calibrating the relationship between ambient light intensity and exposure time, the average grayscale value of visible light images captured under different ambient light intensity conditions is ensured to remain within a preset grayscale range (defined by the first and second grayscale thresholds in this embodiment). The curve fitting effect in the figure is good, and the calibrated data points closely match the fitted curve, indicating that this exposure time mapping relationship can accurately guide the automatic adjustment of exposure time.

[0023] This technique ensures that the visible light images acquired by the multispectral imaging unit under different lighting conditions during the fermentation process of loose tea are of stable quality, which is beneficial for the subsequent region self-segmentation unit to accurately identify the fermentation area and measure the degree of fermentation based on the texture and grayscale features of the image.

[0024] In specific implementation, please refer to Figure 3 The region self-segmentation unit receives a multispectral image pair, which includes a visible light image and a near-infrared image pixel-level registered with the visible light image. The region self-segmentation unit performs Gaussian filtering on the visible light image. The Gaussian filtering uses a Gaussian kernel with a standard deviation of σ and a kernel size of (2k+1)×(2k+1), where k is the radius of the Gaussian kernel and σ is the standard deviation of the Gaussian kernel. The Gaussian filtering operation iterates through every pixel in the visible light image, calculating a weighted average of the gray values ​​of all pixels within the coverage area of ​​the Gaussian kernel, centered on each pixel. The weighted average result replaces the original gray value of the center pixel, resulting in the filtered visible light image.

[0025] In the filtered visible light image, the region self-segmentation unit calculates the local gray-level variance of each pixel. For a pixel with coordinates (u,v) in the filtered visible light image, a local window with a width of w pixels and a height of h pixels is taken centered on this pixel. The local window contains N pixels, N=w×h. The gray values ​​of all pixels within the local window are extracted, and the mean μ(u,v) and local gray-level variance σ²(u,v) of the pixel gray values ​​within the local window are calculated. The formula for calculating the local gray-level variance σ²(u,v) is as follows: in, This represents the grayscale value of the i-th pixel within the local window. This represents the mean grayscale value of all pixels within a local window centered at coordinates (u,v). This represents the total number of pixels within the local window. The value is set to 25, meaning the width w of the local window is set to 5 pixels and the height h is set to 5 pixels. The local window contains 5 rows and 5 columns, totaling 25 pixels. This setting is based on the fact that, while taking into account local texture sensitivity and computational efficiency, a 5×5 window can effectively capture the brightness and darkness changes at the unit level of loose tea leaves.

[0026] The region self-segmentation unit compares the local gray-level variance σ²(u,v) with a first threshold, which is a pre-set gray-level variance judgment threshold. The value of the first threshold is determined as follows: The local gray-level variance of all pixels is calculated on the visible light image of loose tea samples in the unfermented or early fermentation stage, and the 90th percentile of the local gray-level variance distribution of all pixels is taken as the first threshold. If the local gray-level variance σ²(u,v) of a pixel is greater than the first threshold, the pixel is marked as an initial fermentation activated pixel. Connectivity analysis is performed on all marked initial fermentation activated pixels. Using the eight-neighborhood connectivity rule, adjacent initial fermentation activated pixels are grouped into the same connected region, and each connected region is considered an initial fermentation region.

[0027] In some embodiments, the region self-segmentation unit performs moisture inversion on the near-infrared images in a multispectral image pair. Moisture inversion is based on a pre-calibrated exponential decay model of moisture content and near-infrared reflectance, independently calculating the relative moisture content value for each pixel in the near-infrared image. The pre-calibration method involves preparing multiple sets of loose tea samples with known moisture content, acquiring near-infrared images of each set of loose tea samples under the same lighting and shooting parameters using the near-infrared sensor in the multispectral imaging unit, extracting the average reflectance value of the near-infrared image corresponding to each set of loose tea samples, and fitting the model parameters of the exponential decay model with the moisture content of the loose tea sample as the vertical axis and the average reflectance value of the near-infrared image as the horizontal axis. After fitting, after acquiring the near-infrared image to be measured, the region self-segmentation unit reads the reflectance value of each pixel in the near-infrared image, substitutes the reflectance value into the exponential decay model, and outputs the relative moisture content value pixel by pixel. The relative moisture content value is a dimensionless normalized value, ranging from zero to one; a larger value indicates a higher moisture content.

[0028] The region self-segmentation unit compares the relative moisture content of each pixel with a second threshold, which is the moisture limitation determination threshold. The second threshold is determined as follows: Multiple points are randomly collected from a loose tea sample where fermentation is known to be proceeding normally to measure the actual moisture content. The 15th percentile value of the measured moisture content at all points is calculated, and the relative moisture content value corresponding to this percentile value is used as the second threshold. If the relative moisture content of a pixel is lower than the second threshold, the pixel is marked as a moisture-limited pixel.

[0029] The region self-segmentation unit removes the portion of the initial fermentation region marked as a moisture-limiting pixel. This removal operation is performed pixel-by-pixel in the spatial domain: for each pixel marked as an initial fermentation activation pixel, if that pixel is also marked as a moisture-limiting pixel, its initial fermentation activation marker is cleared. After the removal operation, voids may appear within the initial fermentation region. The region self-segmentation unit performs a morphological closing operation on the removed region, using a radius of... The circular structuring element is used to first dilate the binary image of the removed region, and then erode the dilated result. Both the dilation and erosion operations use the same circular structuring element, and the radius of the circular structuring element is... The value is set to 3 pixels. This setting is based on the principle of filling small voids while preventing adhesion between different fermentation areas. In the binary image output after morphological closing operations, the connected regions formed by all pixels with a value of 1 are the sub-regions in the dynamic fermentation partitioning map.

[0030] After obtaining the dynamic fermentation partition map, the region self-segmentation unit calculates the area of ​​each sub-region in the dynamic fermentation partition map. The area of ​​a sub-region is defined as the number of pixels within the sub-region. The region self-segmentation unit compares the area of ​​each sub-region with a minimum area threshold, which is set based on the average projected area of ​​loose tea leaves, taking one-third of the average projected area of ​​a single leaf in the image as the minimum area threshold. For sub-regions with an area smaller than the minimum area threshold, the region self-segmentation unit finds the largest sub-region among the spatially adjacent sub-regions and merges all pixels of the sub-region with an area smaller than the minimum area threshold into the largest adjacent sub-region. After the merging operation is completed, the region self-segmentation unit re-performs connected component labeling on the merged dynamic fermentation partition map, assigning a unique region identifier to each re-labeled sub-region. The region identifier is a positive integer starting from 1 and incrementing. The region self-segmentation unit sends the region identifier and corresponding spatial range coordinates of each sub-region to the texture evolution mapping unit to guide the texture evolution mapping unit to extract the local binary pattern feature sequence and gray-level co-occurrence matrix feature sequence of each sub-region according to the region identifier.

[0031] See Figure 6 In the figure, the horizontal axis represents near-infrared reflectance (dimensionless), ranging from approximately 0.25 to 0.92, and the vertical axis represents relative moisture content (dimensionless), ranging from approximately 0.05 to 0.48. The light blue scatter plots represent the measured data from multiple calibration sample points. These plots are evenly distributed across the graph and exhibit a clear monotonically decreasing trend, meaning that higher near-infrared reflectance corresponds to lower relative moisture content. The black solid line represents the exponential decay model curve fitted based on the calibration sample data. The curve fits the scatter plot data well, demonstrating the exponential decay relationship between near-infrared reflectance and relative moisture content.

[0032] As described in this embodiment, using the exponential decay model to retrieve moisture content from each pixel in a near-infrared image accurately reflects the spatial distribution characteristics of moisture content in loose tea samples. The effectiveness of this model ensures the accuracy of moisture-limited pixel identification based on moisture-limited thresholds in the region self-segmentation units, providing a reliable basis for boundary correction of the dynamic fermentation zoning map. The parameters of the exponential decay model illustrated are obtained by fitting the average reflectance data of near-infrared images of multiple loose tea samples with known moisture content acquired under the same lighting and shooting conditions, ensuring the scientific validity and stability of the moisture retrieval.

[0033] See Figure 7 In the figure, the horizontal axis represents the collection time during the fermentation process of loose tea, in minutes, ranging from 0 minutes to approximately 180 minutes; the vertical axis represents the dimensionless value of the overall fermentation degree of loose tea, ranging from 0 to 1. The blue dots in the figure represent the actual fermentation degree data measured by the system described in this invention at each collection time point, and the black solid line represents the curve fitted to the measured fermentation degree data based on the logistic regression model (LSTIC).

[0034] As can be observed from the graph, the fermentation degree of loose tea exhibits a typical S-shaped increasing trend with collection time. In the initial fermentation stage (collection time 0 to approximately 50 minutes), the fermentation degree is low and increases slowly, indicating that the loose tea is in the initial stage of fermentation with weak fermentation activity. Subsequently, it enters a rapid fermentation stage (approximately 50 to 130 minutes), where the fermentation degree rises rapidly, and the curve slope increases significantly, reflecting an accelerated fermentation process. During this stage, the fermentation reaction is active, and the enzyme activity is significant. In the later stage (130 to 180 minutes), the fermentation degree tends to saturate, and the curve gradually flattens and approaches 1, indicating that the fermentation of the loose tea is nearing completion, the overall fermentation degree reaches its maximum, and the fermentation activity tends to stabilize.

[0035] The logistic regression curve in the figure shows a good fit to the measured scatter data, accurately reflecting the nonlinear characteristics of fermentation degree changes over time. This verifies the effectiveness of the fermentation degree inversion unit combined with graph convolutional networks and fully connected layers in accurately regressing the fermentation state. This curve also provides a reliable basis for subsequent fermentation completion judgment and adjustment of the data acquisition time interval, ensuring real-time monitoring and control of the fermentation process.

[0036] In specific implementation, please refer to Figure 4 After outputting the overall fermentation degree of loose tea, the measurement result output unit performs fermentation completion judgment and data collection time interval adjustment operations. The measurement result output unit internally stores a preset fermentation completion threshold, which is a dimensionless value ranging from 0 to 1. The specific value of the fermentation completion threshold is preset according to the target fermentation process requirements. The measurement result output unit reads the overall fermentation degree of loose tea at the current time t and records it as... And the overall fermentation degree of loose tea Compare with the fermentation completion threshold.

[0037] If the overall fermentation degree of loose tea If the fermentation completion threshold is greater than or equal to the specified value, the measurement result output unit generates a fermentation termination signal. This fermentation termination signal is a digital control signal, which the measurement result output unit sends to the temperature control actuator of the fermentation tank via the signal output interface. Upon receiving the fermentation termination signal, the temperature control actuator stops heating and humidity regulation within the fermentation tank and lowers the internal temperature of the fermentation tank to the target termination temperature, thereby terminating the loose tea fermentation process.

[0038] If the overall fermentation degree of loose tea If the value is below the fermentation completion threshold, the measurement result output unit reads the previous time step from the internal memory. Output of historical overall fermentation degree The previous moment The output time is the time immediately preceding the current time t. The measurement result output unit is based on the overall fermentation degree of loose tea. Overall fermentation degree of history Calculate the fermentation rate by the difference Fermentation rate The calculation formula is: in, Indicates the fermentation rate. This represents the overall fermentation degree of loose tea at the current time t. Indicates the previous moment Output of historical overall fermentation degree This represents the difference between the current time t and the previous time. The time interval between The unit is minutes. Fermentation rate. The unit is the change in fermentation degree per minute. The measurement result output unit will display the calculated fermentation rate. The data is transmitted to the multispectral imaging unit via the data communication interface.

[0039] The multispectral imaging unit received the fermentation rate. Then, based on the fermentation rate Adjust the time interval for the next acquisition of multispectral image pairs. The multispectral imaging unit has a preset reference acquisition time interval. and a rate adjustment coefficient Rate adjustment coefficient These are positive real numbers, calibrated based on the dynamic response characteristics of the fermentation equipment. The multispectral imaging unit is based on the fermentation rate. The absolute value is used to adjust the next data collection interval. The adjustment method is to adjust the fermentation rate. When the absolute value is large, shorten the sampling time interval; when the fermentation rate is large... When the absolute value is small, the sampling time interval is extended. The adjusted next sampling time interval... The timing trigger module, which is written into the multispectral imaging unit, is used to control the trigger time of the next image acquisition.

[0040] In some embodiments, the overall implementation process of the image recognition-based method for determining the fermentation degree of loose tea includes a multispectral imaging acquisition step, a region self-segmentation step, a texture evolution mapping step, a fermentation degree inversion step, and a measurement result output step.

[0041] In the multispectral imaging acquisition step, the multispectral imaging unit synchronously acquires visible light and near-infrared images at preset time intervals during the loose tea fermentation process, generating registered multispectral image pairs. The multispectral imaging unit controls the multispectral camera to take pictures at a fixed focal length and fixed white balance parameters at the observation window position of the loose tea fermentation tank. Before each shot, the multispectral imaging unit reads the ambient light intensity value collected by the ambient light intensity sensor and automatically adjusts the exposure time of the visible light sensor based on the ambient light intensity value, ensuring that the average grayscale value of the visible light image after exposure remains within a preset grayscale range. During shooting, the multispectral imaging unit simultaneously triggers the visible light sensor and the near-infrared sensor; the visible light sensor acquires the visible light image, and the near-infrared sensor acquires the near-infrared image. Using feature points at the edge of the loose tea pile as control points, the multispectral imaging unit performs affine transformation registration on the visible light and near-infrared images, aligning each pixel in the registered visible light image with the corresponding pixel in the near-infrared image, forming a multispectral image pair.

[0042] In the region self-segmentation step, the region self-segmentation unit receives multispectral image pairs. The unit performs Gaussian filtering on the visible light image and calculates the local gray-level variance of each pixel in the filtered image. Pixels with a local gray-level variance greater than a first threshold are marked as initial fermentation active pixels, and an initial fermentation region is formed from all initial fermentation active pixels through eight-neighbor connected component analysis. The unit calculates the relative water content value pixel-by-pixel on the near-infrared image based on a pre-calibrated exponential decay model of moisture content and near-infrared reflectance, and marks pixels with a relative water content value lower than a second threshold as moisture-limited pixels. The unit removes the portions of the initial fermentation region that are also marked as moisture-limited pixels, performs morphological closing operations on the removed regions to fill internal holes, and obtains a dynamic fermentation partition map. After obtaining the dynamic fermentation partition map, the unit calculates the area of ​​each sub-region, merges sub-regions with areas smaller than the minimum area threshold into the adjacent maximum area sub-region, and re-performs connected component labeling and assigns region identifiers to the merged dynamic fermentation partition map.

[0043] In the texture evolution mapping step, the texture evolution mapping unit extracts the local binary pattern feature sequence from the visible light image and the gray-level co-occurrence matrix (GLCM) feature sequence from the near-infrared image for each sub-region in the dynamic fermentation partition map. The local binary pattern feature sequence is obtained by calculating the local binary pattern values ​​of eight sampling points within a circular neighborhood of radius two for each pixel in each sub-region, statistically plotting the histogram, and normalizing it to obtain the local binary pattern features at each time point, then arranging them in chronological order. The GLCM feature sequence is obtained by calculating the GLCM in four directions on the near-infrared image of each sub-region, extracting the average values ​​of contrast, energy, correlation, and homogeneity features to obtain the GLCM features at each time point, then arranging them in chronological order. The texture evolution mapping unit aligns the two types of feature sequences according to the acquisition time and inputs them into a pre-trained temporal alignment network. The temporal alignment network is a bidirectional long short-term memory network based on an attention mechanism, containing two parallel feature encoding branches and an attention fusion layer, outputting the fermentation evolution vector of each sub-region at the current time point.

[0044] In the fermentation degree inversion step, the fermentation degree inversion unit receives the fermentation evolution vectors of all sub-regions and constructs a fermentation evolution graph according to the spatial adjacency relationship of each sub-region in the dynamic fermentation partitioning graph. Each sub-region is mapped to a vertex of the graph, and the fermentation evolution vector of the sub-region is used as the vertex feature vector. Undirected edges are established between vertices corresponding to two sub-regions whose Euclidean distance is less than a preset neighborhood radius. The fermentation degree inversion unit uses Chebyshev graph convolution kernels to perform first-order neighbor aggregation on the constructed fermentation evolution graph, calculates higher-order feature vectors for each vertex, and concatenates the higher-order feature vectors of all vertices in spatial order of sub-regions to form a global fermentation representation. The fermentation degree inversion unit inputs the global fermentation characterization into three cascaded fully connected layers for regression. The first fully connected layer outputs a 256-dimensional vector and passes it sequentially through batch normalization and a linear rectified activation function. The second fully connected layer outputs a 128-dimensional vector and passes it sequentially through a dropout layer with a dropout rate of 0.5 and a linear rectified activation function. The third fully connected layer outputs a one-dimensional scalar, which is compressed to the range of zero to one using a logistic function to obtain the overall fermentation degree of loose tea at the current moment.

[0045] In the result output step, the result output unit outputs the overall fermentation degree of loose tea, and then performs fermentation completion judgment and acquisition time interval adjustment operations after outputting the overall fermentation degree. The result output unit compares the overall fermentation degree of loose tea with a preset fermentation completion threshold. If the overall fermentation degree of loose tea is greater than or equal to the fermentation completion threshold, a fermentation termination signal is generated and sent to the temperature control actuator of the fermentation tank. If the overall fermentation degree of loose tea is less than the fermentation completion threshold, the fermentation rate is calculated based on the difference between the overall fermentation degree of loose tea and the historical overall fermentation degree output at the previous moment, and the fermentation rate is sent to the multispectral imaging unit to adjust the time interval for the next acquisition of multispectral image pairs.

[0046] See Figure 8 In the figure, the horizontal axis represents the absolute value of the fermentation rate, |v|, in units of change in fermentation degree per minute, ranging from 0 to 0.06; the vertical axis represents the time interval between the next acquisition of a multispectral image pair. The unit is minutes, and the value ranges from 2 to 5. The blue dots in the graph represent the actual adjusted data collection time intervals, and the black solid line is the adjustment mapping function curve obtained by mapping based on the absolute value of the fermentation rate.

[0047] As can be seen from the figure, with the increase of the absolute value of the fermentation rate |v|, the next sampling time interval... It shows a clear monotonically decreasing trend. Specifically, when the absolute value of the fermentation rate is low (close to 0), the sampling interval is close to 5 minutes, indicating that the fermentation changes are slow and the sampling frequency is relatively low. As the absolute value of the fermentation rate gradually increases, the sampling interval gradually shortens. When the absolute value of the fermentation rate reaches 0.06, the sampling interval is about 2 minutes, reflecting that the system automatically increases the sampling frequency to monitor the fermentation process more carefully when the fermentation changes accelerate.

[0048] The high degree of agreement between the adjusted mapping function curve and the actual adjustment interval data points indicates that the fermentation rate v calculated by the output unit of this invention effectively guides the multispectral imaging unit to adjust the acquisition time interval, achieving dynamic image acquisition scheduling in response to changes in the fermentation rate. This adjustment method meets the design requirements of the multispectral imaging unit in this embodiment, which shortens or extends the acquisition time interval based on the absolute value of the fermentation rate v. This ensures that the acquisition frequency is increased when the fermentation rate changes significantly and decreased when the fermentation rate changes slowly, thereby optimizing resource utilization and ensuring accurate monitoring of fermentation progress.

[0049] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A system for determining the fermentation degree of loose tea based on image recognition, characterized in that, include: The multispectral imaging unit is used to synchronously acquire visible light and near-infrared images at preset time intervals during the fermentation process of loose tea, and generate registered multispectral image pairs. The region self-segmentation unit is used to receive the multispectral image pair, identify the initial fermentation region based on the light and dark distribution of loose tea piles in the visible light image, and then correct the boundary of the initial fermentation region based on the moisture content distribution in the near-infrared image to obtain a dynamic fermentation partition map. The texture evolution mapping unit is used to extract the local binary mode feature sequence of the visible light image and the gray-level co-occurrence matrix feature sequence of the near-infrared image for each sub-region in the dynamic fermentation partition map, and input the two types of feature sequences into a pre-trained temporal alignment network after aligning them according to the acquisition time, and output the fermentation evolution vector of each sub-region. The fermentation degree inversion unit is used to receive the fermentation evolution vectors of all sub-regions, construct a fermentation evolution map according to the spatial adjacency relationship of each sub-region in the dynamic fermentation partition map, aggregate the evolution vectors of adjacent sub-regions with graph convolutional layers to calculate the global fermentation characterization, and then regress through fully connected layers to obtain the current comprehensive fermentation degree of loose tea. The measurement result output unit is used to output the overall fermentation degree of the loose tea as the final measurement result.

2. The image recognition-based loose tea fermentation degree determination system according to claim 1, characterized in that, The multispectral imaging unit specifically includes: The multispectral camera was positioned at the observation window of the loose tea fermentation tank to take pictures with a fixed focal length and fixed white balance parameters; During each shot, the visible light sensor and the near-infrared sensor are simultaneously triggered to acquire visible light images and near-infrared images under the same field of view, respectively. For the acquired visible light image and near-infrared image, affine transformation registration is performed using feature points at the edge of the loose tea pile as control points, so that each pixel in the visible light image and the corresponding spatially located pixel in the near-infrared image form the multispectral image pair.

3. The image recognition-based loose tea fermentation degree determination system according to claim 2, characterized in that, Before each shot, the multispectral imaging unit automatically adjusts the exposure time of the visible light sensor according to the ambient light intensity, so that the average gray value of the visible light image is maintained within a preset gray value range.

4. The image recognition-based loose tea fermentation degree determination system according to claim 1, characterized in that, The region self-segmentation unit specifically includes: Gaussian filtering is applied to the visible light image in the multispectral image pair, and the local gray-level variance of each pixel is calculated. Pixels with local gray-level variance greater than a first threshold are marked as initial fermentation activation pixels. The initial fermentation region is formed by connecting all the initial fermentation activation pixels. Moisture inversion is performed on the near-infrared image in the multispectral image pair to obtain the relative moisture content value of each pixel, and pixels with relative moisture content values ​​lower than a second threshold are marked as moisture-limited pixels. Remove the pixels marked as moisture-limited from the initial fermentation region, and then perform a morphological closing operation on the removed region to fill the internal voids to obtain the dynamic fermentation partition map.

5. The image recognition-based loose tea fermentation degree determination system according to claim 4, characterized in that, The water content inversion of the near-infrared image in the region self-segmentation unit specifically involves: The relative moisture content is calculated pixel by pixel based on a pre-calibrated exponential decay model of moisture content and near-infrared reflectance.

6. The image recognition-based loose tea fermentation degree determination system according to claim 1, characterized in that, The temporal alignment network in the texture evolution mapping unit is a bidirectional long short-term memory network based on an attention mechanism, and its training method is as follows: The local binary pattern feature sequence and gray-level co-occurrence matrix feature sequence of loose tea samples with known fermentation degree at multiple consecutive time points are used as training input, and the fermentation stage label of each time point is manually labeled as the training target. The network parameters are adjusted by minimizing the cross-entropy loss function. The temporal alignment network contains two parallel feature encoding branches, which are used to independently encode the local binary pattern feature sequence and the gray-level co-occurrence matrix feature sequence, respectively. The hidden states of the two branches are then weighted and summed through an attention fusion layer to obtain the fused hidden state at each time point. Finally, the fused hidden state is used as the fermentation evolution vector.

7. The image recognition-based loose tea fermentation degree determination system according to claim 6, characterized in that, The attention fusion layer in the temporal alignment network uses a self-attention mechanism to calculate the hidden state weights of the local binary pattern feature branch and the gray-level co-occurrence matrix feature branch. The weights are obtained by normalizing the concatenated vector of the hidden states of the two branches using a single-layer perceptron.

8. The image recognition-based loose tea fermentation degree determination system according to claim 1, characterized in that, The specific method for constructing the fermentation evolution diagram using the fermentation degree inversion unit is as follows: Each sub-region in the dynamic fermentation partitioning diagram is mapped to a vertex in the diagram; For any two sub-regions, if their Euclidean distance in the dynamic fermentation partitioning diagram is less than the preset neighborhood radius, then an undirected edge is established between the corresponding two vertices. The feature vector assigned to each vertex takes the value of the fermentation evolution vector of that sub-region; The graph convolutional layer uses Chebyshev graph convolution kernels to perform first-order neighbor aggregation on the feature vectors of each vertex, and then obtains the higher-order features of each vertex after nonlinear activation. The higher-order features of all vertices are then spliced ​​together in spatial order to form a global fermentation representation.

9. The image recognition-based loose tea fermentation degree determination system according to claim 6, characterized in that, The method for extracting the local binary pattern feature sequence in the texture evolution mapping unit is as follows: For each pixel in each sub-region of the dynamic fermentation partition map, take a circular neighborhood with a radius of two centered on the pixel, calculate the relationship between the gray values ​​of the eight sampling points in the circular neighborhood and the gray value of the center pixel, obtain an eight-bit binary code, and then convert the eight-bit binary code into a decimal number as the local binary mode value of the pixel. A histogram is generated by statistically analyzing the local binary pattern values ​​of all pixels within a sub-region. The histogram is then normalized and used as the local binary pattern feature of the sub-region at the current time point. The local binary pattern features at all time points are arranged in chronological order to obtain the local binary pattern feature sequence.

10. A method for determining the fermentation degree of loose tea based on image recognition, characterized in that, It includes all modules and method flows of the image recognition-based loose tea fermentation degree determination system as described in any one of claims 1 to 9.

Images