Machine vision-based tea segment heating fixation system and method

Abstract

The application discloses a tea segment heating and fixation system and method based on machine vision, relates to the technical field of tea processing and automatic control, and realizes real-time acquisition of comprehensive state information such as color, moisture, temperature and three-dimensional stacking thickness of tea through a multi-modal visual perception module; a central control and processing module fuses and processes these data, dynamically divides a tea processing area into multiple microzones, and independently performs adaptive and accurate segment heating and physical parameter collaborative regulation on each microzone in combination with a machine learning prediction model. The advanced prediction model of the application can continuously learn and evolve based on the evaluation results of the final product quality, thereby the application can significantly improve the quality and uniformity of tea fixation, improve production efficiency and automation level, enhance the process adaptability and flexibility to different raw materials and processing requirements, and realize intelligent, traceable process control and continuous optimization oriented to the final sensory quality.

Description

Technical Field

[0001] This invention relates to the field of tea processing and automation control technology, specifically a segmented heating and fixing system and method for tea based on machine vision. Background Technology

[0002] Fixing tea leaves is a crucial step in the processing of green tea, oolong tea, and other teas. Its purpose is to rapidly deactivate the enzymes in fresh leaves through high temperatures, inhibit the enzymatic oxidation of polyphenols, evaporate some moisture, soften the leaf texture, and remove grassy odors, thus laying the foundation for the formation of excellent color, aroma, and flavor qualities in tea. Traditional fixing processes rely heavily on manual experience, resulting in problems such as high labor intensity, poor quality uniformity, and low production efficiency. With the development of technology, automation and intelligentization have become important directions for upgrading tea processing equipment. Machine vision technology, due to its advantages of non-contact, speed, and objectivity, shows broad application prospects in tea processing.

[0003] Chinese invention patent CN108782797B discloses a control method for stir-frying tea leaves in an arm-type tea frying machine and an arm-type tea frying machine. The method involves capturing images of tea leaves in the frying pan in real time, analyzing the distribution shape of the tea leaves, and calculating a rotation matrix if the shape is not circular. The method then controls the robotic arm to rotate at a specific angle according to the rotation matrix to stir-fry the tea leaves, aiming to achieve uniform stir-frying and thorough fixation of the tea leaves. This invention utilizes machine vision to obtain macroscopic spatial distribution information of the tea leaves and adjusts the stirring action of the robotic arm accordingly, aiming to improve the uniformity of physical stir-frying.

[0004] Chinese invention patent CN115736046B discloses an intelligent tea rolling machine based on machine vision and its intelligent control method. This invention is applied to the tea rolling process. Its intelligent control part includes machine vision, precise temperature and humidity control, and precise oxygen control. Through machine vision, it identifies the characteristics of raw materials, collects color and texture features in real time, and constructs a convolutional neural network model to determine the degree of rolling, thereby realizing intelligent control of the rolling operation. At the same time, this invention emphasizes the multi-dimensional and precise control of rolling process parameters such as temperature, humidity, air, pressure, time, and frequency to improve the rolling effect and product quality.

[0005] The above designs have achieved a certain degree of automation and intelligent control in the tea frying or kneading process through technologies such as machine vision. However, there are still some limitations. For the core heat processing step of tea fixation, which is extremely sensitive to changes in temperature, time, and material state, the shortcomings of the above-mentioned existing technologies are mainly reflected in the following aspects: it is difficult to comprehensively and accurately perceive the core state of fixation (such as internal moisture, surface temperature, actual stacking thickness, and color changes) in a real time; there is a lack of effective means to dynamically and accurately divide micro-zones and apply differentiated heat control based on the real-time state of the tea; there is a lack of effective predictive ability for the fixation process; and in particular, there is a lack of adaptive learning and closed-loop optimization mechanisms that are linked to the quality of the final product. In terms of dynamic coupling and global collaborative optimization of multiple links such as heating, material conveying, and frying, the precision and intelligence need to be improved.

[0006] Therefore, there is an urgent need to develop an advanced tea fixation technology solution. This solution should be able to perceive the multi-dimensional state of tea leaves during the fixation process (including color, internal moisture, surface temperature, actual stacking thickness, etc.) in real time and with precision. Based on this comprehensive perception data, it should achieve dynamic intelligent zoning of the processing area and adaptive, differentiated, and precise control of the heat in each zone. At the same time, the solution should also integrate effective predictive models and adaptive learning capabilities. Through in-depth analysis of the processing process and correlation learning with the final product quality, it should achieve continuous optimization of the fixation process and an overall leap in the level of intelligence. This would effectively solve the shortcomings of current technology in terms of fixation uniformity, quality stability, and depth of intelligence, and meet the urgent needs of the modern tea industry for high-quality, high-efficiency, and standardized production. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and to propose a segmented heating and fixing system and method for tea leaves based on machine vision, so as to solve the above-mentioned problems.

[0008] The objective of this invention is achieved through the following technical solution: a machine vision-based segmented heating and fixing system and method for tea leaves, comprising:

[0009] The tea conveying and turning module is used to carry, convey, and turn the tea leaves to be processed. The tea conveying and turning module includes an adjustable speed conveyor belt and an adjustable speed, variable angle turning mechanism.

[0010] A multimodal visual perception module is configured above or to the side of the tea conveying and stirring module. The multimodal visual perception module includes at least one high-resolution RGB industrial camera, at least one near-infrared spectrometer or hyperspectral camera, at least one infrared thermal imager, and a light source system that provides illumination for the imaging unit, for real-time, non-contact acquisition of RGB image data, NIR spectral data and thermal imaging data of tea leaves.

[0011] The array-type segmented heating execution module includes multiple heating units whose power, energy output, temperature, and air volume can be controlled independently. The heating units are arranged in an array and cover the processing area on the tea conveying and stir-frying module.

[0012] The central control and processing module is connected to both the multimodal visual perception module and the array-type segmented heating execution module. The central control and processing module includes:

[0013] The multimodal data fusion and processing unit is used to receive data collected by the multimodal visual perception module and perform image spatial registration, feature fusion and abnormal noise filtering to obtain leaf color characteristics, moisture content distribution characteristics, surface temperature distribution characteristics and stacking thickness information of tea leaves.

[0014] The dynamic partitioning and heat map construction unit is used to dynamically divide the processing area into multiple independently controllable micro-regions based on the feature data obtained by the multimodal data fusion and processing unit, and to generate and update dynamic heat maps reflecting the blanching status of each micro-region in real time.

[0015] The zone heating control unit is used to independently calculate heating parameters for each micro-zone based on the withering state presented on the dynamic heat map and the preset withering target parameters, and output them to the corresponding heating unit in the array-type segmented heating execution module to achieve differentiated and adaptive heating of tea leaves in each micro-zone.

[0016] The predictive blanching model and adaptive learning unit are used to combine historical blanching data and real-time acquired data through machine learning to predict the heat and heating time required to reach the target blanching state for each micro-area, optimize the heating strategy online, and provide real-time warnings for abnormal states.

[0017] The collaborative control and stirring linkage unit is used to comprehensively consider the heat conduction effect between adjacent micro-areas when outputting the heating parameters of the zone heating control unit, to collaboratively optimize the heating command of the target micro-area, and automatically adjust the conveyor belt speed and the working parameters of the stirring mechanism in the tea conveying and stirring module according to the tea distribution uniformity or insufficient stirring detected by the multimodal visual perception module, so as to ensure that the tea is heated evenly.

[0018] The multimodal visual perception module further includes a 3D vision sensor or a structured light sensor to acquire information on the stacking thickness distribution of tea leaves on the tea conveying and stirring module. The stacking thickness distribution information participates in the dynamic micro-area division and heating decision of the central control and processing module.

[0019] The heating unit in the array-type segmented heating execution module is one or more of the following: an infrared heating lamp with independently adjustable power, a microwave feed port with independently controllable energy output, or a hot air nozzle with independently controllable temperature and air volume. The heating range of each heating unit corresponds to at least one micro-zone dynamically divided by the central control and processing module.

[0020] The multimodal data fusion and processing unit in the central control and processing module is further used to extract the average chromaticity value and chromaticity uniformity index of tea leaves from RGB image data, extract the average moisture content and moisture content standard deviation of the tea surface from NIR spectral data, and extract the average temperature and highest temperature point of each micro-region from thermal imaging data. These indicators are used by the dynamic zoning and heat map construction unit to construct the withering heat map and make zoning heating decisions.

[0021] The dynamic partitioning and heat map construction unit in the central control and processing module uses multi-factor threshold segmentation and clustering algorithms to achieve dynamic division of micro-regions based on parameters such as tea distribution, temperature gradient, moisture content distribution, and stacking thickness. The physical boundaries of the micro-regions are adjusted in real time as the tea moves or changes in its state.

[0022] The predictive blanching model in the central control and processing module and the predictive blanching model based on machine learning adopted by the adaptive learning unit are jointly trained by historical blanching data and real-time acquired data, and online parameter optimization is performed during system operation. They can predict the heating power curve and heating time required for each micro-area based on the current multimodal visual characteristics.

[0023] The predictive fixation model is based on a deep neural network, and its parameters are continuously optimized using process data of each batch of tea fixation and corresponding finished product quality evaluation results.

[0024] The collaborative control and stir-fry linkage unit in the central control and processing module explicitly aims to prevent over- or under-cooking of the target micro-area or its adjacent micro-areas when collaboratively optimizing the heating command of the target micro-area.

[0025] The system is further equipped with a parameter setting interface and a human-computer interaction terminal, allowing operators to input parameters such as tea variety, target degree of fixation, initial micro-zone quantity reference value, and zone adjustment sensitivity. It can also display dynamic heat maps, status parameters of each micro-zone, historical data trends, and real-time alarm information in real time. All key process data are automatically archived for tea quality traceability.

[0026] A machine vision-based method for segmented heating and fixing of tea leaves includes the following steps:

[0027] S1. Real-time acquisition of RGB image data, NIR spectral data, thermal imaging data, and stacking thickness distribution information of tea leaves using a multimodal visual perception module;

[0028] S2. The multimodal data fusion and processing unit of the central control and processing module receives and processes the data collected in step S1, performs image spatial registration, feature fusion and abnormal noise filtering to obtain the leaf color characteristics, moisture content distribution characteristics, surface temperature distribution characteristics and stacking thickness information of tea leaves.

[0029] S3. The dynamic partitioning and heat map construction unit of the central control and processing module dynamically divides the processing area into multiple micro-zones based on the feature data obtained in step S2, constructs a real-time heat map of the blanching state, and continuously adjusts the micro-zone division according to the analysis results and preset parameters.

[0030] S4. For each micro-zone, the partitioned heating control unit of the central control and processing module independently calculates the heating parameters based on the status of the dynamic heat map and the preset blanching target, and instructs the corresponding heating unit in the array-type segmented heating execution module to carry out differentiated heating of the tea leaves in that micro-zone; at the same time, the collaborative control and stir-frying linkage unit performs collaborative optimization of the heating strategy of adjacent micro-zones.

[0031] S5. The coordinated control and stirring linkage unit of the central control and processing module continuously monitors and automatically adjusts the stirring and conveying parameters of the tea conveying and stirring module based on the tea distribution uniformity or insufficient stirring detected by the multimodal visual perception module, so as to ensure uniform tea distribution and heating.

[0032] S6. Repeat steps S2 to S5 until the central control and processing module determines that all micro-area tea leaves have reached the set fixation standard, and the fixation process is completed.

[0033] The beneficial effects of this invention are:

[0034] 1. This invention achieves refined and adaptive heating of the tea fixing process through multimodal visual perception (covering color, moisture content, temperature, and precise 3D stacking thickness) and dynamic micro-area precision control. The medium-depth neural network continuously learns based on the feedback of the final product quality, which can ensure the uniformity and thoroughness of fixing to the greatest extent, effectively avoid defects such as scorched edges, red stems, and yellowing, and stabilize and improve the aroma, taste and color of the tea.

[0035] 2. The system’s high degree of automation and intelligence (including automatic data acquisition, dynamic zoning, intelligent heating decision-making, stir-frying and conveying linkage, etc.) significantly reduces the reliance on the manual experience of skilled technicians in the traditional withering process and reduces the uncertainty of human operation. This not only improves production efficiency and processing throughput, but also makes it possible to standardize and scale up the production of tea, solving the industry pain point of the scarcity of high-level withering masters.

[0036] 3. With precise multidimensional data perception, especially direct 3D thickness measurement and dynamic partitioning algorithms such as multi-factor threshold segmentation and clustering, and the powerful generalization and adaptive learning capabilities of deep learning models, this invention can effectively cope with the high uncertainty and volatility of fresh leaf raw materials in terms of variety, age, moisture content, stacking state, etc. The system can "tailor-make" the best withering solution for tea with different characteristics.

[0037] 4. By introducing a prediction model based on deep neural networks and establishing a continuous learning loop with the final product quality evaluation as the core feedback, the system goes beyond simple parameter optimization and has the self-evolutionary ability of "experience accumulation" and "skill improvement". It can learn from each production, continuously optimize its blanching strategy, and autonomously explore and solidify the optimal process.

[0038] 5. The human-machine interface (HMI) and comprehensive data management functions provide operators with full real-time process monitoring, parameter adjustment, and historical data analysis capabilities. All key process data and control decisions are automatically recorded and archived, which not only greatly enhances the transparency and controllability of the production process, but also provides a solid data foundation for comprehensive product quality traceability, continuous process improvement, and scientific research. Attached Figure Description

[0039] Figure 1 This is a system architecture diagram of the present invention;

[0040] Figure 2 This is a timing diagram of the present invention;

[0041] Figure 3 This is a flowchart of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0043] It should be noted that the directional concepts of "left", "right", "up", "down", "front", "back", "inner", and "outer" in the following scheme are all relative directions, and will not be listed one by one here.

[0044] Example 1:

[0045] like Figures 1 to 3As shown in the figure, this embodiment discloses a segmented heating and fixing system for tea based on machine vision and its working method. The system can perform intelligent and precise closed-loop control of the fixing process of tea to ensure the uniformity and high quality of fixing. The core of the system lies in its multimodal perception capability, dynamic partitioning processing logic, and adaptive heating and physical parameter adjustment mechanism.

[0046] In this embodiment, the tea conveying and turning module adopts a stainless steel mesh conveyor belt with a width of 1 meter and a length of 5 meters. Its conveying speed can be precisely adjusted by the central control and processing module between 0.1 m / min and 1.0 m / min according to the withering requirements. Above the conveyor belt, three sets of adjustable speed and variable angle turning mechanisms are evenly arranged, such as multi-axis rotating plates. Their rotation speed can be adjusted within the range of 10 to 60 rpm, and the angle between the plate and the plane of the conveyor belt can be adjusted between 15 and 45 degrees to adapt to the turning requirements of tea leaves with different tenderness and moisture content. This ensures that the tea leaves can be fully and evenly spread and turned during the conveying process, avoiding local accumulation or long-term one-sided heating.

[0047] Following closely behind, or positioned above key observation points on the conveyor belt, is a multimodal visual perception module. This module specifically includes: a 5-megapixel RGB industrial camera to capture the color, luster, and shape of the tea leaves; a near-infrared hyperspectral camera covering the 900-nanometer to 1700-nanometer wavelength range to detect the moisture content of the tea leaves' surface and interior; and a 320x240-pixel infrared thermal imager to monitor the real-time temperature distribution on the tea leaf surface. To ensure image quality, the module is also equipped with a light source system consisting of multiple sets of highly uniform LED flat panel lights to provide stable and uniform illumination for RGB and near-infrared imaging. These sensors work together to continuously and non-contactly scan the tea leaves on the conveyor belt, acquiring multi-dimensional data in real time.

[0048] The heating system is the key execution part of this embodiment, namely the array-type segmented heating execution module. This module consists of 50 independently controllable heating units arranged in a 5x10 array above the conveyor belt, covering the main processing area. These heating units can use far-infrared ceramic heating lamps, each with a power of 500 watts. Their radiation intensity can be precisely controlled from 0-100% by the central control and processing module by adjusting the input voltage or PWM (pulse width modulation) signal. Alternatively, a microwave feed array can be used, with each feed array independently controlling the microwave output energy. Another option is a zoned hot air nozzle, with each nozzle independently adjusting the outlet air temperature (e.g., between 120°C and 280°C) and airflow. The physical heating area of ​​each heating unit corresponds to or can be combined with the subsequently dynamically divided micro-zones, ensuring the feasibility of fine and differentiated heating of tea leaves.

[0049] The core central control and processing module is a high-performance industrial computer, which houses the core algorithm of this invention. The five units under this module work collaboratively:

[0050] The multimodal data fusion and processing unit is responsible for receiving the raw data stream from the vision module. This unit first performs image spatial registration on the RGB, NIR, and thermal imaging data to ensure that the data from different sensors can accurately correspond to the same physical location of the tea leaves. Then, it performs feature fusion and filtering of abnormal noise (such as sudden changes in light intensity and water vapor interference). The key is that it extracts the average Lab chromaticity value and chromaticity standard deviation (reflecting uniformity) of the tea leaves from the RGB image, inversely derives the average moisture content and moisture content standard deviation of the tea leaf surface from the NIR spectral data based on the intensity of specific moisture absorption peaks (such as around 1450nm), and extracts the average temperature and the highest temperature point of each potential micro-region from the thermal imaging data. In addition, it indirectly infers the stacking thickness information of the tea leaves by analyzing the texture of the RGB image or combining it with the attenuation characteristics of the NIR signal.

[0051] The dynamic partitioning and heatmap construction unit receives the multidimensional feature data processed above. It first dynamically optimizes and adjusts the processing area into multiple micro-regions with variable shapes and sizes based on the preset initial grid (e.g., dividing the processing area into 10x5 initial blocks) and real-time feature data, such as the actual distribution boundary of tea leaves, moisture content gradient, temperature difference, leaf color difference, and stacking thickness. For example, an area with significantly high moisture content will be divided into a separate micro-region. At the same time, the unit generates and updates a multidimensional "heatmap of the withering state" in real time. This map not only includes temperature, but may also use different colors or values ​​to indicate the moisture content, leaf color status, etc. of each micro-region, providing an intuitive basis for subsequent control.

[0052] For each dynamically divided micro-zone, the zone heating control unit independently calculates the required heating power and action time based on its current withering status (e.g., leaf color L value, moisture content percentage, current temperature) and preset target withering parameters for the tea variety (e.g., target L* value range, target moisture content range) as presented on the thermogram. For example, for a micro-zone with greenish leaves, high moisture content, and low temperature, a higher heating command will be output.

[0053] In this embodiment, the predictive fixation model and adaptive learning unit adopt a machine learning model based on gradient boosting decision tree (GBDT). This model is trained offline using a large amount of historical fixation data (including the initial state, process parameters, heating strategies and final fixation results of different batches of tea). In real-time operation, it combines the real-time feature data of each micro-region to predict the heat and approximate time required for the micro-region to reach the ideal fixation state (such as complete enzyme inactivation and moisture content dropping to a specific value). At the same time, the system will make small online parameter fine-tuning of the model's weights or decision thresholds based on the deviation between the actual fixation effect and the prediction. If a micro-region is detected to have a temperature rise that is too fast or an abnormal decrease in moisture content, an early warning will be triggered.

[0054] The collaborative control and stirring linkage unit is responsible for overall coordination. When outputting heating commands, it takes into account heat conduction. For example, if the heating power of a micro-area is high, the heating power of its adjacent micro-areas (especially those downstream in the conveying direction) may be appropriately reduced in advance. More importantly, if the RGB camera detects that the tea leaves are piled up too thickly in some areas or have not been stirred for a long time (e.g., by analyzing texture changes or color uniformity), the unit will immediately instruct the tea conveying and stirring module to appropriately reduce the conveyor belt speed and increase the rotation speed of the stirring mechanism in the corresponding area or adjust its deflector angle to promote the tea leaves to spread out and be heated evenly.

[0055] In addition to multimodal vision, an optional physical pressure sensor array can be installed for real-time detection of local stacking thickness and leaf compaction, improving the accuracy of thickness information. An RF / capacitive moisture detection module can be used as an alternative or supplement to NIR. The heating unit can adopt an ultrasonic heating array to rapidly heat the interior of specific areas of tea leaves using ultrasonic energy, resulting in a more sensitive response; or a fiber laser heating array can be used to achieve point-to-point precise heat treatment of micro-areas, suitable for high-end tea processing requiring extremely high spatial resolution. A liquid convection cooling / micro-mist cooling module can achieve local cooling and anti-scorching compensation in extreme high-temperature abnormal situations.

[0056] The stirring and material distribution can be assisted by an electromagnetically driven flexible plate, which can achieve precise angle and force changes, reduce mechanical wear, and improve the uniformity of stirring. A local micro airflow generator can assist in the local material dispersal.

[0057] Optional intelligent voice / touch interaction terminals allow operators to adjust process sensitivity, zoning parameters, and alarm response levels through natural language or graphical interfaces. They also support remote process operation and maintenance and big data analysis interfaces, providing interconnected and intelligent management for large-scale, multi-location factories.

[0058] Work process

[0059] (S1) Tea leaf import and multimodal real-time sensing:

[0060] Freshly picked tea leaves, such as a batch of tender buds from Longjing spring tea with a moisture content of approximately 75%, are evenly spread onto the stainless steel mesh conveyor belt of the tea conveying and turning module via an automatic feeding device. The conveyor belt starts smoothly at a preset initial speed (e.g., 0.3 m / min), delivering the tea leaves into the withering processing area. Once the tea leaves enter the sensing area located above the conveyor belt, the multimodal vision sensing module is immediately activated. At least one high-resolution RGB industrial camera within this module begins continuously capturing real-time images of the tea leaves, recording their initial bright green color, leaf shape, and gloss. Simultaneously, at least one near-infrared hyperspectral camera... (Or a spectrometer) scans the tea leaves on the conveyor belt at a specific frequency, collecting spectral data in the range of 900-1700nm, providing a basis for subsequent analysis of moisture content and some internal chemical composition changes; at the same time, at least one infrared thermal imager continuously records the temperature distribution map on the surface of the tea leaves. All the raw RGB image data, NIR spectral data and thermal imaging data acquired by all these sensors, together with the imaging parameters under stable lighting conditions provided by the light source system, and the initial stacking thickness information of the tea leaves indirectly evaluated through image analysis, are transmitted to the central control and processing module in real time at a high frame rate in a non-contact manner.

[0061] (S2) Multidimensional feature extraction and deep data fusion:

[0062] After the data arrives at the central control and processing module, its internal multimodal data fusion and processing unit first performs precise image spatial registration on the multi-source heterogeneous data from different sensors. This ensures that during subsequent analysis, a pixel in the RGB image can accurately correspond to the same physical location in the NIR spectral data and thermal imaging data. Next, the system executes feature fusion algorithms and filters for abnormal noise (such as ambient light interference and artifacts caused by momentary water vapor obstruction). This unit accurately calculates a series of key feature indicators from the registered and purified data. For example, from the RGB image data, it not only extracts the average CIELab chromaticity value of tea leaves (…), but also… L represents brightness, a represents red-green hue, and b represents yellow-blue hue. The standard deviation of chromaticity within a specific region is also calculated as a quantitative indicator of color uniformity. From NIR spectral data, by analyzing the absorbance or reflectance of specific strong moisture absorption peaks (such as those near 1450nm and 1180nm), combined with a pre-built calibration model, the average moisture content of the tea surface and the standard deviation of its spatial distribution are accurately derived. From thermal imaging data, the average surface temperature, the highest temperature point, and the uniformity of temperature distribution in each resolvable small area on the conveyor belt are extracted. These multi-dimensional features together constitute a comprehensive digital description of the current state of the tea.

[0063] (S3) Dynamic intelligent partitioning and real-time status heatmap construction:

[0064] Based on the aforementioned multidimensional feature data, the dynamic partitioning and heatmap construction unit begins operation. It does not employ fixed physical partitions, but rather, based on the real-time perceived state of the tea leaves, logically and dynamically divides the entire processing area into several "micro-regions" (segments) of variable shape and size. The division is based on the spatial continuity and variability of the feature data: for example, a tea leaf that has just entered the heating zone, with a generally high moisture content (e.g., 70-75%), bright green color (significantly negative a value), and low temperature (e.g., 30-40℃), will be either entirely or subdivided into several larger initial micro-regions; if tea leaves on the conveyor belt accumulate accidentally, causing a significant increase in stack thickness... If a certain area heats up particularly rapidly due to its proximity to the edge of a heat source, these areas may be separately divided into smaller micro-zones that require special attention. This unit will generate and continuously update a multi-layered "heat map of the withering state" in real time. This heat map not only displays the temperature distribution of each micro-zone in a pseudo-color, but may also overlay key information such as moisture content contour lines and leaf color change trend indicators (such as the rate of change of the a value), providing visual decision support and data foundation for subsequent precise control. The boundaries of the micro-zones will be dynamically adjusted as the tea leaves move on the conveyor belt and their own state evolves (such as moisture evaporation and color change), ensuring that the state of the tea leaves within each micro-zone is as uniform as possible.

[0065] (S4) Differentiated heating and coordinated optimization control by region:

[0066] For each dynamically formed micro-zone, the zone heating control unit undertakes the core regulation task. Based on the specific state parameters of that micro-zone as presented on the real-time heat map of the withering state (e.g., micro-zone X: current average temperature 95℃, average moisture content 68%, average a value -15) and the preset ideal withering curve and final target parameters for this batch of tea (e.g., first-stage target: leaf temperature reaches 160℃, moisture content drops to 65%; final target: leaf temperature stabilizes within a specific range, leaf color a value approaches 0, moisture content is approximately 60%), and combined with the predictive withering model and the adaptive learning unit's predicted suggestions regarding the heat and time required for that micro-zone to reach the next stage target, it independently and in real-time calculates the optimal heating parameters. These parameters are then translated into specific control commands. The system precisely corresponds to several heating units in the array-type segmented heating execution module (such as the power percentage of infrared lamps, or the temperature and wind speed settings of hot air nozzles). For example, for micro-areas with still high moisture content and insufficient heating, the system will instruct the corresponding heating units to increase power or raise temperature; while for micro-areas that have approached or reached the current stage target, the heating intensity will be reduced or temporarily turned off, achieving true differentiated and adaptive heating. At the same time, the collaborative control and stirring linkage unit will intervene to ensure that the heating between micro-areas will not produce adverse mutual effects. For example, if a micro-area is undergoing strong heating, the initial heating command of the micro-area immediately downstream in its conveying direction may be appropriately reduced to pre-compensate for possible heat drift and cumulative effects, and prevent the downstream tea leaves from heating up too quickly too early.

[0067] (S5) Intelligent linkage adjustment between physical conveying and stir-frying:

[0068] Throughout the withering process, heating control and physical handling are not separate. The collaborative control and stirring linkage unit continuously monitors the physical state of the tea leaves on the conveyor belt in real time through image data (especially RGB images and stacking information analyzed by them) fed back by the multimodal visual perception module. Once the visual analysis algorithm detects that the tea leaves are obviously piled up in certain areas (e.g., the brightness of a local area is abnormally low, and the texture features show clumps), or a large area of ​​tea leaves has not been effectively stirred for a long time (e.g., the temperature difference between the upper and lower surfaces of the leaves in a certain area is too large or the color change is inconsistent), the unit will take immediate action, instructing the tea leaf conveying and stirring. Module: Specifically increase the rotation speed of the stirring mechanism above the accumulation area or adjust the turning angle and depth of its paddles to quickly break up the tea clumps; if it detects that the overall material is too thick or that the tea leaves are prone to uneven distribution during the forward movement, the system may decide to moderately reduce the overall speed of the conveyor belt to give the stirring mechanism more time to work, or improve the uniformity of the tea leaf spreading by using more complex stirring combinations. Under certain preset conditions, the system can even issue an alarm to prompt manual intervention to ensure the uniformity of the feeding. This series of linkages ensures that the tea leaves are heated evenly at the microscopic level, creating a good physical condition for the uniformity of the chemical reaction.

[0069] (S6) Closed-loop iterative optimization until completion:

[0070] The system repeatedly performs the precise closed-loop control process described in S2 to S5 at an extremely high frequency: "sensing - feature extraction - dynamic zoning and heat map update - decision control (heating and physical handling) - effect feedback". Every time the tea leaves advance a short distance on the conveyor belt, their state is reassessed, the heat map and micro-zone divisions are dynamically updated, and the heating strategy and stirring parameters are adaptively adjusted accordingly. This is a continuous learning and optimization process until the central control and processing module comprehensively determines that the tea leaves in the vast majority of micro-zones on the conveyor belt (e.g., 95% of the processing area with a set threshold) have reached the preset fixation stage. The system determines that the entire withering process is complete when the following criteria are met: the leaf color changes completely from bright green to a uniform dark green (Lab* value enters the target range), the average moisture content drops to the target value (e.g., 58-62%), the leaf edges are not charred and the center is not reddened, certain key precursors related to aroma formation (such as amino acids and soluble sugars) as shown by NIR specific band analysis have been moderately transformed without excessive loss, the leaves are soft and elastic, the grassy smell has completely dissipated, and a roasted chestnut or bean curd aroma is revealed. At this point, the tea leaves will leave the system from the discharge port and enter the subsequent rolling, drying and other processes.

[0071] Specific application examples

[0072] Taking premium Longjing tea leaves as an example, the requirements for blanching are extremely strict. It is necessary to blanch the leaves thoroughly and evenly to completely destroy enzyme activity and prevent red stems and leaves, while also preserving their bright green color and rich aroma of bean curd and chestnuts to the maximum extent, and avoiding scorching, popping, or yellowing. Traditional hand-stirred Longjing tea relies heavily on the experience of master craftsmen.

[0073] The system described in this embodiment is used for fixing Longjing tea:

[0074] Fresh leaves (with a moisture content of about 75-78%) are fed into the system.

[0075] (S1-S2) The multimodal vision system immediately captures its tender green color, high water content and room temperature state, the RGB camera accurately records the initial color (e.g., high L value, negative a value and large absolute value), and the NIR spectrometer focuses on monitoring the water absorption peak near 1450nm.

[0076] (S3) The system dynamically divides micro-regions based on the initial uniformity of the fresh leaves (even the same batch of fresh leaves may have local differences) and the spreading condition, and the heat map shows that the whole is in the "awaiting killing" state.

[0077] (S4) In response to the temperature requirements of "high temperature and fast frying, high temperature first and then low temperature" for the withering of Longjing tea, the zone heating control unit will issue a high initial heating command to all micro-zones. For example, control the infrared heating lamp or hot air nozzle to quickly raise the leaf temperature to 200-250℃ (the surface temperature is monitored by a thermal imager and the leaf core temperature is calculated by the model). At this time, the prediction model will estimate the time required for rapid water loss in the first stage based on the tenderness of the fresh leaves and the initial moisture content.

[0078] As leaf temperature rises and moisture evaporates, NIR data shows a rapid decrease in moisture content, and RGB data shows that leaf color begins to turn dark green. The central control module will dynamically reduce the heating intensity of the corresponding micro-area based on these changes, and switch to the lower temperature (about 120-150℃) stage of "strip shaping". If a certain micro-area loses water more slowly due to slightly thicker stacking and the leaf color transformation is slow, the system will specifically maintain or slightly increase the heating intensity of that micro-area, and at the same time, link the stirring mechanism to strengthen the turning of this area.

[0079] (S5) Throughout the process, if the vision system detects that leaves are stuck together or form small clumps, the angle and speed of the stirring mechanism will be adjusted adaptively to break them up, and the conveying speed will also be finely adjusted according to the overall withering progress and the state of the tea leaves.

[0080] (S6) Finally, when the system detects that most of the tea leaves in the micro-area have reached the target state: the leaf color is uniform dark green (such as the L value reaches the set threshold, the a value is close to 0 or slightly positive), the moisture content drops to about 60-62%, the leaves are soft to the touch, the grassy smell dissipates, and a unique fragrance is emitted, the withering is judged to be complete.

[0081] Traditional fixation control often relies on a single parameter (such as temperature) or isolated threshold judgments of multiple parameters. By weighted fusion and nonlinear mapping of key features from multimodal visual perception (color, moisture content, temperature) and physical morphological features (such as stacking thickness, uniformity), a single scalar value that can more comprehensively, accurately, and dynamically characterize the current overall fixation degree of tea leaves can be generated.

[0082] The Dynamic Integrated Finishing Index (IDIFI) is used for any perceptible smallest unit (or dynamically formed micro-region Z) within the processing area. j At time t, its dynamic comprehensive completion index I DIFI (Zj,t) is defined as:

[0083]

[0084] I DIFI (Z j ,t): Microregion Z j The dynamic comprehensive blanching index at time t has a design target value range of [0,1], where 0 represents the initial state of fresh leaves and 1 represents the ideal blanching state.

[0085] k: Represents the index of key state features, set as {C,M,T}, corresponding to color, moisture content, and temperature respectively.

[0086] w k (t): The dynamic weight coefficient of the k-th feature at time t (or different stages of filming), ∑w k (t)=1, for example, in the early stage of blanching, the temperature w T and moisture content w M The weight of [a specific ingredient] may be relatively high to ensure rapid heating and dehydration; in the later stages of blanching, the color [is important]. C The weights can be increased to precisely control the final sensory quality; these weights can be preset or adaptively adjusted by the system based on the tea variety and processing stage.

[0087] Microzone Z j The average measurement value of the k-th feature at time t.

[0088] Average color characteristic value (e.g., the a* value in the CIEL*a*b* space obtained by analyzing RGB data or the overall color difference ΔE).

[0089] Average moisture content (obtained by analyzing NIR spectral data).

[0090] Average surface temperature (obtained by analyzing infrared thermal imaging data).

[0091] S k,target : The ideal target value of the kth feature (the preset completion standard).

[0092] S k,initial : The initial fresh leaf reference value for the k-th feature.

[0093] Φ k (⋅): The normalized state contribution function of the k-th feature. This is a non-linear function used to map the current value of the feature to its contribution to the overall completion level (range [0,1]).

[0094] For example, regarding color Φ C When the color of tea leaves changes from bright green (such as...) ≈−20) towards the target dark green (such as ≈−2) During the transformation, Φ C As the color changes from 0 to 1, it can be designed as a sigmoid function or a Gaussian cumulative function to reflect the nonlinear characteristics of the color change and its sensitivity near the target value.

[0095]

[0096] (This is an example of a sigmoid function, σ) C Control the steepness of the curve and adjust it to be within S... C,initial When it approaches 0, in S C,target When it is close to 1).

[0097] For moisture content Φ M When the moisture content decreases from the initial value to the target value, Φ M From 0 to 1.

[0098] For temperature Φ T When the temperature reaches and is maintained within the optimal blanching temperature range, Φ T The value tends to be 1; values ​​below or significantly above this range are considered low.

[0099] Ψ Th ( (Z j ,t): Stack thickness penalty factor, (Z j ,t) is a microregion Z jThe average stacking thickness at time t (indirectly obtained from a 3D sensor or image texture) is designed such that when the thickness exceeds a certain optimized value, the factor is less than 1 for I. DIFI Suppression was carried out to reflect the adverse effects of thick stacking on blanching efficiency and uniformity.

[0100]

[0101] Among them Th ideal The ideal thickness of the paving material is λ. Th It is the thickness penalty coefficient.

[0102] Ψ U (σ Sk (Z j ,t): The homogeneity factor within the micro-region, σ Sk (Z j ,t) represents the micro-region Z j Internal key features S k The standard deviation (i.e., the degree of non-uniformity) is less than 1 when the internal states (such as temperature, color) are highly non-uniformly distributed. DIFI The correction indicates that even if the average condition meets the standard, if the internal differences are too large, the overall quality of the finishing process is still not optimal. ;

[0103] Among them κ k It is the influence weight of each feature non-uniformity, norm(σ) Sk ) is the normalized standard deviation.

[0104] By accurately sensing the state of tea leaves in real time across multiple dimensions and using dynamic micro-zone division and differentiated heating based on this, the uneven withering caused by factors such as varying leaf age, uneven leaf thickness, and inaccurate pan temperature control in traditional withering processes is effectively solved. Each micro-zone of tea receives near-customized heat treatment, ensuring highly consistent withering results. Precise control of key physicochemical indicators (leaf color, moisture content, temperature) better deactivates enzymes and reduces excessive chlorophyll destruction and polyphenol oxidation, thus better preserving the tea's color, aroma, and flavor. For example, in the case of Longjing tea, this effectively avoids the formation of red stems, scorched edges, and bursting spots, resulting in a finished tea with a vibrant green color, high aroma, and mellow taste. The quality stability between batches is also greatly enhanced. The automated and continuous closed-loop control replaces the heavy and experience-dependent manual operation, reducing labor intensity and minimizing the impact of human factors on product quality. Predictive models and adaptive learning capabilities enable the system to cope with fluctuations in different raw materials, improving the overall operating efficiency and intelligence level of the production line. Precise zone heating avoids continuous overheating of areas that have already met the blanching requirements, thereby saving energy. At the same time, the improved and stable blanching quality reduces raw material losses caused by blanching failures or substandard quality. The system records various sensor data, control parameters, and decision-making processes throughout the blanching process, providing valuable data support for subsequent process analysis, quality traceability, and further optimization of the blanching model.

[0105] In summary, this embodiment, by constructing a core system integrating multimodal perception, dynamic zoning, adaptive heating, intelligent prediction, and collaborative control, and supplementing it with specific methodologies, can significantly improve the uniformity, quality, and intelligence level of tea processing, providing strong technical support for the standardized and large-scale production of tea.

[0106] Example 2:

[0107] like Figures 1 to 3 As shown, the tea fixing system disclosed in this embodiment inherits the core architecture described in Embodiment 1, including a tea conveying and stirring module, a multimodal visual perception module, an array-type segmented heating execution module, and a central control and processing module, as well as its five core functional units. On this basis, this embodiment introduces more advanced sensing methods, more precise algorithms, and more complete human-computer interaction and management functions, enabling the system to reach a new level of understanding and control of the tea fixing process, and is particularly good at handling complex situations where the raw materials are uneven in state.

[0108] Compared to Example 1, this example features a key upgrade to the multimodal visual perception module. Building upon the existing RGB camera, near-infrared hyperspectral camera, and infrared thermal imager, this module further integrates a high-precision 3D visual sensor, such as a line-scanning 3D contour sensor or a structured light sensor employing laser triangulation principles. This 3D sensor is precisely calibrated and synchronously triggered with other visual sensors, enabling real-time, non-contact acquisition of the precise three-dimensional contour of the tea leaves on the conveyor belt. This allows for direct calculation of the stacking thickness distribution information of the tea leaves at various locations, with an accuracy down to the millimeter level. Compared to the indirect inference of thickness information in Example 1, this provides more direct and reliable physical dimension data. The acquired stacking thickness data stream, along with the RGB, NIR, and thermal imaging data, is timestamped and sent to the central control and processing module.

[0109] Within the central control and processing module, its multimodal data fusion and processing unit can now directly integrate this precise stacking thickness information as another key input feature dimension alongside leaf color, moisture content, and surface temperature. The functionality of the dynamic partitioning and heatmap construction unit has been significantly enhanced. When performing dynamic partitioning, it no longer relies solely on general feature gradient analysis but explicitly employs multi-factor threshold segmentation and advanced clustering algorithms. For example, the system might use an adaptive iterative threshold segmentation method. First, based on multiple parameters such as the real-time temperature gradient of the tea leaves, the rate of change in moisture content, the degree to which the leaf color Lab* value deviates from the target, and the stacking thickness directly measured by 3D sensors, it initially identifies the boundaries of regions with significant state differences. Subsequently, it might apply algorithms such as DBSCAN (Density-Based Spatial Clustering of Applications with...). Clustering algorithms such as Noise or improved K-Means can aggregate pixels or small blocks with similar and spatially adjacent feature vectors (composed of temperature, moisture content, color, thickness, etc.) into a dynamic "micro-region". This algorithm can more intelligently identify and delineate micro-regions with irregular shapes but uniform states. Furthermore, the physical boundaries of the micro-regions will be adjusted in real time and smoothly as the tea leaves move on the conveyor belt or as their internal state (such as the thickness reduction caused by moisture evaporation) evolves, ensuring that the partitioning always accurately reflects the actual state of the tea leaves.

[0110] Furthermore, the collaborative control and stir-fry linkage unit has a more specific and refined objective when executing heating commands. When adjusting the heating command for the target micro-area, this unit will explicitly prevent local over-firing (such as burnt edges or bursting points) or under-firing (such as red stems or green leaves) in the target micro-area or its adjacent (especially upstream or downstream of the conveying direction) micro-area as one of its core optimization objectives. This means that the system not only considers the physical phenomenon of heat conduction, but also combines the heat sensitivity of the tea in this state, as well as the risk threshold of over-firing or under-firing (which can be read from the process parameter library of the tea variety), to make forward-looking, prevention-oriented power fine-tuning.

[0111] Finally, the system in this embodiment is further equipped with a complete parameter setting interface and a human-machine interface (HMI). This is typically represented by a 15-inch industrial-grade touch screen mounted on the control cabinet. The interface is user-friendly, allowing operators to make detailed settings before production. For example, operators can select the tea variety being processed (such as "West Lake Longjing" or "Biluochun Grade 1"), and the system will automatically load the corresponding recommended fixation process parameter curves; set the target fixation degree (such as "70% fixation" or "thoroughly fixated and dried"); input the reference value for the initial number of micro-zones and the sensitivity of the dynamic adjustment of the zones (for example, high sensitivity will result in finer zones even with small state differences, while low sensitivity will result in relatively coarser zones). During operation, the HMI will display dynamic heat maps (including multi-layered visualizations of information such as temperature, moisture content, and leaf color) in real time, status parameters of each key micro-zone (such as average temperature, maximum temperature, average moisture content, leaf color Lab* value, etc.), historical data trend charts (such as the curve of average leaf temperature change over a period of time), and any real-time alarm information (such as a heating unit failure, severe tea accumulation, etc.). All these key process data, including sensor readings, control commands, micro-zone evolution history, alarm records, etc., will be automatically archived and stored, and can be associated with batch information, providing detailed data support for subsequent tea quality traceability, process analysis, and system performance optimization.

[0112] Work process

[0113] The workflow of this embodiment is more refined and efficient than that of Embodiment 1 due to the introduction of new technical features:

[0114] (S1) Tea leaf introduction and enhanced multimodal real-time sensing:

[0115] After the tea leaves enter the conveyor belt, in addition to the RGB, NIR, and thermal imaging sensors, a newly added 3D vision sensor is activated simultaneously to accurately scan the three-dimensional contour of the tea leaf surface and output a high-resolution stacking thickness distribution map in real time. The data from all four modes are fused with timestamps and then sent to the central control and processing module.

[0116] (S2) Multidimensional feature extraction containing precise thickness information:

[0117] While performing operations such as image registration, feature fusion, and noise filtering as described in Example 1, the multimodal data fusion and processing unit of the central control and processing module incorporates the directly measured stacking thickness data as an independent, high-weight basic feature into subsequent analysis. At this point, the system's description of the tea's state is extended from a two-dimensional plane to a three-dimensional space.

[0118] (S3) Refined dynamic partitioning and heatmap construction based on multi-factor algorithm:

[0119] The dynamic partitioning and heatmap construction unit initiates a multi-factor threshold segmentation and clustering algorithm. This algorithm comprehensively considers leaf color characteristics, moisture content distribution, surface temperature distribution, and now highly accurate stacking thickness information. For example, if an area has normal leaf color according to RGB data, moderate moisture content according to NIR data, and uniform temperature according to thermal imaging, but the 3D sensor shows that its stacking thickness is far beyond the average, the algorithm will determine that this is a potential "stale green" risk area that requires special attention and classify it as a micro-region. Even if other two-dimensional features are not significantly different, the boundary of the micro-region will be adjusted in real time and smoothly in milliseconds according to the rolling, spreading, or thickness collapse caused by water loss during the transportation of tea leaves. As a result, the heatmap becomes more three-dimensional and information-rich.

[0120] (S4) Collaborative optimization of zoned heating based on prevention objectives:

[0121] When calculating heating parameters for each micro-zone, the zone heating control unit makes full use of accurate thickness information. For thicker micro-zones, not only may the initial heating power be higher, but the heating strategy may also include a longer duration or a specific pulse heating mode to ensure that the heat can penetrate. At the same time, the coordinated control and stir-frying linkage unit will actively prevent over- or under-heating when optimizing the heating strategy of adjacent micro-zones. For example, if a thick micro-zone requires strong heating, the system will predict its impact on the heat radiation and hot air drift of the downstream thin micro-zone through the model, and reduce the heating power of the downstream micro-zone in advance, or activate airflow isolation between them (if the heating module supports it) to ensure that the downstream tea leaves are not damaged by the strong fire of the "neighbor".

[0122] (S5) Smarter physical conveying and stirring linkage:

[0123] With precise thickness data, the collaborative control and stirring linkage unit can more accurately judge the physical state of the tea leaves. When tea leaves are found to be piling up, it will not only link the stirring mechanism, but may also adjust the depth of the stirring arm, the stirring frequency, or the instantaneous speed change of the conveyor belt more precisely according to the severity (thickness value) and range of the piling up.

[0124] (S6) Closed-loop iterative optimization and traceable completion:

[0125] The entire process remains a closed-loop iteration from S2 to S5. However, due to the richness of information in each stage and the optimization of the algorithm, the system's control is more precise and efficient. Operators can monitor the entire process in real time through the HMI. When necessary, they can fine-tune certain parameters (such as the overall speed of the withering process) according to preset permissions. After the withering process is completed, all sensor data, control parameters, micro-area evolution maps, alarm records, etc. related to this batch of tea are automatically archived into the database and can generate detailed process reports, providing a solid basis for quality management and traceability.

[0126] Specific application examples:

[0127] Suppose that a batch of fresh leaves contains a lot of coarse and old leaves or leaf stalks due to picking or initial sorting, or that tender leaves are mixed with mature leaves. This unevenness is a huge challenge for traditional blanching processes, and it is very easy to over-blanch tender leaves and under-blanch old leaves.

[0128] The system used in this embodiment:

[0129] 3D vision sensors can accurately identify areas of localized thickening or abnormal morphology caused by leaf stalks or coarse, old leaves. Multi-factor threshold segmentation and clustering algorithms quickly and precisely separate these areas from the surrounding normal young leaf areas, forming independent micro-regions. For micro-regions of leaf stalks or coarse, old leaves, due to their denser tissue structure, the water content may differ from that of young leaves. The system will match specific heating strategies (which may require longer penetrating heating or microwave-assisted heating in a specific wavelength band (if the heating unit supports it)). For micro-regions of delicate leaves, a relatively gentle but rapid heating method is adopted. The co-control unit will pay special attention to the heat crosstalk between these adjacent micro-regions with different characteristics, strictly implementing the optimization goal of preventing over-killing (young leaves) or under-killing (old stalks). Operators can observe the formation, heating status, and system response strategies of these special micro-regions in real time through the HMI, and adjust the zoning sensitivity as needed to ensure the best treatment effect.

[0130] Through precise 3D stacking thickness sensing and a refined dynamic zoning algorithm based on multiple factors, the system can better identify and independently process tea leaves in different states (such as uneven thickness, varying maturity, and uneven moisture content), significantly improving the uniformity and overall quality of the fixation process. This is particularly advantageous when processing low-grade or mixed raw materials. The collaborative control unit, with the explicit goal of preventing over- or under-fixation, makes the system more proactive and protective in its control, effectively reducing common fixation defects such as scorched edges, bursting points, red stems, and green leaves, resulting in a higher yield. The introduction of HMI allows operators to easily set and adjust process parameters according to different tea varieties and quality requirements, monitor the fixation process in real time, and promptly obtain alarm information. Automatic archiving of key data provides strong data support for production management, quality traceability, and continuous process improvement, enhancing the standardization and informatization of tea processing. More precise micro-zone division and targeted heating strategies, especially the understanding of the actual three-dimensional distribution of tea leaves, enable heat to act more effectively on the target tea leaves, reducing ineffective heating and energy waste.

[0131] In summary, through enhancements in perception, partitioning algorithms, collaborative control objectives, and system management, Example 2 enables the machine vision-based segmented heating and fixing system for tea to exhibit superior performance and practical value in handling complex raw materials, improving product uniformity and quality, and enhancing production management.

[0132] Example 3:

[0133] like Figures 1 to 3 As shown, the tea fixing system described in this embodiment inherits the precise hardware configuration (including tea conveying and stirring module, multimodal visual perception module with 3D vision sensor, array-type segmented heating execution module) and refined control logic (such as dynamic partitioning based on multi-factor algorithm, collaborative control aimed at preventing defects, and improved human-computer interaction and data management) described in Embodiments 1 and 2. At the same time, the predictive fixing model and adaptive learning unit inside its core central control and processing module have undergone a qualitative leap. This unit no longer only adopts conventional machine learning models, but deploys a more complex and powerful predictive fixing model based on deep neural network (DNN).

[0134] In this embodiment, the core of the predictive withering model and adaptive learning unit is a deep learning architecture. For example, a hybrid neural network model can be used: Convolutional Neural Networks (CNNs) are used to efficiently extract features from image data (RGB images, thermal images, and thickness and moisture distribution maps generated by 3D sensors and NIR data) of each micro-region obtained from the multimodal visual perception module. This automatically learns key visual representations of tea leaves at different withering stages, such as spatial texture, color distribution, and thermal field morphology. These deep visual features are then combined with other numerical sensor data of the micro-region (such as average moisture content, average temperature, historical heating power, etc.) and time-varying parameters. Inter-sequence information (such as the state and processing of the micro-region on the conveyor belt at the previous moment) is combined and fed into a variant of recurrent neural network (RNN) such as long short-term memory network (LSTM) or gated recurrent unit (GRU). LSTM or GRU can effectively capture the dynamic characteristics of the tea state evolving over time and the temporal dependencies in the heating process. Finally, the output layer of this hybrid DNN model is designed to directly predict the entire heating power change curve (rather than a single power value) required for each dynamic micro-region to reach the ideal withering state (which is preset by the operator through HMI or learned by the system from the historical best batch) and the accurate predicted total heating time.

[0135] The training of this DNN model is an ongoing process. The initial model is jointly trained using massive amounts of historical withering data and real-time multimodal visual feature data collected synchronously with these data. The historical data not only includes all sensor readings and system control parameters (heating unit power, conveyor belt speed, stirring frequency, etc.) of tea leaves of different varieties, tenderness, and moisture content during previous withering processes, but also crucially includes the corresponding finished product quality evaluation results of these batches of tea leaves. These quality evaluation results may come from professional sensory evaluation (such as scores for appearance, color, aroma, taste, and leaf residue), or from quantitative analysis of key chemical components (such as the content of tea polyphenols, amino acids, caffeine, chlorophyll, and the concentration of key aroma substances such as linalool and phenylacetaldehyde) in the laboratory.

[0136] During the actual operation of the system, the DNN model not only outputs prediction results, but also continuously optimizes its parameters online. This means that the model will make small-scale dynamic adjustments to certain hyperparameters of the current heating strategy (such as the overall scaling factor of the heating curve and the time fine-tuning of key inflection points) based on the real-time feedback data of the current batch of tea (such as the deviation between the leaf temperature rise rate, moisture evaporation efficiency, leaf color change speed and prediction) to more accurately match the actual response characteristics of the current tea.

[0137] More importantly, it has a continuous learning mechanism based on finished product quality feedback. After each batch of tea leaves completes the withering process and goes through subsequent rolling, drying and other processes to make dry tea, its quality is evaluated. This finished product quality evaluation result, which includes specific scores or grades, along with the complete withering process data of that batch (all sensor records, control command sequences, micro-region evolution history, etc.), is used as a new high-quality training sample and fed back to the predictive withering model and adaptive learning unit. The system will periodically (e.g., after the end of daily production or weekly) use this new data with clear quality labels to incrementally train or fine-tune the core DNN model. In this way, the model can learn which complex combinations of processing parameters have a stronger positive correlation with high-quality finished tea, and which subtle process deviations may lead to a decline in quality, thereby continuously improving the accuracy of its predictions and the effectiveness of its control strategies, enabling the system to autonomously and continuously evolve towards the goal of producing higher quality tea.

[0138] Work process

[0139] The workflow of this embodiment, based on the refined operation of embodiment 2, deeply integrates the prediction and learning capabilities of the DNN model:

[0140] The initial steps, such as tea leaf import, multimodal real-time perception (including 3D thickness information), multidimensional feature extraction, and dynamic partitioning and heat map construction based on multi-factor algorithms, are similar to those in Example 2, providing high-quality, high-dimensional input for the DNN model.

[0141] The key difference lies in the (S4) zone-specific heating and collaborative optimization control stage. At this stage, for each dynamically generated micro-zone, the predictive fixation model and the deep neural network model within the adaptive learning unit comprehensively analyze all its current feature vectors (color, temperature, moisture, shape, thickness, and historical state sequence). Based on its powerful nonlinear fitting and pattern recognition capabilities, the DNN model directly outputs an optimal heating power curve that varies over time for that micro-zone, as well as the expected total time to reach the ideal fixation state. This curve may not be a simple constant power or linear change, but rather a curve based on the physiological and chemical changes of the tea leaves at this stage (such as rapid heating and dehydration in the early stage, and stable heating in the middle stage). The complex waveform (customized for enzyme activity destruction, appropriate aroma enhancement or color preservation in the later stage) is received by the zone heating control unit. After receiving this "guidance curve", the control unit will drive the corresponding heating unit to reproduce the dynamic process of heating power or energy output as accurately as possible through high-frequency closed-loop control. At the same time, the expected duration output by the DNN model also provides an important reference for overall production scheduling and the connection of downstream processes. During the heating process, if the actual state (such as leaf temperature) deviates from the "ideal trajectory" implicit in the DNN model, the online parameter optimization mechanism will fine-tune the execution of the current heating curve, for example, slightly advance or delay a certain stage of the curve, or slightly increase or decrease the peak power.

[0142] Similarly, (S5) the intelligent linkage adjustment of physical conveying and stir-frying will also be indirectly affected by the output of the DNN model. For example, if the DNN model predicts that the tea leaves in a certain micro-area need to maintain a high leaf temperature for a long time due to their special state, the coordinating control unit may correspondingly instruct the stir-frying mechanism to adopt a gentler stir-frying mode that focuses more on uniform heating rather than rapid water loss, and adjust the conveying speed to ensure that the micro-area has enough processing time.

[0143] In the entire loop of (S6) closed-loop iterative optimization until the completion of the fixation process, in addition to the real-time adjustment described in Example 2, the system in this embodiment does not simply end after each batch of production tasks is completed. The operator or the automated quality inspection system will input the quality data of the finished products of that batch into the system (through HMI or data interface). The predictive fixation model and the adaptive learning unit will automatically add these new "process-result" data pairs to the training database and start retraining or parameter fine-tuning of the DNN model at appropriate times (such as when the system is idle). This means that if the system produces a batch of high-scoring tea according to the current model, the relevant weights of the model will be strengthened; conversely, if the quality of a batch of tea is poor, the model will analyze the difference between the processing data of that batch and the high-scoring batch and adjust its internal parameters to avoid repeating decisions that may lead to a decline in quality in similar situations in the future.

[0144] Specific application examples:

[0145] Suppose a tea company wants to produce a limited-edition premium green tea with a specific floral aroma or unique aftertaste. Its quality standards far exceed those of the norm, and it is extremely sensitive to subtle differences in raw materials (such as fresh leaves from different mountains or at different picking times).

[0146] The system used in this embodiment:

[0147] Initially, the system may be based on existing general green tea fixation models or a small amount of trial production data of this high-end tea. Operators will record the source information and characteristics of each small batch of raw materials in detail through HMI (as described in Example 2), and conduct very detailed sensory evaluation and physicochemical analysis on each batch of finished products. These finished product quality evaluation results containing rich details (e.g., the content of a certain component in the aroma, the freshness score of the taste, the brightness level of the soup color, etc.) are input into the system.

[0148] After several production cycles, the DNN model, through learning from these "small batches, high-precision feedback" data, gradually begins to grasp the complex mapping relationship between the fresh leaf characteristics and specific quality targets of this high-end tea. It may discover that to bring out that unique floral aroma, a special heating curve is needed at a specific stage of the withering process (defined by a specific leaf temperature and moisture content window), which involves first rapidly raising the temperature, then slightly lowering it, and then slowly raising it again, while simultaneously coordinating with gentle stirring at a specific frequency. This precise combination of process parameters may be difficult to accurately summarize and stably reproduce using traditional experience.

[0149] When a company tries to introduce a new tea variety with unknown characteristics for trial production, the system can also demonstrate strong adaptability. The first few batches of trial production may not be very effective, but as long as the finished products are carefully evaluated and fed back to the system, the DNN model can quickly learn the processing characteristics of the new variety and gradually explore the withering process parameters that are suitable for it to bring out its best quality potential. This greatly shortens the time for new product development and process finalization. For example, after several rounds of learning, the system may find that the new variety is more sensitive to high temperatures and the peak withering temperature needs to be reduced as a whole, but the medium-temperature treatment time needs to be extended to obtain the best aroma and flavor balance.

[0150] Deep neural networks can capture and learn more complex nonlinear relationships than traditional machine learning models, thus making extremely accurate predictions of the heating power curve and duration required for tea leaves in each micro-region to reach the ideal withering state. This makes the heating process no longer a simple "heat preservation" or "heating up", but a true "on-demand empowerment" and "rhythmic empowerment".

[0151] By incorporating actual finished product quality evaluation results as key feedback into model training, the system can transcend merely pursuing the achievement of intermediate process parameters and truly optimize and iterate itself with the goal of improving the quality of the final product. The system will become increasingly adept at "understanding" how to make good tea.

[0152] The powerful fitting ability and continuous learning mechanism of DNNs enable the system to quickly learn new patterns from limited samples, adapt to the characteristics of fresh leaves from different origins, seasons, harvesting standards, and even entirely new tea varieties, and quickly find or approximate their optimal withering process parameters. This accelerates product innovation and the speed of response to market changes. Through deep learning on a large amount of high-quality batch data, the system may discover some subtle combinations of process parameters that traditional experience has not clearly defined or is difficult to pass down stably. This "tacit knowledge" can be solidified into the model, realizing the digital inheritance and large-scale replication of top-notch withering techniques, and raising the overall quality ceiling of the tea industry. Although the DNN model itself is sometimes called a "black box," by analyzing the decisions made by the model when processing different inputs (such as the selection of specific heating curves), combined with a large amount of process data and quality results, it can, in turn, help researchers to more deeply understand the complex chemical changes in tea during the withering process and their correlation with the final sensory quality, thus promoting the development of tea science.

[0153] In summary, by introducing an advanced deep neural network prediction model and a continuous learning feedback mechanism based on finished product quality, Example 3 enables the system to not only complete current production tasks accurately and efficiently, but also to continuously learn and evolve, approaching or even surpassing the level of top-notch human skills. This provides a powerful technological engine for improving the quality of high-end tea, personalized customization, and continuous innovation in the industry.

[0154] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be modified within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A machine vision based tea segment heating fixation system characterized in that, The tea leaf conveying and tossing module is used for carrying, conveying and tossing tea leaves to be fixed, and comprises a speed-adjustable conveying belt and a speed-adjustable and angle-variable tossing mechanism. The multi-modal visual perception module is arranged above or on the side of the tea leaf conveying and tossing module, and comprises at least one high-resolution RGB industrial camera, at least one near-infrared spectrometer or hyperspectral camera, at least one infrared thermal imager, and a light source system for providing illumination for imaging units, and is used for real-time and non-contact acquisition of RGB image data, NIR spectral data and thermal imaging data of tea leaves. The arrayed segmented heating execution module comprises a plurality of heating units which can be independently controlled in power, energy output or temperature and air volume, and the heating units are arrayed and cover a processing area on the tea leaf conveying and tossing module. The central control and processing module is connected with the multi-modal visual perception module and the arrayed segmented heating execution module respectively, and comprises: The multi-modal data fusion and processing unit is used for receiving data acquired by the multi-modal visual perception module, and performing image space registration, feature fusion and abnormal noise filtering processing, so as to obtain leaf color features, water content distribution features, surface temperature distribution features and stacking thickness information of tea leaves. The dynamic partitioning and heat map construction unit is used for dynamically dividing the processing area into a plurality of independently controllable micro-zones based on feature data obtained by the multi-modal data fusion and processing unit, and generating and updating a dynamic heat map reflecting the fixation state of each micro-zone in real time. The partitioned heating control unit is used for independently calculating heating parameters for each micro-zone according to the fixation state presented on the dynamic heat map and preset fixation target parameters, and outputting the heating parameters to corresponding heating units in the arrayed segmented heating execution module, so as to realize differential and adaptive heating of tea leaves in each micro-zone. The predictive fixation model and adaptive learning unit is used for predicting the heat and heating time required for each micro-zone to reach the target fixation state by combining historical fixation data and real-time acquisition data based on a machine learning method, and performing online parameter optimization on the heating strategy, and real-time early warning of abnormal states. The collaborative control and tossing linkage unit is used for comprehensively considering the heat conduction effect between adjacent micro-zones when the heating parameters of the partitioned heating control unit are output, optimizing the heating instructions of the target micro-zone in collaboration, and automatically adjusting the conveying belt speed and the working parameters of the tossing mechanism in the tea leaf conveying and tossing module according to the tea leaf distribution uniformity or tossing deficiency detected by the multi-modal visual perception module, so as to ensure uniform heating of tea leaves. The multi-modal visual perception module further comprises a 3D visual sensor or a structured light sensor, which is used for acquiring stacking thickness distribution information of tea leaves on the tea leaf conveying and tossing module, and the stacking thickness distribution information participates in the dynamic micro-zone division and heating decision of the central control and processing module.

2. The machine vision based tea segment heating fixation system as claimed in claim 1 wherein: ​ 3. The machine vision based tea segment heating fixation system as claimed in claim 1 wherein: The heating units in the arrayed segmented heating execution module are one or more of infrared heating lamps with independently adjustable power, microwave feed ports with independently controllable energy output, or hot air jet ports with independently controllable temperature and air volume, and the heating range of each heating unit corresponds to at least one microzone dynamically divided by the central control and processing module.

4. The machine vision based tea segmental heat fixation system as claimed in claim 1, wherein: The multi-modal data fusion and processing unit in the central control and processing module is further configured to extract the average chroma value and chroma uniformity index of the tea leaves from the RGB image data, extract the average moisture content and moisture content standard deviation of the tea surface layer from the NIR spectrum data, and extract the average temperature and maximum temperature point of each microzone from the thermal imaging data, wherein the indexes participate in the construction of the fixation thermal map and the partition heating decision making by the dynamic partitioning and thermal map construction unit.

5. The machine vision based tea segmental heat fixation system as claimed in claim 1, wherein: The dynamic partitioning and thermal map construction unit in the central control and processing module adopts a multi-factor threshold segmentation and clustering algorithm during dynamic partitioning, realizes dynamic division of microzones based on tea distribution, temperature gradient, moisture content distribution, and stacking thickness parameters, and adjusts the physical boundaries of the microzones in real time as the tea moves or changes state.

6. The machine vision based tea segmental heat fixation system as claimed in claim 1, wherein: The predictive fixation model and adaptive learning unit in the central control and processing module adopts a machine learning-based predictive fixation model that is trained jointly based on historical fixation data and real-time acquisition data, and performs online parameter optimization during system operation, and can predict the heating power curve and heating duration required for each microzone according to the current multi-modal visual features.

7. The machine vision based tea segmental heat fixation system as claimed in claim 6, wherein: The predictive fixation model is a deep neural network-based model that is continuously optimized based on process data and corresponding finished product quality evaluation results for each batch of tea fixation.

8. The machine vision based tea segmental heat fixation system as claimed in claim 1, wherein: The collaborative control and tossing linkage unit in the central control and processing module optimizes the heating instructions for the target microzone to prevent over-fixation or under-fixation of the target microzone or its adjacent microzones.

9. The machine vision based tea segmental heat fixation system as claimed in claim 1, wherein: The system further comprises a parameter setting interface and a human-machine interaction terminal, which allow the operator to input parameters such as tea variety, target fixation degree, initial microzone quantity reference value, and partition adjustment sensitivity, and can display the dynamic thermal map, microzone state parameters, historical data trends, and real-time alarm information in real time, and automatically archive all key process data for tea quality traceability.

10. A method of segmental heating fixation of tea leaves based on machine vision, applied to the system of any one of claims 1-9, characterized in that: The method comprises the following steps: S1, acquiring RGB image data, NIR spectrum data, thermal imaging data, and stacking thickness distribution information of the tea in real time using a multi-modal visual perception module; S2, a multi-modal data fusion and processing unit of the central control and processing module receives and processes the data acquired in step S1, performs image space registration, feature fusion, and abnormal noise filtering to obtain leaf color features, moisture content distribution features, surface temperature distribution features, and stacking thickness information of the tea; S3, a dynamic partitioning and thermal map construction unit of the central control and processing module dynamically divides the processing area into multiple microzones based on the feature data obtained in step S2, constructs a real-time fixation state thermal map, and continuously adjusts the microzone division based on the analysis results and preset parameters. S4, for each micro-zone, the partition heating control unit of the central control and processing module independently calculates the heating parameters according to the state of the dynamic thermal map and the preset fixation target, and instructs the corresponding heating unit in the array type segmented heating execution module to implement differential heating on the tea leaves in the micro-zone; at the same time, the collaborative control and turning linkage unit optimizes the heating strategy of adjacent micro-zones collaboratively; S5, the collaborative control and turning linkage unit of the central control and processing module continuously monitors and automatically adjusts the turning and conveying parameters of the tea leaf conveying and turning module according to the tea leaf distribution uniformity or insufficient turning detected by the multi-modal visual perception module, so as to ensure the uniformity of tea leaf distribution and heating; S6, repeat steps S2 to S5 until the central control and processing module determines that all micro-zone tea leaves reach the set fixation standard, and the fixation process is completed.

Images