Impurity sorting and removing system based on machine vision

Abstract

The invention discloses an impurity sorting and removing system based on machine vision, and the system comprises a data collection and preprocessing layer which is responsible for obtaining multi-dimensional physical characteristics of tea leaves and impurities, and constructing three-dimensional scene representation; the feature analysis and decision-making layer is used for extracting light field fusion features by utilizing a ResNeSt-50 backbone network, directly outputting a preliminary classification confidence coefficient through a full connection layer, forming a first layer of decision-making candidates, simulating branches by combining rigid body dynamics to predict a blade movement track, performing Bayesian correction on the preliminary classification confidence coefficient after a shielding probability graph is sampled and calculated through Monte Carlo Dropout, and forming a second layer of decision-making candidates; meanwhile, a Cook-Torrent BRDF model is integrated to carry out material reflection correction, a feature map after material correction is output, and a correction value is converted into decision threshold offset; and the knowledge storage and update layer is used for constructing a bimodal memory pool composed of a gradient direction matrix and an LRU elastic cache, and generating a pseudo sample supplement long-tail category in combination with the four-layer full-connection pulse neural network.

Description

Technical Field

[0001] The present invention relates to the technical field of impurity sorting and removal, and particularly to an impurity sorting and removal system based on machine vision. Background Art

[0002] With the increasing demand for product quality in the tea consumption market and the accelerating need for the intelligent and refined transformation and upgrading of the tea processing industry, an intelligent impurity sorting and removal system based on machine vision technology has emerged. This system constructs a full-process automated solution from tea raw material transportation, multi-dimensional image acquisition, real-time defect analysis to precise mechanical removal by integrating a high-resolution industrial camera array, a multi-spectral light source module, and an image processing algorithm driven by deep learning. It forms a closed-loop control system with the capabilities of adaptive light compensation, parallel detection of multiple impurities, and dynamic threshold adjustment, significantly improving the standardization level and product added value of tea processing, and becoming a key technical support for promoting the digital transformation of the tea industry.

[0003] In the prior art, when tea leaves are overlapped and stacked, the underlying impurities are blocked by the upper leaves, and the impurities are similar in color to the tea leaves. Relying on HSV threshold segmentation is susceptible to the influence of light changes, and the deep learning model does not explicitly model the occlusion relationship. Therefore, an impurity sorting and removal system based on machine vision is proposed. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the prior art, and to propose an impurity sorting and removal system based on machine vision.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] An impurity sorting and removal system based on machine vision, comprising:

[0007] Data acquisition and preprocessing layer: responsible for obtaining multi-dimensional physical characteristics of tea leaves and impurities, constructing a three-dimensional scene representation, using ultraviolet laser two-photon excitation technology to penetrate the tea leaf surface to obtain deep impurity fluorescence signals, combining a four-way polarization camera and a Mueller matrix inversion algorithm to reconstruct the surface normal vector field to generate a 3D point cloud, and aligning the spectral-polarization-depth three-modal data through a U-Fusion network to generate a high-dimensional feature cube, outputting a fused feature map and a 3D point cloud to the feature analysis and decision-making layer, and receiving real-time spectral tuning parameters from the long-term adaptation layer;

[0008] Feature Analysis and Decision-making Layer: The ResNeSt-50 backbone network is used to extract light field fusion features. The preliminary classification confidence is directly output through the fully connected layer to form the first-layer decision candidates. Combining with the rigid body dynamics simulation branch to predict the blade movement trajectory, after calculating the occlusion probability map through Monte Carlo Dropout sampling, the preliminary classification confidence is corrected by Bayesian method. The attention guidance mechanism is applied to enhance the feature weights in the occluded area. At the same time, the Cook-Torrance BRDF model is integrated for material reflection correction, and the feature map after material correction is output. The correction amount is converted into a decision threshold offset. The final classification result is generated through a weighted voting mechanism, forming a three-dimensional decision fusion of light field features, movement trajectories, and material properties;

[0009] Knowledge Storage and Update Layer: A dual-modal memory pool composed of a gradient direction matrix and an LRU elastic cache is constructed, and a 4-layer fully connected spiking neural network is combined to generate pseudo-samples to supplement long-tail categories;

[0010] Long-term Adaptation Layer: The growth dynamics of tea leaves are modeled by the LSTM time series network. Combining with the improved color evolution equation to predict the change of leaf curl and dynamically adjust the spectral tuning parameters. At the same time, StyleGAN-ADA is used to generate cross-season synthetic images for time-domain data augmentation.

[0011] The above technical solutions further include:

[0012] Further, the specific steps of using the ultraviolet laser two-photon excitation technology to penetrate the tea leaf surface layer to obtain deep impurity fluorescence signals:

[0013] Two-photon Excitation Principle and Parameter Design: Based on the two-photon excitation principle, a 405nm ultraviolet laser parameter system is designed. By controlling the laser power density (<5mW / μm 2 ), repetition frequency (80MHz), pulse width (100fs), and numerical aperture (0.8), on the premise of ensuring Class 1 safety level, the probability of two photons being simultaneously absorbed by impurity molecules such as lignin / chitin is maximized. The product of its two-photon absorption cross-section (σ (2) = 1.2×10 -50 cm 4 ·s / photon) and fluorescence quantum yield (η = 0.15) determines the excitation efficiency;

[0014] Penetration Depth and Signal Detection Threshold Calculation: A penetration depth model is established through the fluorescence signal attenuation equation. Combining with the real-time monitored laser power (I0), impurity concentration (C_impurity), and tea leaf tissue attenuation coefficient (α = 0.15mm -1 ), the maximum detectable depth is dynamically calculated. The calculation formula is expressed as z_max = 1 / (2α)·ln(I0σ (2) φ2 ηC_impurity / I_thr);

[0015] Fluorescence signal acquisition and image reconstruction: A photomultiplier tube (PMT, H10721-20, anode sensitivity 150 A / W) is used in conjunction with a scanning galvanometer (6215H, scanning speed 1 m / s) to reconstruct the fluorescence intensity map through spatio-temporal integration. The fluorescence intensity map is expressed as I_image(x,y) = ∫I_fluor(t)·rect((x - vt) / d)dt;

[0016] Feature fusion: The fluorescence intensity map is spatio-temporally aligned and fused with the RGB image and surface normal vector (error < 0.1 pixel) to generate a multi-modal feature tensor containing chemical composition and geometric structure.

[0017] Furthermore, the specific steps for constructing the surface normal vector field to reconstruct the 3D point cloud by combining the four-way polarization camera and the Mueller matrix inversion algorithm are as follows:

[0018] Four-way polarization image acquisition: Use a four-way polarization camera to capture the light intensity distribution on the tea surface at four polarization directions of 0°, 45°, 90°, and 135°, and encode the surface microstructure information using the anisotropic characteristics of the interaction between polarized light and matter;

[0019] Stokes vector calculation: Convert the four groups of polarization images into a parameter matrix containing the complete polarization state through Stokes vector calculation. The parameter matrix quantitatively describes the light intensity, linear polarization component, and circular polarization characteristics;

[0020] Mueller matrix inversion: The Mueller matrix inversion technique constructs a physical model of the interaction between light and the tea surface, expressed as S_out = M(n)·S. The mapping relationship between the Fresnel reflection coefficient and the surface normal vector is described by a 4×4 matrix. A non-linear equation system is established specifically for the reflectivity difference between s-wave and p-wave, and the Levenberg-Marquardt optimization algorithm is used to iteratively solve to minimize the difference between the observed Stokes vector and the model prediction value, and the surface normal vector field is inverted;

[0021] 3D point cloud generation: To convert the normal vector information into three-dimensional space coordinates, integrate the camera calibration parameters and the dynamic light source azimuth field through photometric stereo vision technology, and use integral transformation to convert the normal vector distribution into the height map of the stacked tea surface, and finally generate the spatial point cloud.

[0022] Furthermore, the data acquisition and preprocessing layer receives the season correction parameters output by the long-term adaptation layer and dynamically adjusts the laser wavelength, expressed as λ_opt = 650 + 30·e^(-0.2W) + ΔT, where W is the humidity sensor reading and ΔT is the influence factor of seasonal temperature difference.

[0023] Further, the specific steps of extracting the light field fusion features by using the ResNeSt-50 backbone network and directly outputting the preliminary classification confidence through the fully connected layer to form the first-layer decision candidates are as follows:

[0024] Multi-modal feature fusion: At the input layer, a multi-modal feature fusion strategy is adopted to structurally analyze the 64-channel feature map output by the transmissive light field imaging system (including the two-photon fluorescence signal of channels 1-16, the Stokes polarization vector of channels 17-32, the 16-bit precision depth map of channels 33-48, and the motion vector field of channels 49-64), and a four-dimensional feature space containing material composition, spatial form, and dynamic information is constructed;

[0025] Split-Attention feature encoding: The channel group attention mechanism (dividing the 512 basic channels into a 4×2 branch structure) is used to dynamically weight the multi-modal features, and attention weights (including ReLU activation and Softmax normalization operations) are generated through global average pooling and a two-layer fully connected network;

[0026] Global context modeling: The 512-channel feature map output by Split-Attention is input, reduced to 64 channels through 1×1 convolution, dilated convolution (Dilation Rate = 3) is applied to capture long-range dependencies, the spatial resolution is restored through bilinear interpolation, and then restored to 512 channels through 1×1 convolution, and then a context-enhanced feature map is output;

[0027] Global average pooling and classification layer: The 512×224×224 feature map is compressed into a 512-dimensional vector through global average pooling, and the original classification confidence is generated through a fully connected layer (including a 9-class output weight matrix) and the Softmax function, forming a primary decision candidate including material identification, morphological characterization, and motion compensation.

[0028] Further, the method for predicting the blade motion trajectory by combining the rigid body dynamics simulation branch, calculating the occlusion probability map through Monte Carlo Dropout sampling, and then performing Bayesian correction on the preliminary classification confidence includes the following steps:

[0029] Temporal prediction network: The spatio-temporal dependence relationship of the blade displacement is established through a two-layer LSTM temporal network, and the hidden state is dynamically updated using the gated recurrent unit, and finally the future displacement field is output and converted into a spatial constraint coefficient, and the spatial constraint coefficient compensates for the positioning deviation caused by the conveyor belt vibration by enhancing the classification confidence of the moving object (such as a 1.21-fold gain for the tea stalk category);

[0030] Occlusion probability calculation: The Monte Carlo Dropout technique is used to randomly inactivate and sample the joint feature vector. The model uncertainty is quantified by calculating the variance of the classification results, and then mapped through the Sigmoid function to generate the occlusion probability, expressed as P_occ = 1 - (1 / (1 + e^(-5 / (σ^2 - 0.02)))). Finally, a confidence correction coefficient is generated, and the correction coefficient and the spatial constraint coefficient output by the dynamics branch form a multiplicative fusion mechanism to achieve probabilistic confidence update under the Bayesian decision framework.

[0031] Furthermore, the integrated Cook-Torrance BRDF model is used for material reflection correction to output the feature map after material correction. The correction amount is converted into a decision threshold offset. The specific steps for generating the final classification result through the weighted voting mechanism to form the three-dimensional decision fusion of light field features, motion trajectories, and material characteristics are as follows:

[0032] Material reflection correction: The roughness and Fresnel terms of the target surface are queried through a pre-calibrated material parameter library (such as plastic α = 0.4, tea α = 0.2). Subsequently, the Cook-Torrance BRDF model is used to calculate the microplane distribution function D(h) and the geometric attenuation term G(l, v). The calculation formula for the microplane distribution function is expressed as D(h) = α^2 / (v((n·h)^2(α^2 - 1) + 1)^2). After estimating the reflected radiance ΔL through Monte Carlo integration, it is converted into a decision threshold offset ΔT = 0.3ΔL;

[0033] Three-dimensional decision fusion: Through the three-dimensional weighted voting of the light field feature confidence, motion trajectory confidence, and material correction confidence, combined with the adaptive weight adjustment mechanism driven by Focal Loss, a fused three-dimensional decision is formed.

[0034] Furthermore, the specific steps for constructing a dual-modal memory pool composed of a gradient direction matrix and an LRU elastic cache and generating pseudo-samples to supplement long-tail categories by combining a 4-layer fully connected spiking neural network are as follows:

[0035] Construction and management of the dual-modal memory pool: The gradient direction matrix is constructed through the stable memory area and updated by exponential weighted average using the momentum coefficient to achieve the stable storage and dynamic optimization of the prototype features of the basic categories. At the same time, the LRU cache mechanism of the elastic memory area is used to selectively retain the newly added long-tail samples in combination with the priority calculation formula. The priority calculation formula is expressed as, and the preliminary accumulation of rare impurity features is completed;

[0036] Pulse Neural Network (SNN) Pseudo-Sample Generation: A 4-layer fully-connected SNN simulates biological spike-timing-dependent plasticity (STDP) through the LIF neuron model, performs spatio-temporal encoding on a 64-dimensional feature matrix extracted from the memory pool to generate a spike sequence, and then decodes it into pseudo-features through spatio-temporal integral decomposition. Pseudo-samples are generated through the joint optimization of reconstruction loss and adversarial loss.

[0037] Furthermore, the growth dynamics of tea leaves are modeled by an LSTM time series network, the change in leaf curl is predicted by combining an improved color evolution equation, and the spectral tuning parameters are dynamically adjusted. At the same time, StyleGAN-ADA is used to generate cross-seasonal synthetic images for time-domain data augmentation, including the following steps:

[0038] Modeling the Growth Dynamics of Tea Leaves by LSTM Time Series Network: A two-layer LSTM time series network is used to model the growth dynamics of tea leaves. After inputting historical environmental data (temperature, humidity, light), the network captures time series dependencies through a peephole connection structure, and combines a discretized curl dynamics equation and a mixed loss function (including MSE current state loss and KL divergence probability distribution loss). The discretized curl dynamics equation is expressed as Predict the leaf curl and perform the inversion calculation of the color evolution parameter ΔT;

[0039] Inversion of Color Evolution Equation Parameters: An impact model of seasonal temperature difference is established through an improved color evolution equation, and the extended Kalman filter is used for online parameter estimation to dynamically adjust the laser wavelength compensation amount and exposure time;

[0040] StyleGAN-ADA Cross-Seasonal Data Augmentation: Deploy an improved StyleGAN-ADA generator. Through the multi-modal input of stitching spring tea images and autumn tea spectral curves, an 8-layer MLP mapping network and a skip connection synthesis structure, combined with an adaptive discriminator enhancement strategy (dynamic difficulty adjustment probability p = 0.5 + 0.5·tanh[(FID - μ) / σ]), generate cross-seasonal synthetic images that retain the morphological characteristics of spring tea but present the color of autumn tea.

[0041] The present invention has the following beneficial effects:

[0042] 1. In the present invention, the process of leaf stacking is simulated through rigid body dynamics, and the pre-trained model's understanding of physical interactions improves the robustness to the natural stacking state. Monte Carlo Dropout quantifies the probability of the existence of occluded areas and dynamically adjusts the classification confidence, reducing mis-removal caused by occlusion.

[0043] 2. In the present invention, a material reflection model is established to distinguish the differences in reflection characteristics between tea leaves and impurities such as plastics and metals, solving the misjudgment caused by traditional methods relying on color / texture.

[0044] 3. In the present invention, the stable memory area solidifies the basic category features, the elastic memory area manages the newly added samples through the LRU strategy to solve the problem of catastrophic forgetting, and uses a spiking neural network to generate rare impurity samples to alleviate the data imbalance problem under the long-tail distribution. Description of the Drawings

[0045] Figure 1 It is a system block diagram of an impurity sorting and removal system based on machine vision proposed by the present invention. Detailed Embodiments

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Please refer to Figure 1 As shown, the present invention is an impurity sorting and removal system based on machine vision, including:

[0048] Data Acquisition and Preprocessing Layer: Responsible for obtaining multi-dimensional physical characteristics of tea leaves and impurities, constructing a three-dimensional scene representation, using ultraviolet laser two-photon excitation technology to penetrate the tea leaf surface to obtain deep impurity fluorescence signals, combining a four-way polarization camera and a Mueller matrix inversion algorithm to construct a surface normal vector field to reconstruct a 3D point cloud, and generating a high-dimensional feature cube by aligning spectral-polarization-depth three-modal data through a U-Fusion network, outputting a fused feature map and a 3D point cloud to the Feature Analysis and Decision Layer, and receiving real-time spectral tuning parameters from the Long-Term Adaptation Layer;

[0049] Feature Analysis and Decision Layer: Using a ResNeSt-50 backbone network to extract light field fusion features, directly outputting preliminary classification confidence through a fully connected layer to form the first layer of decision candidates, combining a rigid body dynamics simulation branch to predict the leaf movement trajectory, calculating an occlusion probability map through Monte Carlo Dropout sampling, then performing Bayesian correction on the preliminary classification confidence, applying an attention guidance mechanism to enhance the feature weight of the occlusion area, and at the same time integrating a Cook-Torrance BRDF model for material reflection correction, outputting a feature map after material correction, the correction amount is converted into a decision threshold offset, and the final classification result is generated through a weighted voting mechanism, forming a three-dimensional decision fusion of light field features, movement trajectories, and material characteristics;

[0050] Knowledge Storage and Update Layer: Constructing a dual-modal memory pool composed of a gradient direction matrix and an LRU elastic cache, and generating pseudo-samples through a 4-layer fully connected spiking neural network to supplement long-tail categories;

[0051] Long-term adaptation layer: Model the growth dynamics of tea leaves through an LSTM time series network, combine an improved color evolution equation to predict the change in leaf curl and dynamically adjust the spectral tuning parameters, and use StyleGAN-ADA to generate cross-season synthetic images for time-domain data augmentation.

[0052] In one embodiment, the specific steps of using the ultraviolet laser two-photon excitation technology to penetrate the tea leaf surface layer to obtain the deep impurity fluorescence signal:

[0053] Two-photon excitation principle and parameter design: Design a 405nm ultraviolet laser parameter system based on the two-photon excitation principle. By controlling the laser power density (<5mW / μm 2 ), repetition frequency (80MHz), pulse width (100fs), and numerical aperture (0.8), on the premise of ensuring Class 1 safety level, maximize the probability that two photons are simultaneously absorbed by impurity molecules such as lignin / chitin. The product of its two-photon absorption cross-section (σ (2) = 1.2×10 -50 cm 4 ·s / photon) and fluorescence quantum yield (η = 0.15) determines the excitation efficiency;

[0054] Penetration depth and signal detection threshold calculation: Establish a penetration depth model through the fluorescence signal attenuation equation, and combine the laser power (I0), impurity concentration (C_impurity), and tea leaf tissue attenuation coefficient (α = 0.15mm -1 ) monitored in real time to dynamically calculate the maximum detectable depth. The calculation formula is expressed as z_max = 1 / (2α)·ln(I0σ (2) φ 2 ηC_impurity / I_thr);

[0055] Fluorescence signal acquisition and image reconstruction: Use a photomultiplier tube (PMT, H10721-20, anode sensitivity 150A / W) in cooperation with a scanning galvanometer (6215H, scanning speed 1m / s) to reconstruct the fluorescence intensity map through spatio-temporal integration. The fluorescence intensity map is expressed as I_image(x,y) = ∫I_fluor(t)·rect((x - vt) / d)dt;

[0056] Feature fusion: Spatially and temporally align and fuse the fluorescence intensity map with the RGB image and surface normal vector (error <0.1 pixel) to generate a multi-modal feature tensor containing chemical composition and geometric structure.

[0057] In one embodiment, the specific steps of combining a four-way polarization camera and a Mueller matrix inversion algorithm to construct a surface normal vector field to reconstruct a 3D point cloud:

[0058] Four-way polarized image acquisition: Use a four-way polarized camera to capture the light intensity distribution on the tea leaf surface at four polarization directions of 0°, 45°, 90°, and 135°. Utilize the anisotropic characteristics of the interaction between polarized light and matter to encode the surface microstructure information;

[0059] Stokes vector calculation: Convert the four groups of polarized images into a parameter matrix containing the complete polarization state through Stokes vector calculation. The parameter matrix quantitatively describes the light intensity, linear polarization component, and circular polarization characteristics;

[0060] Mueller matrix inversion: The Mueller matrix inversion technique constructs a physical model of the interaction between light and the tea leaf surface, expressed as S_out = M(n)·S. Describe the mapping relationship between the Fresnel reflection coefficient and the surface normal vector through a 4×4 matrix. Establish a non-linear equation system specifically for the reflectivity difference between s-wave and p-wave. Use the Levenberg-Marquardt optimization algorithm to iteratively solve to minimize the difference between the observed Stokes vector and the model prediction value, and invert the surface normal vector field;

[0061] 3D point cloud generation: To convert the normal vector information into three-dimensional space coordinates, integrate the camera calibration parameters and the dynamic light source azimuth field through photometric stereo vision technology. Use integral transformation to convert the normal vector distribution into a height map of the stacked tea leaf surface, and finally generate the spatial point cloud.

[0062] In one embodiment, the data acquisition and preprocessing layer receives the seasonal correction parameters output by the long-term adaptation layer and dynamically adjusts the laser wavelength, expressed as λ_opt = 650 + 30·e^(-0.2W) + ΔT, where W is the humidity sensor reading and ΔT is the seasonal temperature difference influence factor.

[0063] In one embodiment, the specific steps of using the ResNeSt-50 backbone network to extract the light field fusion features and directly output the preliminary classification confidence through the fully connected layer to form the first-layer decision candidates:

[0064] Multi-modal feature fusion: Adopt a multi-modal feature fusion strategy in the input layer to structurally analyze the 64-channel feature map output by the transmissive light field imaging system (including the two-photon fluorescence signal of channels 1-16, the Stokes polarization vector of channels 17-32, the 16-bit precision depth map of channels 33-48, and the motion vector field of channels 49-64), and construct a four-dimensional feature space containing material composition, spatial form, and dynamic information;

[0065] Split-Attention Feature Encoding: Dynamically weight multi-modal features using the channel grouping attention mechanism (dividing 512 base channels into a 4×2 branch structure), and generate attention weights (including ReLU activation and Softmax normalization operations) through global average pooling and a two-layer fully connected network;

[0066] Global Context Modeling: Input the 512-channel feature map output by Split-Attention, reduce the dimension to 64 channels through 1×1 convolution, apply dilated convolution (Dilation Rate = 3) to capture long-range dependencies, restore the spatial resolution through bilinear interpolation, and restore to 512 channels through 1×1 convolution, and then output a context-enhanced feature map;

[0067] Global Average Pooling and Classification Layer: Compress the 512×224×224 feature map into a 512-dimensional vector through global average pooling, generate the original classification confidence through a fully connected layer (including a 9-class output weight matrix) and the Softmax function, and form a primary decision candidate including material discrimination, morphological representation, and motion compensation.

[0068] In one embodiment, the rigid body dynamics simulation branch is combined to predict the blade motion trajectory. After calculating the occlusion probability map through Monte Carlo Dropout sampling, Bayesian correction is performed on the preliminary classification confidence, including the following steps:

[0069] Temporal Prediction Network: Establish the spatio-temporal dependence relationship of blade displacement through a two-layer LSTM temporal network, dynamically update the hidden state using the gated recurrent unit, and finally output the future displacement field and convert it into a spatial constraint coefficient. The spatial constraint coefficient compensates for the positioning deviation caused by conveyor belt vibration by enhancing the classification confidence of moving objects (such as a 1.21-fold gain for the tea stalk category);

[0070] Occlusion Probability Calculation: Use the Monte Carlo Dropout technique to randomly inactivate and sample the joint feature vector, quantify the model uncertainty by calculating the variance of the classification results, and then generate the occlusion probability through the Sigmoid function mapping, expressed as P_occ = 1 - (1 / (1 + e^(-5 / (σ^2 - 0.02)))). Finally, generate a confidence correction coefficient. The correction coefficient and the spatial constraint coefficient output by the dynamics branch form a multiplicative fusion mechanism to achieve probabilistic confidence update under the Bayesian decision framework.

[0071] In one embodiment, the integrated Cook-Torrance BRDF model is used for material reflection correction, and the feature map after material correction is output. The correction amount is converted into a decision threshold offset. The final classification result is generated through a weighted voting mechanism, and the specific steps of three-dimensional decision fusion of light field features, motion trajectories, and material characteristics are as follows:

[0072] Material reflection correction: Query the roughness and Fresnel terms of the target surface through a pre-calibrated material parameter library (such as plastic α = 0.4, tea α = 0.2). Subsequently, use the Cook-Torrance BRDF model to calculate the microplane distribution function D(h) and the geometric attenuation term G(l, v). The calculation formula for the microplane distribution function is expressed as D(h) = α^2 / (π((n·h)^2(α^2 - 1) + 1)^2). After estimating the reflected radiance ΔL through Monte Carlo integration, it is converted into a decision threshold offset ΔT = 0.3ΔL;

[0073] Three-dimensional decision fusion: Through three-dimensional weighted voting of the light field feature confidence, motion trajectory confidence, and material correction confidence, combined with an adaptive weight adjustment mechanism driven by Focal Loss, a fused three-dimensional decision is formed.

[0074] In one embodiment, the specific steps of constructing a dual-modal memory pool composed of a gradient direction matrix and an LRU elastic cache, and generating pseudo-samples by combining a 4-layer fully connected spiking neural network for long-tail categories are as follows:

[0075] Construction and management of the dual-modal memory pool: Construct a gradient direction matrix through a stable memory area and perform exponential weighted average update using a momentum coefficient to achieve stable storage and dynamic optimization of the prototype features of basic categories. At the same time, use the LRU cache mechanism of the elastic memory area and the priority calculation formula to selectively retain newly added long-tail samples. The priority calculation formula is expressed as, and the preliminary accumulation of rare impurity features is completed;

[0076] Generation of pseudo-samples by a spiking neural network (SNN): The 4-layer fully connected SNN simulates biological pulse timing-dependent plasticity (STDP) through the LIF neuron model, performs spatio-temporal encoding on the 64-dimensional feature matrix extracted from the memory pool to generate a pulse sequence, and then decodes it into pseudo-features through spatio-temporal integration. Pseudo-samples are generated through the joint optimization of reconstruction loss and adversarial loss.

[0077] In one embodiment, the steps of modeling the growth dynamics of tea leaves through an LSTM time series network, predicting the change in leaf curl degree by combining an improved color evolution equation, and dynamically adjusting the spectral tuning parameters, and at the same time using StyleGAN-ADA to generate cross-seasonal synthetic images for time-domain data augmentation include the following:

[0078] Modeling the growth dynamics of tea leaves by an LSTM time series network: Use a double-layer LSTM time series network to model the growth dynamics of tea leaves. After inputting historical environmental data (temperature, humidity, light), the network captures temporal dependencies through a peephole connection structure, combined with a discretized curl degree dynamics equation and a mixed loss function (including MSE current state loss and KL divergence probability distribution loss). The discretized curl degree dynamics equation is expressed as Predict the leaf curl and perform the inversion calculation of the color evolution parameter ΔT;

[0079] Inversion of the parameters of the color evolution equation: Establish a seasonal temperature difference influence model through the improved color evolution equation, use the extended Kalman filter for online parameter estimation, and dynamically adjust the laser wavelength compensation amount and exposure time;

[0080] StyleGAN-ADA cross-seasonal data augmentation: Deploy an improved StyleGAN-ADA generator. Through the multi-modal input of splicing spring tea images and autumn tea spectral curves, an 8-layer MLP mapping network and a skip connection synthesis structure, combined with an adaptive discriminator enhancement strategy (dynamic difficulty adjustment probability p = 0.5 + 0.5·tanh[(FID - μ) / σ]), generate cross-seasonal synthetic images that retain the morphological characteristics of spring tea but present the color of autumn tea.

[0081] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An impurity sorting and removing system based on machine vision, characterized in that, Including: Data acquisition and preprocessing layer: Responsible for obtaining multi-dimensional physical characteristics of tea leaves and impurities, constructing a three-dimensional scene representation, using ultraviolet laser two-photon excitation technology to penetrate the tea leaf surface to obtain deep impurity fluorescence signals, combining a four-way polarization camera and a Mueller matrix inversion algorithm to construct a surface normal vector field to reconstruct a 3D point cloud, and generating a high-dimensional feature cube by aligning spectral-polarization-depth three-modal data through the U-Fusion network; Feature analysis and decision-making layer: Using the ResNeSt-50 backbone network to extract light field fusion features, directly outputting the preliminary classification confidence through a fully connected layer to form the first layer of decision candidates, combining a rigid body dynamics simulation branch to predict the leaf movement trajectory, calculating the occlusion probability map through Monte Carlo Dropout sampling, then performing Bayesian correction on the preliminary classification confidence, and at the same time integrating the Cook-Torrance BRDF model for material reflection correction, outputting the feature map after material correction, converting the correction amount into a decision threshold offset, and finally generating the classification result through a weighted voting mechanism to form a three-dimensional decision fusion of light field features, movement trajectories, and material characteristics; Knowledge storage and update layer: Constructing a dual-modal memory pool composed of a gradient direction matrix and an LRU elastic cache, and generating pseudo-samples to supplement long-tail categories by combining a 4-layer fully connected spiking neural network; Long-term adaptation layer: Modeling the tea leaf growth dynamics through an LSTM time series network, combining an improved color evolution equation to predict the change in leaf curl and dynamically adjusting the spectral tuning parameters, and at the same time using StyleGAN-ADA to generate cross-season synthetic images for time-domain data augmentation.

2. The impurity sorting and rejection system based on machine vision according to claim 1, characterized in that, The specific steps of using ultraviolet laser two-photon excitation technology to penetrate the tea leaf surface to obtain deep impurity fluorescence signals: Two-photon excitation principle and parameter design: Designing a 405nm ultraviolet laser parameter system based on the two-photon excitation principle, maximizing the probability that two photons are simultaneously absorbed by impurity molecules by controlling the laser power density, repetition frequency, pulse width, and numerical aperture, and the product of the two-photon absorption cross-section and the fluorescence quantum yield determines the excitation efficiency; Penetration depth and signal detection threshold calculation: A penetration depth model is established through the fluorescence signal attenuation equation. Combining the laser power, impurity concentration, and tea tissue attenuation coefficient monitored in real time, the maximum detectable depth is dynamically calculated. The calculation formula is expressed as z_max = 1 / (2α)·ln(I0σ (2) φ 2 ηC_impurity / I_thr); Fluorescence signal acquisition and image reconstruction: Using a photomultiplier tube to reconstruct the fluorescence intensity map through spatio-temporal integration, and the fluorescence intensity map is expressed as I_image(x,y)=∫I_fluor(t)·rect((x-vt) / d)dt; Feature fusion: Spatially and temporally aligning and fusing the fluorescence intensity map with the RGB image and the surface normal vector to generate a multi-modal feature tensor containing chemical composition and geometric structure.

3. The impurity sorting and removal system based on machine vision according to claim 1, characterized in that, The specific steps of combining a four-way polarization camera and a Mueller matrix inversion algorithm to construct a surface normal vector field to reconstruct a 3D point cloud: Four-way polarization image acquisition: Using a four-way polarization camera to capture the light intensity distribution on the tea leaf surface in four polarization directions of 0°, 45°, 90°, and 135°, and encoding the surface microstructure information using the anisotropic characteristics of the interaction between polarized light and matter; Stokes vector calculation: Through Stokes vector calculation, four groups of polarization images are converted into a parameter matrix containing the complete polarization state, and the parameter matrix quantitatively describes the light intensity, linear polarization component, and circular polarization characteristics; Mueller matrix inversion: The Mueller matrix inversion technology constructs a physical model of the interaction between light and the tea leaf surface, expressed as S_out = M(n)·S. The mapping relationship between the Fresnel reflection coefficient and the surface normal vector is described by a 4×4 matrix. A non-linear equation set is established specifically for the reflectivity difference between the s-wave and p-wave. The Levenberg-Marquardt optimization algorithm is used to iteratively solve to minimize the difference between the observed Stokes vector and the model prediction value, and the surface normal vector field is inverted; 3D point cloud generation: To convert the normal vector information into three-dimensional space coordinates, the photometric stereo vision technology integrates the camera calibration parameters and the dynamic light source azimuth field, and uses integral transformation to convert the normal vector distribution into the height map of the tea leaf stack surface, and finally generates the spatial point cloud.

4. A foreign matter sorting and removing system based on machine vision according to claim 1, characterized in that The data acquisition and preprocessing layer receives the seasonal correction parameters output by the long-term adaptation layer and dynamically adjusts the laser wavelength, expressed as λ_opt = 650 + 30·e^(-0.2W) + ΔT, where W is the humidity sensor reading and ΔT is the seasonal temperature difference influence factor.

5. A foreign matter sorting and removal system based on machine vision according to claim 1, characterized in that, The specific steps of using the ResNeSt-50 backbone network to extract the light field fusion features and directly output the preliminary classification confidence through the fully connected layer to form the first-layer decision candidates: Multi-modal feature fusion: At the input layer, a multi-modal feature fusion strategy is adopted to structurally analyze the 64-channel feature map output by the transmissive light field imaging system, and a four-dimensional feature space containing material composition, spatial form, and dynamic information is constructed; Split-Attention feature encoding: The channel group attention mechanism is used to dynamically weight the multi-modal features, and the attention weights are generated through global average pooling and a two-layer fully connected network; Global context modeling: The 512-channel feature map output by Split-Attention is input, reduced to 64 channels through 1×1 convolution, dilated convolution is applied to capture long-range dependencies, the spatial resolution is restored through bilinear interpolation, and restored to 512 channels through 1×1 convolution, and then the context-enhanced feature map is output; Global average pooling and classification layer: The 512×224×224 feature map is compressed into a 512-dimensional vector through global average pooling, and the original classification confidence is generated through the fully connected layer and the Softmax function, forming the primary decision candidates including material identification, morphological characterization, and motion compensation.

6. The impurity sorting and rejection system based on machine vision according to claim 5, characterized in that, Combining the rigid body dynamics simulation branch to predict the leaf motion trajectory, calculating the occlusion probability map through Monte Carlo Dropout sampling, and then performing Bayesian correction on the preliminary classification confidence, including the following steps: Temporal prediction network: Establish the spatio-temporal dependence relationship of blade displacement through a two-layer LSTM temporal network, use the gated recurrent unit to dynamically update the hidden state, and finally output the future displacement field and convert it into a spatial constraint coefficient. The spatial constraint coefficient compensates for the positioning deviation caused by conveyor belt vibration by strengthening the classification confidence of moving objects; Occlusion probability calculation: Use the Monte Carlo Dropout technique to randomly inactivate and sample the joint feature vector, quantify the model uncertainty by calculating the variance of the classification results, and then generate the occlusion probability through the Sigmoid function mapping, expressed as P_occ = 1 - (1 / (1 + e^(-5 / (σ^2 - 0.02)))), and finally generate a confidence correction coefficient. The correction coefficient and the spatial constraint coefficient output by the dynamics branch form a multiplicative fusion mechanism to achieve probabilistic confidence update under the Bayesian decision framework.

7. The impurity sorting and removal system based on machine vision according to claim 1, wherein, The specific steps of performing material reflection correction on the integrated Cook-Torrance BRDF model, outputting the feature map after material correction, converting the correction amount into a decision threshold offset, and finally generating the classification result through a weighted voting mechanism to form a three-dimensional decision fusion of light field features, motion trajectories, and material properties are as follows: Material reflection correction: Query the roughness and Fresnel terms of the target surface through a pre-calibrated material parameter library, and then use the Cook-Torrance BRDF model to calculate the microplane distribution function D(h) and the geometric attenuation term G(l, v). The calculation formula of the microplane distribution function is expressed as D(h) = α^2 / (π((n·h)^2(α^2 - 1) + 1)^2). After estimating the reflected radiance ΔL through Monte Carlo integration, it is converted into a decision threshold offset ΔT = 0.3ΔL; Three-dimensional decision fusion: Through the three-dimensional weighted voting of the light field feature confidence, motion trajectory confidence, and material correction confidence, combined with the adaptive weight adjustment mechanism driven by FocalLoss, a fused three-dimensional decision is formed.

8. The impurity sorting and rejection system based on machine vision according to claim 1, characterized in that, The specific steps of constructing a dual-modal memory pool composed of a gradient direction matrix and an LRU elastic cache and generating pseudo-samples by combining a 4-layer fully connected spiking neural network to supplement long-tail categories are as follows: Dual-modal memory pool construction and management: Construct a gradient direction matrix through a stable memory area and update it by exponential weighted average using a momentum coefficient. At the same time, use the LRU cache mechanism of the elastic memory area and the priority calculation formula to selectively retain newly added long-tail samples. The priority calculation formula is expressed as, and the preliminary accumulation of rare impurity features is completed; Spiking neural network pseudo-sample generation: The 4-layer fully connected SNN simulates the biological pulse temporal dependence plasticity through the LIF neuron model, performs spatio-temporal encoding on the 64-dimensional feature matrix extracted from the memory pool to generate a pulse sequence, and then decodes it into pseudo-features through spatio-temporal integration, and generates pseudo-samples through the joint optimization of the reconstruction loss and the adversarial loss.

9. The impurity sorting and removal system based on machine vision according to claim 1, characterized in that Modeling the growth dynamics of tea leaves through an LSTM time series network, combining an improved color evolution equation to predict the change in leaf curl and dynamically adjust spectral tuning parameters, and simultaneously using StyleGAN-ADA to generate cross-seasonal synthetic images for time-domain data augmentation, including the following steps: LSTM Time-Series Network Modeling of Tea Growth Kinetics: Using a double-layer LSTM time-series network to model tea growth kinetics, after inputting historical environmental data, the network captures time-series dependencies through a peephole connection structure, combined with a discretized curl kinetics equation and a hybrid loss function. The discretized curl kinetics equation is expressed as Perform prediction of leaf curl and inversion calculation of the color evolution parameter ΔT; Inverse parameter of the color evolution equation: Establish a seasonal temperature difference influence model through an improved color evolution equation, use the extended Kalman filter for online parameter estimation, and dynamically adjust the laser wavelength compensation amount and exposure time; StyleGAN-ADA cross-seasonal data augmentation: Deploy an improved StyleGAN-ADA generator, through the multi-modal input of stitching spring tea images and autumn tea spectral curves, an 8-layer MLP mapping network and a skip connection synthesis structure, combined with an adaptive discriminator enhancement strategy, to generate cross-seasonal synthetic images that retain the morphological characteristics of spring tea but present the color of autumn tea.

Images