A real-time nut sorting system based on a hyperspectral camera

Abstract

This invention relates to the field of quality inspection and discloses a real-time nut quality sorting system based on a hyperspectral camera. The system includes a conveying device, a hyperspectral imaging device, a data processing and analysis module, and an execution module. Hyperspectral imaging acquires a three-dimensional data cube of the nut raw material in a specific wavelength band. The data processing module preprocesses the data, extracts spectral and spatial morphological features, and employs a two-stage cascaded discrimination algorithm for quality determination: the first stage quickly removes morphologically abnormal products based on morphological rules; the second stage accurately identifies internal defects such as mold by calculating the spectral angle and fusing the first derivative and reflectance features into a mold characteristic index. The execution module is a high-pressure jet valve array, which achieves precise removal of defective products through spatiotemporal mapping and delay control. This invention enables simultaneous detection of the internal and external quality of nuts, significantly improving sorting accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the field of quality inspection, and specifically to a real-time nut quality sorting system based on a hyperspectral camera. Background Technology

[0002] In the food processing industry, the quality of nuts (such as sunflower seeds and peanuts) directly affects the quality and safety of the final product. Traditional manual sorting methods suffer from low efficiency, high labor intensity, and subjective sorting standards, making them unsuitable for modern large-scale production. Currently, there are automated sorting devices on the market based on RGB vision technology, but they can only rely on surface features such as color and shape for judgment, and have limited ability to identify invisible defects such as internal mold, minor damage, and early spoilage in nuts.

[0003] Hyperspectral imaging, as a technique that combines imaging and spectral analysis, can acquire spectral information of objects in a continuous narrow wavelength range, thereby reflecting their internal chemical composition and physical structure characteristics. In recent years, hyperspectral technology has been gradually applied to the field of agricultural product testing, and some sorting systems using hyperspectral cameras have emerged.

[0004] However, existing sorting systems of this type still have significant shortcomings in practical applications. Specifically, for complex 3D data acquired by hyperspectral cameras, feature extraction relies heavily on single or limited spectral indices, failing to effectively integrate depth spectral features extracted from continuous spectra (such as specific absorption peak shapes) with spatial morphological features in high-resolution images (such as subtle damage contours), resulting in insufficient sensitivity and discriminative power for complex defects. Simultaneously, at the recognition algorithm level, a single static classification model is often used, making it difficult to balance recognition speed and accuracy, and prone to misjudging boundary samples with ambiguous spectral features or those falling between good and defective products. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time nut quality sorting system based on a hyperspectral camera, thereby solving at least one of the above-mentioned technical problems.

[0006] The objective of this invention can be achieved through the following technical solutions: A real-time nut quality sorting system based on a hyperspectral camera includes: The conveying device is used to uniformly and in a single layer transport the raw nut material to be sorted to the detection area; The hyperspectral imaging device is used to acquire hyperspectral image data of the nut raw materials and obtain a three-dimensional data cube containing spatial information and continuous spectral information. The data processing and analysis module is used to receive and process the hyperspectral image data. The processing flow includes: preprocessing the three-dimensional data cube to obtain corrected spectral reflectance data; extracting spectral feature parameters and spatial morphological parameters characterizing the quality of raw materials from the spectral reflectance data; and determining whether the nut raw materials are defective based on the spectral feature parameters and spatial morphological parameters using a classification and recognition algorithm and outputting sorting instructions. The execution module is used to remove the identified defective nut raw materials from the conveying path according to the sorting instructions.

[0007] As a further technical solution, the preprocessing in the data processing flow includes at least one of dark current correction, flat field correction, and spectral radiometric calibration, in order to eliminate the influence of sensor noise and uneven illumination, and convert the original digital quantization value into standard reflectance data.

[0008] As a further technical solution, the extraction of the spectral feature parameters includes: By analyzing the continuous reflectance spectrum curve, the characteristic wavelengths related to mold and deterioration are located, and the spectral reflectance value at the characteristic wavelength is directly obtained. The identification of spectral inflection points and absorption edges is enhanced by calculating the first derivative spectral value of the continuous reflectance spectral curve at the characteristic wavelength. The calculation of the first derivative spectral value is based on the difference in reflectance at two adjacent sampling wavelengths of the characteristic wavelength, divided by the corresponding wavelength interval.

[0009] As a further technical solution, the classification and recognition algorithm is a two-level cascaded discrimination algorithm: First-level discrimination: Based on the spatial morphology parameters, a preliminary screening is performed using preset morphology rules to remove defective products with obvious morphological abnormalities; Second-level discrimination: For samples that pass the initial screening, the corresponding spectral characteristic parameters are input into a composite spectral analysis model for discrimination; the working process of the composite spectral analysis model is as follows: Based on the spectral reflectance data and the corresponding first derivative spectral values, calculate the spectral angle and mold characteristic index: The spectral angle is obtained by calculating the angle between the spectral reflectance vector of the sample to be tested and the average spectral vector of the pre-stored standard normal sample, and is used to quantify the similarity of the spectral curve shape. The mold growth characteristic index is obtained by calculating the difference between the first derivative spectral values ​​at two selected characteristic wavelengths and then dividing it by the normalized difference index constructed from the reflectance at two other selected characteristic wavelengths; this difference index is used to fuse composite information reflecting the difference in spectral change rate and the absorption characteristics of a specific band. If the mold growth characteristic index is greater than the first high threshold, it is directly determined to be a defective product; If the moldy characteristic index is less than the first low threshold, it is directly determined to be a normal product; If the mold characteristic index is between the first low threshold and the first high threshold, then a comprehensive judgment is made: the spectral angle is compared with the second threshold. If the spectral angle is greater than the second threshold, it is judged as a defective product; otherwise, it is judged as a normal product. If the product is determined to be defective based on the judgment result, the sorting instruction is triggered.

[0010] As a further technical solution, the execution module is a high-pressure jet valve array arranged laterally along the conveying device; the data processing and analysis module drives the execution module through the following control process: Based on the fixed installation distance L between the hyperspectral imaging device and the high-pressure jet valve array along the conveying direction, and the real-time operating speed v of the conveying device, the time delay from detection to trigger is calculated as t = L / v. The pixel coordinates of the identified defective nut raw materials in the hyperspectral image data are combined with the pre-calibrated parameters of the hyperspectral imaging device and converted into actual physical position coordinates on the plane of the conveying device. The corresponding spray valve unit is determined based on the actual physical location coordinates, and a trigger signal is sent to the spray valve unit after the time delay t to spray high-pressure airflow, so as to accurately blow the defective nut raw materials away from the conveying path.

[0011] As a further technical solution, the data processing and analysis module also includes: The real-time monitoring unit is used to calculate and monitor local anomaly indices; The local anomaly index is calculated by the anomaly score of the current nut raw material and multiple consecutive preceding samples; wherein, the anomaly score of a single sample is generated by mapping the spectral angle calculated by the composite spectral analysis model with the mold characteristic index. When the local anomaly index continues to exceed the first preset threshold, the data processing and analysis module determines that there is a quality risk in the current detection area and outputs an area removal instruction to the execution module. Upon receiving the area rejection instruction, the execution module activates the enhanced rejection mode to indiscriminately reject all materials currently in the detection area and the subsequent conveying path.

[0012] As a further technical solution, the system also includes: The spray valve health management module is used to monitor and maintain the working status of the high-pressure jet blow valve array; specifically, it includes: Record the cumulative number of operations and the continuous working time within a set period for each spray valve unit; calculate the real-time health score for each spray valve unit by weighted summation based on the cumulative number of operations and the continuous working time. When a sorting instruction needs to be triggered, the data processing and analysis module selects the valve unit with the highest real-time health score to perform the rejection action; When the real-time health score of any spray valve unit is lower than the preset maintenance threshold, the spray valve health management module automatically marks the spray valve unit as needing maintenance, and the data processing and analysis module suspends the use of the spray valve unit in subsequent sorting.

[0013] As a further technical solution, the data processing and analysis module also includes a dynamic sensitivity adjustment unit; specifically including: Real-time statistics are provided on the proportion of defective products rejected by the first level of identification and the proportion of defective products rejected by the second level of identification in the current batch of nut raw materials. When the rejection rate of the first stage continues to be higher than the preset third threshold, the area low threshold, aspect ratio threshold and / or density threshold in the first stage discrimination are lowered to reduce the rejection of good products due to overly strict sorting. When the second-level rejection ratio continues to be lower than the preset fourth threshold, the first high threshold and / or the second threshold of the mold characteristic index in the composite spectral analysis model are raised to tighten the spectral discrimination criteria and prevent minor defects from being missed.

[0014] The beneficial effects of this invention are: This invention acquires continuous spectral information through hyperspectral imaging technology and combines it with classification and recognition algorithms. It can not only reliably identify internal chemical changes such as mold and deterioration, but also detect external morphological defects such as fragments, deformities, and damage, overcoming the limitations of traditional RGB sorting and single spectral models. Among them, the first-level morphological screening quickly removes obvious defects, reducing the computational load of the second-level spectral analysis and ensuring the real-time performance of the system on high-speed production lines. At the same time, the local anomaly index monitoring and regional elimination mechanism enables the system to intelligently perceive areas where quality risks are concentrated and adopt batch processing strategies, which not only prevents cross-contamination, but also improves the overall processing efficiency. Attached Figure Description

[0015] The invention will now be further described with reference to the accompanying drawings.

[0016] Figure 1 This is a system structure block diagram of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This system is deployed in an assembly line configuration. The conveying device uses a food-grade belt conveyor equipped with a vibrating feeder and height-limiting baffles to ensure that the raw nuts pass through the detection area in the middle section at a constant speed in a single layer. A hyperspectral imaging device, covering a spectral range of 400-1000nm, is fixedly installed directly above the detection area and equipped with a linear light source to provide stable and uniform illumination. The data processing and analysis module (industrial computer) is connected to the hyperspectral camera via a high-speed data cable to receive hyperspectral data cubes in real time. The execution module consists of a set of high-pressure electromagnetic spray valves arranged laterally at the end of the belt, controlled by sorting instructions issued by the data processing module.

[0019] Please see Figure 1 As shown, the present invention is a real-time nut quality sorting system based on a hyperspectral camera, comprising: The conveying device is used to uniformly and in a single layer transport the raw nut material to be sorted to the detection area; A hyperspectral imaging device is used to acquire hyperspectral image data of the nut raw materials in the 400-1000nm band and obtain a three-dimensional data cube containing spatial information and continuous spectral information. The data processing and analysis module is used to receive and process the hyperspectral image data. The processing flow includes: The three-dimensional data cube is preprocessed to obtain corrected spectral reflectance data; specifically, it includes at least one of dark current correction, flat field correction, and spectral radiometric calibration to eliminate the effects of sensor noise and uneven illumination, and to convert the original digital quantization values ​​into standard reflectance data.

[0020] Preprocessing is performed immediately after data reception. Dark current correction is performed after each power-on warm-up, with the lens cap on, by acquiring a dark background image; all subsequent images are subtracted from this dark current background. Flat-field correction uses a standard diffuse reflection white board (such as a PTFE board) placed in the detection area, a reference image is acquired, and the original image is divided by the reference image to eliminate uneven illumination and lens vignetting. Spectral radiometric calibration is performed by acquiring a series of standard color charts with known reflectance to establish a linear mapping relationship between the DN values ​​of each band of the camera and the absolute reflectance, thus completing the radiometric calibration. Preprocessing eliminates sensor dark noise, uneven illumination distribution, and system response differences, transforming the original grayscale values ​​into physically meaningful standard reflectance data as the basis for subsequent analysis; it also reduces the impact of slight changes in ambient light or slow drift of the equipment itself on the detection results, improving the long-term reliability of the system.

[0021] Spectral characteristic parameters and spatial morphological parameters characterizing the quality of raw materials are extracted from the spectral reflectance data. Based on the spectral characteristic parameters and spatial morphological parameters, a classification and identification algorithm is used to determine whether the nut raw materials are defective and output a sorting instruction. The execution module is used to remove the identified defective nut raw materials from the conveying path according to the sorting instructions.

[0022] The extraction of the spectral feature parameters includes: By analyzing the continuous reflectance spectrum curve, the characteristic wavelengths related to mold and deterioration are located, and the spectral reflectance value at the characteristic wavelength is directly obtained. The identification of spectral inflection points and absorption edges is enhanced by calculating the first derivative spectral value of the continuous reflectance spectral curve at the characteristic wavelength. The calculation of the first derivative spectral value is based on the difference in reflectance at two adjacent sampling wavelengths of the characteristic wavelength, divided by the corresponding wavelength interval.

[0023] Specifically, the characteristic wavelengths are determined through the following steps: collecting hyperspectral data of normal and defective samples in known states; calculating the average spectral curves of each type of sample and comparing the differences; using algorithms such as continuous projection, competitive adaptive reweighted sampling, or partial least squares discriminant analysis to screen out the wavelength variables that contribute most to distinguishing normal and defective samples from the entire band; finally, determining a set of optimal characteristic wavelengths for real-time sorting through model validation. Specifically, the selected candidate characteristic wavelength combinations are input into a preset classification and recognition model (such as support vector machine, random forest, or deep learning-based classifier), and the classification performance indicators (such as accuracy, recall, and specificity) of the model under different wavelength combinations are evaluated using an independent validation sample set; by comparing the classification performance indicators, a set of wavelengths that achieves optimal model performance or meets a preset threshold while ensuring real-time requirements is selected as the finally determined characteristic wavelengths. For example, the characteristic wavelengths for mold detection include the bands around 720nm and 850nm.

[0024] By selecting a small number of key wavelengths from hundreds of spectral bands, the subsequent computation is significantly reduced, meeting the real-time requirements. At the same time, redundant and noisy bands are removed, making the features more discriminative. Meanwhile, the first derivative spectrum can effectively amplify the weak inflection points and absorption edge information in the original reflection spectrum. These are often characteristics of early mold growth or chemical changes, thereby improving the ability to detect internal or early defects that are difficult to detect by the human eye and traditional images.

[0025] The classification and recognition algorithm is a two-level cascaded discrimination algorithm: First-level discrimination: Based on the spatial morphology parameters, a preliminary screening is performed using preset morphology rules to remove defective products with obvious morphological abnormalities; The specific implementation process is as follows: The purpose of this stage of discrimination is to utilize the high spatial resolution of hyperspectral images to quickly eliminate nut raw materials with obvious morphological defects, such as fragments, severe damage, deformities, and multiple adhering nuts, thereby reducing the burden of the complex spectral analysis in the second stage.

[0026] After extracting the spectral reflectance data, the hyperspectral images of the same spatial location (usually a characteristic band image, such as an image near 700nm, is selected due to its good contrast) are simultaneously binarized and segmented to separate the nut-like target from the background. Subsequently, the following core morphological parameters are calculated for each connected component (i.e., a suspected nut-like particle): Area: The total number of target pixels, reflecting particle size; Aspect ratio of the bounding rectangle: The ratio of the longer side to the shorter side of the bounding rectangle, used to identify abnormally long or flat particles; Circularity: Calculated as 4π * area / P 2Where P is the target perimeter. The closer the roundness is to 1, the closer the shape is to a perfect circle; compactness: defined as area / convex hull area, this parameter can effectively identify severely concave or C-shaped damaged particles.

[0027] The system has a pre-configured morphological rule base. The thresholds are not completely fixed, but are fine-tuned based on sampling statistics of the current batch of raw materials. The specific rules are as follows: Rule 1 (Fragment Rejection): If the particle area is less than the minimum area threshold, it is judged as a small fragment or dander and directly marked as a defective product. The minimum area threshold can be determined by statistically analyzing 40% of the average area of ​​normal particles.

[0028] Rule 2 (Deformity Rejection): If the aspect ratio of the bounding rectangle of a nut is greater than the aspect ratio threshold (e.g., 2.0), or the roundness is less than the roundness threshold (e.g., 0.7), it is judged as a deformed nut and directly marked as a defective product. The aspect ratio threshold, roundness threshold, and density threshold are all preset based on the statistical distribution of morphological parameters of normal nut samples. Rule 3 (Rejection of Damaged Products): If the particle density is less than the density threshold (e.g., 0.85), it indicates that there is a serious indentation in its outline, and it is judged as a damaged product and directly marked as a defective product.

[0029] Rule 4 (Adhesion Handling): If the particle area is greater than the area threshold, which is usually 180% of the average area, it is initially judged as a possible multi-particle adhesion.

[0030] The system then performs concave point detection and segmentation on the large connected region: first, it calculates the convex hull defects of its contour, locates the concave points (i.e., the gaps between particles), and then uses these points as seeds to perform watershed algorithm or contour segmentation to separate it into multiple independent particles. The segmented particles will re-enter the first-level discrimination process; if the algorithm determines that it is a severely adhered block that cannot be effectively segmented, it will be directly rejected as a defective product.

[0031] The coordinates of all particles marked as defective by the above rules will be immediately transmitted to the execution module, triggering a sorting instruction. Only when a particle passes all morphological rule checks and is determined to be a normal morphological unit will its corresponding spatial coordinates be used to accurately extract the continuous reflectance spectrum curve of the nut from the three-dimensional hyperspectral data cube, and then sent to the second stage for in-depth spectral feature analysis and discrimination.

[0032] The first-level discrimination acts as a fast filter, utilizing simple image processing and rule-based judgment to quickly eliminate obviously defective items (fragments, deformities, and breaks) and indivisible aggregates. On the one hand, this greatly reduces the computational burden of the subsequent complex second-level spectral analysis, allowing the system to concentrate its valuable computational resources on the precise discrimination of samples that are morphologically normal but may have internal deterioration, achieving an optimal balance between sorting speed and accuracy. On the other hand, by pre-separating aggregated particles, it ensures that the spectral signals input to the second-level analysis all come from an independent, complete nut, avoiding misjudgments caused by signal mixing and guaranteeing the accuracy of the second-level discrimination from the data source.

[0033] Second-level discrimination: For samples that pass the initial screening, the corresponding spectral characteristic parameters are input into a composite spectral analysis model for discrimination; the working process of the composite spectral analysis model is as follows: Based on the spectral reflectance data and the corresponding first derivative spectral values, calculate the spectral angle and mold characteristic index: The spectral angle is obtained by calculating the angle between the spectral reflectance vector of the sample to be tested and the average spectral vector of the pre-stored standard normal sample, and is used to quantify the similarity of the spectral curve shape. The reflectance spectrum of the sample in the 400-1000nm range is regarded as a multidimensional vector. The cosine of the angle between the vector and the average spectral vector of the standard healthy nuts is calculated and then converted into an angle θ. The smaller the θ value, the more similar the spectral shape.

[0034] The mold growth characteristic index is obtained by calculating the difference between the first derivative spectral values ​​at two selected characteristic wavelengths and then dividing it by the normalized difference index constructed from the reflectance at two other selected characteristic wavelengths; this difference index is used to fuse composite information reflecting the difference in spectral change rate and the absorption characteristics of a specific band. The specific expression is: MDI = (D1 - D2) / (ND + ε); D1 and D2 are the first derivative values ​​at the first characteristic wavelength λ1 and the second characteristic wavelength λ2 determined through feature screening, respectively; ND is the normalized difference exponent calculated at the other two selected characteristic wavelengths λ3 and λ4, and its value is: ND = (R3 - R4) / (R3 + R4), where R3 and R4 are the spectral reflectance values ​​at wavelengths λ3 and λ4, respectively; ε is a very small positive constant (e.g., 1 × 10⁻⁶). -6 (), used to prevent the denominator from being zero and to ensure calculation stability.

[0035] The numerator, D1-D2, quantifies the difference in spectral change rates at the key bands λ1 and λ2, and is sensitive to subtle spectral inflection points caused by mold. The normalized difference index ND in the denominator effectively amplifies the changes in reflectance absorption characteristics caused by mold in the λ3 and λ4 bands. Therefore, the mold characteristic index MDI, by combining the two, constitutes a comprehensive discriminant index that is highly specific to nut mold.

[0036] The specific values ​​of the characteristic wavelengths λ1, λ2, λ3, and λ4 can be determined by analyzing the differences in the spectra and derivative spectra of normal and moldy samples, and by using methods such as continuous projection algorithms or analysis of variance. In a preferred embodiment, λ1 = 720 nm and λ2 = 760 nm are selected, and their first derivative values ​​D1 and D2 are calculated; λ3 = 850 nm and λ4 = 900 nm are selected, and the reflectances R3 and R4 are obtained, and ND = (R3 - R4) / (R3 + R4) is calculated; ε is set to 1 × 10⁻¹⁰. -6 Calculate MDI = (D1 - D2) / (ND + ε). If MDI > 2.5 (first high threshold), it is judged as severely moldy; if MDI < 0.8 (first low threshold), it is judged as normal; if 0.8 ≤ MDI ≤ 2.5, then judge in combination with the spectral angle θ: if θ > 0.15 radians (second threshold), it is judged as spectral abnormality and is a defective product; otherwise, it is a normal product.

[0037] The first low threshold can be set to the 95th percentile of the MDI value of a normal sample, and the first high threshold can be set to the 5th percentile of the MDI value of a moldy sample; the second threshold can be set to the 99th percentile of the spectral angle θ value of a normal sample.

[0038] By constructing the Mold Characteristic Index (MDI), the first derivative information reflecting the difference in spectral change rate is combined with the reflectance normalized difference index reflecting specific absorption characteristics, forming a comprehensive index that is highly specific and sensitive to nut mold.

[0039] If the mold growth characteristic index is greater than the first high threshold, it is directly determined to be a defective product; If the moldy characteristic index is less than the first low threshold, it is directly determined to be a normal product; If the mold characteristic index is between the first low threshold and the first high threshold, then a comprehensive judgment is made: the spectral angle is compared with the second threshold. If the spectral angle is greater than the second threshold, it is judged as a defective product; otherwise, it is judged as a normal product. If the product is determined to be defective based on the judgment result, the sorting instruction is triggered.

[0040] On the one hand, it allows for rapid and clear decisions on samples with extremely high or low MDI values; on the other hand, for difficult samples with MDI values ​​in the intermediate ambiguity zone, a secondary verification is performed using the spectral angle θ. The θ value provides supplementary information from the macroscopic perspective of the overall spectral shape similarity. This effectively solves the industry problem of easy misjudgment of boundary samples by single features or simple models. While ensuring a high recall rate (detecting defects), it achieves a very low false alarm rate (falsely rejecting good products), and the overall sorting accuracy is significantly improved.

[0041] The execution module is a high-pressure jet valve array arranged laterally along the conveying device; the data processing and analysis module drives the execution module through the following control process: Based on the fixed installation distance L between the hyperspectral imaging device and the high-pressure jet valve array along the conveying direction, and the real-time operating speed v of the conveying device, the time delay from detection to trigger is calculated as t = L / v. The pixel coordinates of the identified defective nut raw materials in the hyperspectral image data are combined with the pre-calibrated parameters of the hyperspectral imaging device and converted into actual physical position coordinates on the plane of the conveying device. The corresponding spray valve unit is determined based on the actual physical location coordinates, and a trigger signal is sent to the spray valve unit after the time delay t to spray high-pressure airflow, so as to accurately blow the defective nut raw materials away from the conveying path.

[0042] By installing at a fixed distance, monitoring speed in real time, accurately mapping coordinates, and dynamically compensating for delays, the coordinates of defective products in the information space identified by the upstream algorithm are synchronized with the rejection action in the physical space of the downstream actuator under high-speed conditions. This ensures that the high-pressure spray valve can hit the moving defective product at the accurate position within an extremely short response window.

[0043] For example, the control flow of the execution module is implemented through a precision timing control card. Assuming the mechanical center distance between the camera and the spray valve array is L = 1.2 meters, and the belt speed is v = 0.5 m / s, then the fixed delay t = L / v = 2.4 seconds. The system reads the belt speed v in real time through an encoder and dynamically updates t. When a defective product is detected, based on its pixel coordinates (x, y) in the hyperspectral image, it is transformed into a two-dimensional physical coordinate system (xb, yb) with the belt center as the origin using the intrinsic and extrinsic parameter matrices calibrated by the camera. Based on the value of yb, it is mapped to a specific one of the 16 horizontally arranged spray valves. At the detection moment, the system starts a timer, and after a dynamically calculated delay time t, sends a millisecond-wide trigger pulse to the designated spray valve, driving it to open and blowing the defective product into the side waste chute. Through spatial coordinate mapping and precise time-delay triggering, even when the raw nut material moves on a high-speed belt, the spray valve can be ensured to operate in the correct position and at the precise moment, achieving an extremely high single-nut rejection accuracy. Real-time speed feedback and dynamic delay adjustment functions enable the system to adapt to the acceleration or deceleration of the production line or speed fine-tuning, maintaining the stability and reliability of the sorting action.

[0044] The data processing and analysis module also includes: The real-time monitoring unit is used to calculate and monitor local anomaly indices; The local anomaly index is calculated by the anomaly score of the current nut raw material and multiple consecutive preceding samples; wherein, the anomaly score of a single sample is generated by mapping the spectral angle calculated by the composite spectral analysis model with the mold characteristic index. When the local anomaly index continues to exceed the first preset threshold, the data processing and analysis module determines that there is a quality risk in the current detection area and outputs an area removal instruction to the execution module. It should be noted that during normal sorting, the local anomaly index data corresponding to the known qualified production batches are recorded, the upper limit of the statistical distribution is calculated (such as the mean plus 3 times the standard deviation), and this upper limit value is set as the first preset threshold. Upon receiving the area rejection instruction, the execution module activates the enhanced rejection mode to indiscriminately reject all materials currently in the detection area and the subsequent conveying path.

[0045] For example, the real-time monitoring unit maintains a queue containing the 100 most recent samples. For each sample, an anomaly score AS is calculated based on its spectral angle θ and mold characteristic index MDI. The anomaly score AS is obtained by standardizing the spectral angle θ and the mold characteristic index MDI, and then calculating the Euclidean distance between the sample and the center point of the normal sample. This value comprehensively quantifies the degree to which a single sample deviates from the normal population in terms of spectral shape and mold characteristics. The specific expression is: , where the mean , with standard deviation , All calculations are based on historical normal samples. The Local Anomaly Index (LAI) is defined as the moving average of the AS values ​​of the current sample and its nine preceding samples (out of ten). A warning threshold of 2.0 is set for LAI. When the system detects that the LAI exceeds 2.0 three times consecutively (i.e., 30 consecutive samples show high overall anomaly), it determines that the material in that area has a risk of systemic contamination or spoilage. At this point, the data processing module no longer individually checks subsequent materials entering the area but instead sends a 500-millisecond area rejection signal to the execution module. During this period, multiple spray valves in the corresponding physical section remain open, blowing away all material on that section of the conveyor belt, regardless of quality. When a clustering trend of defects is detected, decisive overall area rejection is implemented to prevent spores or contaminants from a few moldy nuts from contaminating a large number of good products in subsequent processes, thus improving the safety and hygiene level of the final product. Furthermore, it not only focuses on the quality of individual nuts but also senses macroscopic quality fluctuation trends in the production flow and takes corresponding upgrade measures, demonstrating higher efficiency in quality control.

[0046] The system also includes: The spray valve health management module is used to monitor and maintain the working status of the high-pressure jet blow valve array; specifically, it includes: Record the cumulative number of operations and the continuous working time within a set period for each spray valve unit; calculate the real-time health score for each spray valve unit by weighted summation based on the cumulative number of operations and the continuous working time. When a sorting instruction needs to be triggered, the data processing and analysis module selects the valve unit with the highest real-time health score to perform the rejection action; When the real-time health score of any spray valve unit is lower than the preset maintenance threshold, the spray valve health management module automatically marks the spray valve unit as needing maintenance, and the data processing and analysis module suspends the use of the spray valve unit in subsequent sorting.

[0047] The system continuously records two key operating parameters for each independent spray valve unit in the high-pressure jet blow valve array: the cumulative number of operations of the spray valve unit, i.e., the total number of spray actions performed since activation; and the continuous operating time within a set period (e.g., the most recent 24 hours or a single production shift), i.e., the total duration for which the spray valve remains open during a single trigger. Based on these two key operating parameters, a weighted summation formula is used to calculate the real-time health score H of each spray valve unit. The specific formula can be designed as: H=α*(Nmax-Nt) / Nmax+β*(Tmax-Tt) / Tmax, where Nt is the cumulative number of operations, Nmax is the rated lifespan of the spray valve model, Tt is the continuous operating time within the period, Tmax is the recommended maximum allowable continuous operating time, and α and β are weighting coefficients, with α+β=1. Through experiments, these can be set to 0.6 and 0.4 respectively to better emphasize the impact of accumulated mechanical fatigue. The system continuously calculates and updates the H of all spray valves in the background and sorts them. When the data processing and analysis module needs to trigger a sorting command based on the judgment results, its control logic does not select the spray valves randomly or in a fixed order. Instead, it queries the current health score list and prioritizes scheduling the spray valve unit with the highest H to perform the rejection action, thereby achieving intelligent and balanced workload distribution. Simultaneously, the system presets a maintenance threshold Hd (e.g., 0.2). When the H of any spray valve unit is detected to be below this threshold, the health management module immediately marks it as red and in a maintenance-pending state on the software interface and sends a warning notification. More importantly, the data processing and analysis module simultaneously adds the spray valve unit to the disabled list, preventing it from being assigned tasks in subsequent sorting decisions until maintenance personnel complete the inspection and manually reset its status. Firstly, it achieves a leap from post-fault repair to pre-emptive predictive maintenance. By quantitatively assessing the fatigue and overheating risk of the spray valves, it can provide early warnings and maintenance before their performance degrades to the point of affecting rejection accuracy or complete failure, greatly reducing the risk of downtime and product loss on the production line due to sudden actuator failures. Secondly, by prioritizing the use of the spray valve in the best health condition, it ensures that each rejection action is completed by the most reliable actuator at the moment. This is crucial for maintaining the consistency of rejection accuracy in high-speed, continuous sorting processes and avoids problems such as missed rejection or incomplete rejection caused by delays in response of individual spray valves or weakened airflow.

[0048] The data processing and analysis module further includes a dynamic sensitivity adjustment unit; specifically, it includes: Real-time statistics are provided on the proportion of defective products rejected by the first level of identification and the proportion of defective products rejected by the second level of identification in the current batch of nut raw materials. When the rejection rate of the first stage continues to be higher than the preset third threshold, the area low threshold, aspect ratio threshold and / or density threshold in the first stage discrimination are lowered to reduce the rejection of good products due to overly strict sorting. When the second-level rejection ratio continues to be lower than the preset fourth threshold, the first high threshold and / or the second threshold of the mold characteristic index in the composite spectral analysis model are raised to tighten the spectral discrimination criteria and prevent minor defects from being missed.

[0049] The core function of this unit is to monitor and analyze the real-time sorting results of the two-stage discrimination process of the system, and adaptively adjust the discrimination threshold accordingly. The implementation process is as follows: During operation, the system continuously collects two key indicators: the proportion of nuts rejected by the first-stage discrimination (based on morphological rules) out of the total processed volume (Pm), and the proportion of nuts rejected by the second-stage discrimination (based on a composite spectral analysis model) out of the total processed volume (Ps). These two proportions are calculated on a rolling basis over a settable time window (e.g., after processing the most recent 1000 nuts) to reflect the immediate characteristics of the current batch of raw materials. The dynamic sensitivity adjustment unit has two preset adjustment thresholds: a third threshold (h, e.g., 0.15) and a fourth threshold (l, e.g., 0.02). When the system detects that the rolling average of Pm continuously exceeds h (e.g., exceeding the threshold for three consecutive time windows), it determines that the proportion of raw materials rejected due to morphological irregularities is abnormally high. This may stem from batch-specific differences in the size and shape of the raw materials themselves, rather than a sudden increase in the defect rate. In this case, continuing to use the original strict morphological standards would lead to too many good products being mistakenly rejected. Therefore, the unit automatically triggers a downward adjustment command, slightly lowering the area threshold, aspect ratio threshold, and density threshold used in the first-level discrimination rules by a preset step size (e.g., 5%), thus appropriately relaxing the morphological screening criteria. Conversely, when the rolling average of Ps is consistently below 1, it indicates that the proportion of defects detected by spectral analysis is too low, potentially indicating a risk of insensitivity to defects such as slight mold. In this case, the unit automatically triggers an upward adjustment command, slightly increasing the first high threshold of the mold characteristic index used in the second-level discrimination and / or the second threshold used for comparison with the spectral angle by a step size, making the spectral discrimination criteria more stringent. On the one hand, it can dynamically fine-tune the sorting scale according to the actual situation of each batch of raw materials (such as natural morphological variations caused by different origins, varieties, and harvest years), maximizing the optimization of the contradictory indicators of false rejection rate and missed rejection rate while ensuring core quality requirements, thereby directly improving the yield of high-quality raw materials. On the other hand, it effectively reduces the over-reliance on the accuracy of preset parameters and the reliance on operator experience. Even in the face of normal fluctuations in raw material characteristics, it can automatically stabilize the sorting effect near the optimal range, greatly reducing the manual calibration work and parameter exploration time required for batch switching, and improving the operational efficiency and stability of the production line.

[0050] It should be noted that the specific characteristic wavelengths, threshold parameters, calculation formula constants, etc., listed in the embodiments of this invention are all preferred examples derived from specific experimental samples and are not intended to limit the invention. In practical applications, they can be re-determined through similar model training and statistical methods according to different nut varieties, origins, and production conditions.

[0051] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A real-time nut quality sorting system based on a hyperspectral camera, characterized in that, include: The conveying device is used to uniformly and in a single layer transport the raw nuts to be sorted to the detection area; The hyperspectral imaging device is used to acquire hyperspectral image data of the nut raw materials and obtain a three-dimensional data cube containing spatial information and continuous spectral information. The data processing and analysis module is used to receive and process the hyperspectral image data. The processing flow includes: preprocessing the three-dimensional data cube to obtain corrected spectral reflectance data; extracting spectral feature parameters and spatial morphological parameters characterizing the quality of raw materials from the spectral reflectance data; and determining whether the nut raw materials are defective based on the spectral feature parameters and spatial morphological parameters using a classification and recognition algorithm and outputting sorting instructions. The execution module is used to remove the identified defective nut raw materials from the conveying path according to the sorting instructions.

2. The real-time nut quality sorting system based on a hyperspectral camera according to claim 1, characterized in that, The preprocessing in the data processing flow includes at least one of dark current correction, flat field correction, and spectral radiometric calibration to eliminate the effects of sensor noise and uneven illumination, and to convert the original digital quantization values ​​into standard reflectance data.

3. The real-time nut quality sorting system based on a hyperspectral camera according to claim 2, characterized in that, The extraction of the spectral feature parameters includes: By analyzing the continuous reflectance spectrum curve, the characteristic wavelengths related to mold and deterioration are located, and the spectral reflectance value at the characteristic wavelength is directly obtained. The identification of spectral inflection points and absorption edges is enhanced by calculating the first derivative spectral value of the continuous reflectance spectral curve at the characteristic wavelength. The calculation of the first derivative spectral value is based on the difference in reflectance at two adjacent sampling wavelengths of the characteristic wavelength, divided by the corresponding wavelength interval.

4. The real-time nut quality sorting system based on a hyperspectral camera according to claim 3, characterized in that, The classification and recognition algorithm is a two-level cascaded discrimination algorithm: First-level discrimination: Based on the spatial morphology parameters, a preliminary screening is performed using preset morphology rules to remove defective products with obvious morphological abnormalities; Second-level discrimination: For samples that pass the initial screening, the corresponding spectral characteristic parameters are input into a composite spectral analysis model for discrimination; the working process of the composite spectral analysis model is as follows: Based on the spectral reflectance data and the corresponding first derivative spectral values, calculate the spectral angle and mold characteristic index: The spectral angle is obtained by calculating the angle between the spectral reflectance vector of the sample to be tested and the average spectral vector of the pre-stored standard normal sample, and is used to quantify the similarity of the spectral curve shape. The mold growth characteristic index is obtained by calculating the difference between the first derivative spectral values ​​at two selected characteristic wavelengths and then dividing it by the normalized difference index constructed from the reflectance at two other selected characteristic wavelengths; this difference index is used to fuse composite information reflecting the difference in spectral change rate and the absorption characteristics of a specific band. If the mold growth characteristic index is greater than the first high threshold, it is directly determined to be a defective product; If the moldy characteristic index is less than the first low threshold, it is directly determined to be a normal product; If the mold characteristic index is between the first low threshold and the first high threshold, then a comprehensive judgment is made: the spectral angle is compared with the second threshold. If the spectral angle is greater than the second threshold, it is judged as a defective product; otherwise, it is judged as a normal product. If the product is determined to be defective based on the judgment result, the sorting instruction is triggered.

5. The real-time nut quality sorting system based on a hyperspectral camera according to claim 1, characterized in that, The execution module is a high-pressure jet valve array arranged laterally along the conveying device; the data processing and analysis module drives the execution module through the following control process: Based on the fixed installation distance L between the hyperspectral imaging device and the high-pressure jet valve array along the conveying direction, and the real-time operating speed v of the conveying device, the time delay t = L / v from detection to trigger is calculated. The pixel coordinates of the identified defective nut raw materials in the hyperspectral image data are combined with the pre-calibrated parameters of the hyperspectral imaging device and converted into actual physical position coordinates on the plane of the conveying device. The corresponding spray valve unit is determined based on the actual physical location coordinates, and a trigger signal is sent to the spray valve unit after the time delay t to spray high-pressure airflow, so as to accurately blow the defective nut raw materials away from the conveying path.

6. The real-time nut quality sorting system based on a hyperspectral camera according to claim 4, characterized in that, The data processing and analysis module also includes: The real-time monitoring unit is used to calculate and monitor local anomaly indices; The local anomaly index is calculated by the anomaly score of the current nut raw material and multiple consecutive preceding samples; wherein, the anomaly score of a single sample is generated by mapping the spectral angle calculated by the composite spectral analysis model with the mold characteristic index. When the local anomaly index continues to exceed the first preset threshold, the data processing and analysis module determines that there is a quality risk in the current detection area and outputs an area removal instruction to the execution module. Upon receiving the area rejection instruction, the execution module activates the enhanced rejection mode to indiscriminately reject all materials currently in the detection area and the subsequent conveying path.

7. The real-time nut quality sorting system based on a hyperspectral camera according to claim 5, characterized in that, The system also includes: The spray valve health management module is used to monitor and maintain the working status of the high-pressure jet blow valve array; specifically, it includes: Record the cumulative number of operations and the continuous working time within a set period for each spray valve unit; calculate the real-time health score for each spray valve unit by weighted summation based on the cumulative number of operations and the continuous working time. When a sorting instruction needs to be triggered, the data processing and analysis module selects the valve unit with the highest real-time health score to perform the rejection action; When the real-time health score of any spray valve unit is lower than the preset maintenance threshold, the spray valve health management module automatically marks the spray valve unit as needing maintenance, and the data processing and analysis module suspends the use of the spray valve unit in subsequent sorting.

8. The real-time nut quality sorting system based on a hyperspectral camera according to claim 4 or 7, characterized in that, The data processing and analysis module further includes a dynamic sensitivity adjustment unit; specifically, it includes: Real-time statistics are provided on the proportion of defective products rejected by the first level of identification and the proportion of defective products rejected by the second level of identification in the current batch of nut raw materials. When the rejection rate of the first stage continues to be higher than the preset third threshold, the area low threshold, aspect ratio threshold and / or density threshold in the first stage discrimination are lowered to reduce the rejection of good products due to overly strict sorting. When the second-level rejection ratio continues to be lower than the preset fourth threshold, the first high threshold and / or the second threshold of the mold characteristic index in the composite spectral analysis model are raised to tighten the spectral discrimination criteria and prevent minor defects from being missed.

Images