Method for identifying walnut kernel surface defects based on hyperspectral imaging technology

Abstract

The invention discloses a method for identifying walnut kernel surface defects based on a hyperspectral imaging technology. The method comprises the following steps: S1, obtaining a walnut kernel hyperspectral image and removing interference noise; s2, qualitatively analyzing the walnut kernels with different defect types through two-dimensional correlation spectrum analysis; the identification capability of rotten walnut kernels is further enhanced through visual analysis of the thermodynamic diagram; and S3, an improved morphological segmentation algorithm is combined with Canny edge recognition to show excellent performance, and the rotten area of the walnut kernels can be effectively detected. Whether the surfaces of the walnut kernels have defects can be quickly identified.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition, and in particular relates to a method for identifying surface defects of walnut kernels based on hyperspectral imaging technology. Background Art

[0002] Walnut kernels are rich in unsaturated fatty acids, such as linoleic acid and linolenic acid, which can reduce the intestinal absorption of cholesterol, lower cholesterol levels, prevent arteriosclerosis, are beneficial to cardiovascular health, and can also reduce insulin resistance and reduce the risk of type 2 diabetes.

[0003] Although walnuts are crushed and sorted by machines and then walnut kernels with surface defects are picked out manually, this selection method is inefficient and has high labor costs. Summary of the Invention

[0004] Based on the above background technology, it is necessary to develop a method for identifying surface defects of walnut kernels based on hyperspectral imaging technology, including the following steps:

[0005] S1: Acquire hyperspectral imaging of walnut kernels and remove interference noise;

[0006] S2: Two-dimensional correlation spectrum analysis can be used to qualitatively analyze different defect types of walnut kernels; the ability to identify rotten walnut kernels is further enhanced through the visualization analysis of heat maps;

[0007] S3: The improved morphological segmentation algorithm combined with Canny edge recognition shows superior performance and can effectively detect the rotten area of ​​walnut kernels.

[0008] Preferably, the two-dimensional correlation spectrum analysis step in step S2 is as follows:

[0009] S21. Collect spectral data that changes with external disturbance; assume that the external disturbance variable is , the spectral variables are , where the unit of the spectral variable is wavelength;

[0010] S22. Calculate the synchronization correlation strength , for a given two spectral variables and ,in and Represents different wavelengths; the calculation formula is as follows; where m is the number of values ​​of the external disturbance variable;

[0011] (2)

[0012] ; (3)

[0013] S23. Then calculate the asynchronous correlation strength ,in is a variable The standard deviation of

[0014] (4)

[0015] S24.By wavelength and As the coordinate axis, the synchronous two-dimensional correlation spectrum and the asynchronous two-dimensional correlation spectrum are drawn separately, and then fused; the upper left corner of the two-dimensional spectrum is the synchronous correlation spectrum, and the lower right corner is the asynchronous correlation spectrum.

[0016] Preferably, the morphological segmentation algorithm of step S3 includes morphological filtering and morphological gradient; erosion, dilation, opening operation and closing operation are basic operations of morphological filtering, which can be used to eliminate weak noise and reduce the influence of strong noise; morphological opening operation can eliminate dark features, and closing operation can eliminate bright features; the morphological opening operation is as follows:

[0017] (5)

[0018] Similarly, the morphological closing operation is as follows:

[0019] (6)

[0020] Where, P, B, , , , • are the input image, disk-shaped structure element, morphological dilation, erosion, opening operation and closing operation respectively; the morphological gradient operation can enhance the contrast between the defect area and the normal area, simplify the boundary of the image, and smooth the image; the morphological gradient operation formula is as follows:

[0021] (7)

[0022] Where, ,P,B, , They are gradient image, input image, disk-shaped structuring element, morphological dilation and erosion; these operations are based on the concept of set theory, and achieve image segmentation and target extraction by analyzing and processing the target shape and structure in the image.

[0023] Beneficial effects of the present invention:

[0024] 1. Through two-dimensional correlation spectrum analysis, we can qualitatively analyze walnut kernels with different defect types, and the visualization analysis of heat maps further enhances the ability to identify rotten walnut kernels.

[0025] 2. The improved morphological segmentation algorithm combined with Canny edge recognition shows superior performance and can effectively detect the rotten area of ​​walnut kernels. DETAILED DESCRIPTION

[0026] The present invention provides a method for identifying surface defects of walnut kernels based on hyperspectral imaging technology;

[0027] Walnut kernels are classified into smooth, rough, and rotten textures. This study used a shortwave infrared hyperspectral imaging system to perform hyperspectral imaging of walnut kernels in a laboratory setting. Hyperspectral images are synthesized from multiple monochromatic images. The hyperspectral imaging setup includes a light source, an imaging spectrometer, a CCD camera, a lens, a translation stage, a controller, a darkroom, and a computer. The hyperspectral imaging system is illuminated by two independent halogen lamps, each with a voltage of 12 V and a rated power of 20 W, covering a spectral range of 400–1000 nm.

[0028] Obtaining hyperspectral imaging of walnut kernels includes: before collecting data, the hyperspectral imaging system needs to be turned on and preheated for 30 minutes to avoid baseline drift affecting the collected image data, and the system imaging parameters are adjusted through the Spectral View software to ensure that the collected images are clear and undistorted. The image acquisition operation steps are as follows: (1) The sample is placed on the displacement platform at intervals. During the image acquisition process, the sample is manually positioned in the inspection room and then aligned with the camera for imaging. The displacement platform is calibrated to move at a speed of 12 cm per second. The exposure time is configured to 8 ms, and the displacement speed is synchronized with the exposure duration of the camera to ensure optimal image capture. Each spectral image is recorded as a three-dimensional data array, represented as (x, y, z), including two spatial dimensions (x, y) and one spectral dimension (z). Here, (x) and (y) correspond to the spatial dimensions, while (z) represents the spectral dimension, indicating the number of spectral bands. (2) After previewing the acquisition in the Spectral View software and confirming that it is correct, click the Capture option to complete the image acquisition. The image includes 176 spectral bands and a spatial dimension of 960*1065. (3) The acquired image data is saved to the computer. This data includes both impact information and spectral information. Because the CCD camera in the imaging spectrometer has dark current when operating, the image acquisition in the low-intensity band has a large noise, which affects the image clarity.

[0029] To remove noise caused by equipment and environmental interference, we performed a black-white correction on the raw hyperspectral images. This correction requires a black image with a reflectance close to 0% (obtained by covering the lens with a black lens cap) and a white image with a reflectance close to 100% (obtained using a uniform white plate made of Teflon). Image correction is performed using the white and black references using the following equation, where R is the corrected image.

[0030] (1)

[0031] Where: R is the corrected image; R orignal is the original hyperspectral image; R dark is the black plate correction image; R white This is a white plate calibration image.

[0032] Two-dimensional correlation spectroscopy is renowned for its exceptionally high spectral resolution. It effectively distinguishes and resolves overlapping peaks that are difficult to discern in conventional one-dimensional spectra. It can highlight the characteristic signals of specific functional groups or molecular structures while suppressing or reducing interference from other non-target signals. It can also effectively extract and resolve weak signals that are difficult to observe in complex systems. In this way, two-dimensional correlation spectroscopy significantly improves the resolution and accuracy of spectral analysis, providing a powerful tool for qualitative analysis of walnut kernels with various defects.

[0033] Synchronous two-dimensional correlation spectroscopy is constructed based on the correlation of the intensity changes of two spectral signals at the same time. A strong correlation peak appears in synchronous correlation spectroscopy, indicating a coordinated change relationship between the two signals; if the change trends are opposite, a negative correlation peak appears. Asynchronous two-dimensional correlation spectroscopy, on the other hand, focuses on analyzing the correlation of the intensity changes of two spectral signals at different times. The correlation peak indicates a temporal sequence and different rates of change between the two signals.

[0034] The following are the steps to generate synchronous-asynchronous two-dimensional correlation spectrum:

[0035] 1. Collect spectral data that changes with external disturbances (such as time, temperature, concentration, etc.). Assume that the external disturbance variable is , the spectral variables are , where the unit of the spectral variable is wavelength. For example, it can be a series of near-infrared spectral data that changes over time, with the wavelength range from a certain starting value to a certain ending value.

[0036] 2. Calculate the synchronization correlation strength , for a given two spectral variables and ,in and Represents different wavelengths. The calculation formula is as follows; where m is the number of values ​​of the external disturbance variable.

[0037] (2)

[0038] (3)

[0039] 3. Secondly, calculate the asynchronous correlation strength ,in is a variable The standard deviation of .

[0040] (4)

[0041] 4. By wavelength and As the coordinate axis, the synchronous two-dimensional correlation spectrum and the asynchronous two-dimensional correlation spectrum are plotted separately and then fused. The upper left corner of the two-dimensional spectrum is the synchronous correlation spectrum, and the lower right corner is the asynchronous correlation spectrum.

[0042] A heat map is a density function-based visualization method used to present the density and distribution of points on a map. The principle is to first collect data, then divide the map space, aggregate the data points into corresponding areas and count them, and then use a color mapping table to map different data ranges into different colors (high-value areas are warm tones, low-value areas are cold tones) based on the statistical results. Finally, the data distribution and density are visualized.

[0043] Heatmaps have advantages in detecting and analyzing surface decay in walnut kernels. They can quickly identify abnormal areas, reduce decay risks, and improve economic benefits. However, they also have limitations. They primarily reflect the surface temperature of the kernel, making them less effective in detecting internal decay and susceptible to interference from external factors. Therefore, combining heatmaps with hyperspectral principal component analysis can enhance feature visualization and make the results more intuitive.

[0044] The watershed algorithm is a typical region-based segmentation algorithm. Its basic principle is to compare the grayscale image to the topological landform in geography, in which the function in the image is The value represents the altitude. Each local minimum and the area affected by it are called a catchment basin, and the boundaries of these catchment basins constitute the watershed. The core goal of the segmentation algorithm based on this concept is to find the watershed. The concept of the watershed and its formation process can be explained by simulating the immersion process. Imagine drilling a hole at the location of the minimum value in each area, and then letting water flow out of these holes at a uniform rising rate. The water starts from a low altitude and gradually rises until it submerges the entire terrain. When water from different catchment basins is about to converge, a dam is built at the confluence of the two catchment basins. This dam represents the formation of a watershed.

[0045] The watershed algorithm can accurately detect edges and segment multiple objects while preserving detailed information. However, it is sensitive to noise, affected by uneven grayscale, has high computational complexity, and is prone to over-segmentation. In practical applications, it requires improvement and optimization based on image characteristics and requirements.

[0046] The Otsu method, proposed by Nobuyuki Otsu in 1979, is an image segmentation algorithm based on the principle of maximum inter-class variance. It automatically binarizes an image by traversing all possible thresholds and selecting the one that maximizes the inter-class variance between foreground and background. This algorithm is computationally simple and fast, suitable for most scenarios requiring a global threshold. However, it is sensitive to image noise and is significantly affected by factors such as illumination variations. It performs poorly for images with small grayscale differences between the target and background.

[0047] The Canny edge detection algorithm was proposed by John F. Canny in 1986. Its method is to first perform Gaussian filtering on the input image to remove noise, use first-order partial derivative finite difference to calculate the gradient amplitude and direction, suppress the non-maximum value of the gradient amplitude image along the gradient direction to refine the edge (retain the pixel with the local maximum gradient), and use high and low thresholds to detect the image after non-maximum suppression (pixels above the high threshold are strong edge points, pixels below the low threshold are non-edge points, and pixels between the two are determined to be edge points based on their connection with strong edge points).

[0048] The Canny edge detection algorithm offers numerous advantages for walnut kernel surface detection. It boasts high accuracy, accurately detecting edges and capturing subtle changes and contour features even in complex backgrounds and lighting conditions. It also exhibits strong noise immunity, using Gaussian filtering to remove noise and improve detection accuracy. It is also highly adaptable, suitable for walnut kernels of various shapes and adaptable to varying lighting and background conditions. The Morphological Segmentation (MSM) algorithm is an image processing method based on mathematical morphology, primarily used for image segmentation and object extraction. The fundamental idea of ​​mathematical morphology is to use structural elements of a given form to measure and extract corresponding shapes within an image, thereby enabling image analysis and recognition.

[0049] Morphological processing includes morphological filtering and morphological gradient. Erosion, dilation, opening, and closing are the basic operations of morphological filtering, which can be used to eliminate weak noise and reduce the impact of strong noise. Morphological opening can eliminate dark features, while closing can eliminate bright features. The morphological opening operation is shown below:

[0050] (5)

[0051] Similarly, the morphological closing operation is as follows:

[0052] (6)

[0053] Where, P, B, , , , • are the input image, disk-shaped structuring element, morphological dilation, erosion, opening, and closing operations, respectively. The morphological gradient operation can enhance the contrast between defective and normal areas, simplify image boundaries, and smooth the image. The morphological gradient operation formula is as follows:

[0054] (7)

[0055] Where, ,P,B, , They are gradient image, input image, disk-shaped structure element, morphological dilation and erosion. These operations are based on the concepts of set theory and can achieve image segmentation and target extraction by analyzing and processing the target shape and structure in the image.

[0056] Compared to the Otsu and watershed segmentation algorithms, the morphological segmentation algorithm improved by the Canny algorithm offers several advantages. First, the Canny algorithm accurately detects decay edges, providing a precise foundation for subsequent morphological segmentation and better adapting to the shape and size of walnut kernel decay regions. Compared to the Otsu threshold segmentation and watershed segmentation algorithms, it is more sensitive to edge information. By utilizing methods such as Gaussian filtering and the morphological operations themselves in detecting decayed areas, this algorithm exhibits excellent noise immunity, whereas the other two algorithms are sensitive to noise. Furthermore, the parameters of the Canny algorithm and morphological operations can be flexibly adjusted, allowing it to adapt to varying degrees of decay in walnut kernels, improving its versatility and adaptability.

[0057] PLS and LS-SVM sugar content prediction models were established for normal walnut kernel samples. The sugar content can indirectly reflect the maturity of the walnut kernel: immature walnut kernels may contain more starch (a carbohydrate), have a rather astringent taste, and have incomplete sugar conversion; mature walnut kernels have lower starch content and are richer in oil and protein.

[0058] Partial least squares (PLS) constructs a mathematical model based on the characteristics of principal component analysis (PCA) and multiple regression. It is applied to multivariate calibration to perform regression analysis on the relationship between chemical composition and spectra. This method is primarily used to address regression scenarios involving multicollinearity among independent variables and small sample sizes. Even with small sample sizes, it can produce relatively reasonable and effective model results and is applicable to high-dimensional data. It can effectively handle situations with severe multiple correlations between independent variables and enables stable regression modeling in such complex data relationships. Compared to traditional multiple linear regression methods, PLS has advantages in handling complex data structures and multicollinearity, and possesses greater predictive power and robustness.

[0059] The least squares support vector machine (LS-SVM) is an improved support vector machine based on statistical theory. Using a least squares linear system as the loss function, it transforms the solution of a quadratic optimization problem into a system of linear equations, simplifying the solution process. LS-SVM replaces the non-equality constraints of the SVM optimization problem with equality constraints, resulting in faster training than support vector machines. Overall, the linear decision function constructed by LS-SVM offers excellent fitting performance and strong model generalization. This simplifies the model training process to a certain extent, making it easier to implement and apply. Compared to traditional SVMs, it offers faster solution speeds, particularly when processing large amounts of data. It can achieve good prediction results with limited samples and is easy to understand and implement.

[0060] This study successfully demonstrated the effectiveness of combining hyperspectral imaging technology with two-dimensional correlation spectroscopy analysis and image segmentation algorithms for surface defect detection and internal quality assessment of walnut kernels. Two-dimensional correlation spectroscopy analysis enabled qualitative analysis of walnut kernels with different defect types, while heat map visualization further enhanced the ability to identify rotten kernels. Regarding image segmentation, an improved morphological segmentation algorithm combined with Canny edge detection demonstrated superior performance, effectively detecting rotten areas within walnut kernels. By analyzing the average spectra of regions of interest in hyperspectral images of normal, defect-free walnut kernel samples, we established a quantitative prediction model for soluble solids based on a least squares support vector machine (LS-SVM) and partial least squares (PLS). The average spectrum optimized using the SG+StandardScaler preprocessing method performed best in the LS-SVM model.

[0061] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for identifying surface defects of walnut kernels based on hyperspectral imaging technology, characterized in that: The following steps are involved: S1: Acquire hyperspectral imaging of walnut kernels and remove interference noise; S2: Two-dimensional correlation spectrum analysis can be used to qualitatively analyze different defect types of walnut kernels; the ability to identify rotten walnut kernels is further enhanced through the visualization analysis of heat maps; S3: The improved morphological segmentation algorithm combined with Canny edge recognition shows superior performance and can effectively detect the rotten area of ​​walnut kernels.

2. The method according to claim 1, characterized in that The steps of the two-dimensional correlation spectrum analysis in step S2 are as follows: S21. Collect spectral data that changes with external disturbance; the external disturbance variable is , the spectral variables are , the unit of spectral variables is wavelength; S22. Calculate the synchronization correlation strength , for a given two spectral variables and ,in and Represents different wavelengths; the calculation formula is as follows; in m is the number of values ​​of the external disturbance variable; (2) ； (3) S23. Then calculate the asynchronous correlation strength ,in is a variable The standard deviation of (4) S24.By wavelength and As the coordinate axis, the synchronous two-dimensional correlation spectrum and the asynchronous two-dimensional correlation spectrum are drawn separately, and then fused; the upper left corner of the two-dimensional spectrum is the synchronous correlation spectrum, and the lower right corner is the asynchronous correlation spectrum.

3. The method according to claim 1, characterized in that The morphological segmentation algorithm in step S3 includes morphological filtering and morphological gradient. Erosion, dilation, opening and closing operations are the basic operations of morphological filtering, which can be used to eliminate weak noise and reduce the influence of strong noise. The morphological opening operation can eliminate dark features, and the closing operation can eliminate bright features. The morphological opening operation is as follows: (5) Similarly, the morphological closing operation is as follows: (6) Where, P, B, , , , • are the input image, disk-shaped structure element, morphological dilation, erosion, opening operation and closing operation respectively; the morphological gradient operation can enhance the contrast between the defect area and the normal area, simplify the boundary of the image, and smooth the image; the morphological gradient operation formula is as follows: (7) Where, ,P,B, , They are gradient image, input image, disk-shaped structuring element, morphological dilation and erosion; these operations are based on the concept of set theory, and achieve image segmentation and target extraction by analyzing and processing the target shape and structure in the image.

Images