A Non-destructive Testing Method for Early Damage in Apples Based on Time-Varying Reflectivity Characteristics

Abstract

This invention discloses a non-destructive testing method for early damage in apples based on time-varying reflectance characteristics. The method utilizes a hyperspectral imaging system to continuously acquire hyperspectral images of sample apples. The average spectral reflectance of each pixel in the characteristic band is extracted as a feature value based on the hyperspectral image. The relationship between the feature value and the sampling time is fitted to obtain a fitting curve. Then, the slope and range K of the curve are obtained by differentiating the fitting curve. R , range K R After the distributed data is normalized and converted into a grayscale image, the grayscale image is segmented, and finally, the damage is accurately segmented and located using the minimum bounding rectangle. This invention utilizes hyperspectral image processing technology to achieve non-destructive detection of early-stage damage in apples, exhibiting good accuracy and practicality.

Description

Technical Field

[0001] This invention relates to the field of non-destructive testing technology for fruit quality, specifically to a non-destructive testing method for early damage in apples based on time-varying reflectance characteristics. Background Technology

[0002] Mechanical damage caused by falls, collisions, squeezing, vibration, and other factors is the main form of fruit damage. Especially in the early stages of damage, because most damaged areas are difficult to detect, the damaged parts further rot and deteriorate during subsequent storage and transportation, forming rotten fruit and spreading to surrounding fruits, causing serious losses to producers and consumers. If damage can be detected in its early stages and rotten fruit can be removed promptly, such losses can be effectively reduced and the quality of marketable fruit improved.

[0003] In the early stages of fruit injury, the damaged area resembles the surrounding healthy tissue, with indistinct surface features, limiting the effectiveness of visible light-based detection methods. Existing visible light imaging methods cannot be used for early fruit injury detection. Recent studies have shown that spectroscopic techniques have the potential to detect early damage. Fan et al. used a pendulum device to create blueberry internal bruise samples and achieved a detection accuracy of 77.5% for early (30 min after injury) damage using a least squares support vector machine (LS-SVM) model (Fan, S., Li, C., Huang, W., & Chen, L. (2017). Detection of blueberry internal bruising over time using NIR hyperspectral reflectance imaging with optimal wavelengths. Postharvest Biology and Technology, 134, 55-66.). Pan et al. also used a pendulum device to cause impact damage to apples and established a classification model of internal bruising in apples based on visible light and near-infrared (VNIR) hyperspectral imaging at seven early injury time periods (0, 12, 24, 36, 48, 60 and 72 h after injury). The results showed that the gradient boosting decision tree (GBDT) model had a classification accuracy of only 70.59% for different types of bruising in apples (Pan, X., Sun, L., Li, Y., Che, W., Ji, Y., Li, J., Li, J., Xie, X., & Xu, Y. (2019). Non-destructive classification of apple bruising time based on visible and near-infrared hyperspectral imaging. J Sci Food Agric, 99(4), 1709-1718.).

[0004] Since the damage characteristics of early-stage fruit damage are not obvious, the accuracy of damage detection models is not high. Therefore, it is necessary to develop a fruit early-stage damage detection method with good recognition accuracy. Summary of the Invention

[0005] To overcome the problems in the prior art, this invention proposes a non-destructive detection method for early damage to apples based on the evolutionary differences in spectral feature parameters between damaged and undamaged areas. This invention utilizes the spectral evolution characteristics of fruit damage sites in the early stages of mechanical damage to overcome the problem that obvious features cannot be obtained by direct observation of damage sites in the early stages of damage.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] The specific steps of the non-destructive testing method for initial damage to apples include:

[0008] S1) Acquisition of hyperspectral images: Multiple hyperspectral images of sample apples are continuously acquired at preset fixed time intervals using a hyperspectral imaging system.

[0009] S2) Preprocessing of hyperspectral images;

[0010] In step S2), the preprocessing includes black and white correction, image smoothing, and DN value normalization.

[0011] Specifically, in this embodiment of the invention, a hyperspectral image background is selected by dividing the 917-1717nm band using a wavelength of 1076nm.

[0012] Specifically, the black-and-white correction described in this embodiment of the invention uses SRAnal710TM.

[0013] Specifically, in this embodiment of the invention, the image smoothing employs the Savitzky-Golay smoothing method to eliminate high-frequency random noise.

[0014] S3) After selecting the region of interest (ROI) from multiple preprocessed hyperspectral images, the feature values ​​of each pixel in the ROI at all sampling times are input into the damage detection model for processing and the segmentation threshold of the damage detection model is determined.

[0015] In step S3), the process of selecting the region of interest is specifically as follows:

[0016] When the preprocessed hyperspectral image is the preprocessed hyperspectral image of the sample apple, the damaged area and the undamaged area are selected as regions of interest from the preprocessed hyperspectral image respectively.

[0017] When the preprocessed hyperspectral image is the preprocessed hyperspectral image of the apple to be tested, the apple detection area, i.e. the area where the apple image is located, is selected from the preprocessed hyperspectral image as the region of interest.

[0018] Specifically, the damaged area and the undamaged area have the same shape and area.

[0019] Specifically, the damaged area and the undamaged area are images of the damaged area and the undamaged area of ​​the sample apple, respectively. The apple detection area includes the entire apple to be tested, meaning that the area of ​​the damaged area and the area of ​​the undamaged area are both smaller than the area of ​​the apple detection area.

[0020] The specific steps of the damage detection model are as follows:

[0021] S3.1) Based on the feature values ​​of each pixel in the region of interest at each sampling time, obtain the temporal fitting curve between the feature values ​​of each pixel and the sampling time. The temporal fitting curve between the feature values ​​of any pixel and the sampling time is... for:

[0022]

[0023] In the formula, t is the sampling time, a is the peak value of the feature value-sampling time curve of any pixel, b is the sampling time corresponding to the peak value a of the feature value-sampling time curve of any pixel, c is the standard deviation between the feature value of any pixel and the mean of the feature values ​​of all pixels, and e is the base of the natural logarithm.

[0024] S3.2) Obtain the slope range of each time series fitting curve based on its slope, and normalize the slope range of all time series fitting curves. Then, convert the normalized result into a range grayscale image; whereby the slope range K of any time series fitting curve is... R for:

[0025]

[0026] In the formula, K max K represents the maximum slope of any time-series fitted curve. min This represents the minimum slope of any time-series fitted curve;

[0027] S3.3) The range grayscale image obtained in step S3.2) is segmented using a segmentation threshold.

[0028] In step S3), the average spectral reflectance within the characteristic band is used as the feature value; the average spectral reflectance within the characteristic band of any pixel Specifically:

[0029]

[0030] In the formula, λ1 is the starting wavelength of the characteristic band, λ2 is the ending wavelength of the characteristic band, and R λ λ is the spectral reflectance of any pixel at wavelength λ (wavelength λ is within the characteristic band).

[0031] Specifically, the characteristic band is the band sensitive to mechanical damage in apples.

[0032] Specifically, in the embodiments of the present invention, the selected characteristic band is 1255-1314nm.

[0033] In step S3), the process of determining the segmentation threshold of the damage detection model is as follows:

[0034] First, the feature values ​​of each pixel in the region of interest at all sampling times are input into the damage detection model for processing to obtain the range grayscale image described in step S3.2).

[0035] Finally, the Otsu threshold segmentation method is used to process the range grayscale image obtained in step S3.2) to determine the segmentation threshold of the damage detection model.

[0036] S4) Damage detection: According to steps S1) to S2), multiple preprocessed hyperspectral images of the apple to be tested are obtained. After selecting the region of interest from the multiple preprocessed hyperspectral images of the apple to be tested, the feature values ​​of each pixel in the region of interest at all sampling times are input into the damage detection model with the segmentation threshold confirmed in step S3). After post-processing, the damage detection and localization results are obtained.

[0037] The post-processing process in step S4) is specifically as follows:

[0038] After performing dilation and erosion operations on the image processed by the damage detection model, the maximum connected component extraction and hole filling operations are performed in sequence to achieve accurate segmentation. Finally, the damage of the apple under test is located using the minimum bounding rectangle to obtain the non-destructive testing results.

[0039] The beneficial effects of this invention are as follows:

[0040] First, this method utilizes the characteristic value (average spectral reflectance) extracted from apples in the mechanical damage-sensitive band. The changes that occur over time provide a theoretical basis for the non-destructive testing of early mechanical damage in apples. Secondly, this method affects the average spectral reflectance. The method fits the relationship between the image sampling time t and the curve, obtaining the maximum and minimum slope values ​​and calculating the slope range. This enhances the damage characteristics while simultaneously reducing data dimensionality. Finally, the method utilizes hyperspectral imaging technology and simple image processing techniques to achieve non-destructive detection of early-stage damage in apples. This invention significantly improves the accuracy of early-stage apple detection, and the constructed non-destructive detection model boasts high precision. It enables early damage detection and timely removal of damaged fruit, improving the quality of marketable fruit. Furthermore, it reduces the risk of further decay and spoilage of damaged parts during storage and transportation, thus mitigating the serious losses suffered by producers and consumers. Attached Figure Description

[0041] Figure 1 Flowchart of a non-destructive testing method for early damage to apples based on time-varying reflectivity characteristics;

[0042] Figure 2High-spectral image (850nm) of early damage in apples and ROI extraction;

[0043] Figure 3 : Fitted curve of the relationship between the average spectral reflectance of the damaged area and time;

[0044] Figure 4 : Slope distribution of the fitted curve in the damaged area;

[0045] Figure 5 : Fitted curve of the relationship between the average spectral reflectance of the undamaged area and time;

[0046] Figure 6 : Slope distribution of the fitted curve in the non-damaged area;

[0047] Figure 7 : Grayscale image of the initial damage to an apple;

[0048] Figure 8 Apple's initial damage segmentation diagram. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] The specific steps of this method include:

[0051] S1) Acquisition of hyperspectral images: Multiple hyperspectral images of sample apples are continuously acquired at preset fixed time intervals using a hyperspectral imaging system.

[0052] Specifically, the acquisition duration is preferably 0–90 min, the acquisition interval is preferably 0–30 min, and one hyperspectral image is acquired at each acquisition time.

[0053] S2) Preprocessing of hyperspectral images: Preprocessing includes black and white correction, image smoothing and DN value normalization.

[0054] Preferably, a hyperspectral image background is selected with a wavelength of 1076 nm to divide the 917–1717 nm band.

[0055] Preferably, black and white correction uses SRAnal710TM.

[0056] Preferably, the image smoothing employs the Savitzky-Golay smoothing method to eliminate high-frequency random noise.

[0057] S3) After selecting the region of interest (ROI) from multiple preprocessed hyperspectral images, the feature values ​​of each pixel in the ROI at all sampling times are input into the damage detection model for processing, and the segmentation threshold of the damage detection model is determined.

[0058] Specifically, in step S3), the process of selecting the region of interest is as follows:

[0059] When the preprocessed hyperspectral image is the preprocessed hyperspectral image of the sample apple, damaged areas and undamaged areas with the same shape and area are selected as regions of interest from the preprocessed hyperspectral image.

[0060] When the preprocessed hyperspectral image is the preprocessed hyperspectral image of the apple to be tested, the apple detection area, i.e. the area where the apple image is located, is selected from the preprocessed hyperspectral image as the region of interest.

[0061] Specifically, in step S3), the average spectral reflectance within the characteristic band is used as the feature value; the average spectral reflectance within the characteristic band of any pixel Specifically:

[0062]

[0063] In the formula, λ1 is the starting wavelength of the characteristic band, λ2 is the ending wavelength of the characteristic band, and R λ λ is the spectral reflectance of any pixel at wavelength λ (wavelength λ is within the characteristic band).

[0064] Specifically, the characteristic band is the mechanical damage sensitive band of apples, which is 1255-1314nm.

[0065] Specifically, the damage detection model follows these steps:

[0066] S3.1) Based on the feature values ​​of each pixel in the region of interest at each sampling time, obtain the temporal fitting curve between the feature values ​​of each pixel and the sampling time; wherein, the temporal fitting curve between the feature values ​​of any pixel and the sampling time... for:

[0067]

[0068] In the formula, t is the sampling time, a is the peak value of the feature value-sampling time curve of any pixel, b is the sampling time corresponding to the peak value a of the feature value-sampling time curve of any pixel, c is the standard deviation between the feature value of any pixel and the mean of the feature values ​​of all pixels, and e is the base of the natural logarithm.

[0069] S3.2) Obtain the slope range of each time series fitting curve based on its slope, and normalize the slope ranges of all time series fitting curves to convert them into a range grayscale image; whereby the slope range K of any time series fitting curve is... R for:

[0070]

[0071] In the formula, K max K represents the maximum slope of any time-series fitted curve. min This represents the minimum slope of any time-series fitted curve;

[0072] S3.3) The range grayscale image obtained in step S3.2) is segmented using a segmentation threshold.

[0073] Specifically, in step S3), the process of determining the segmentation threshold of the damage detection model is as follows: the range grayscale image obtained in step S3.2) is processed using the Otsu threshold segmentation method to obtain the segmentation threshold of the damage detection model.

[0074] S4) Damage detection: According to steps S1) to S2), multiple preprocessed hyperspectral images of the apple to be tested are obtained. After selecting the region of interest from the multiple preprocessed hyperspectral images of the apple to be tested, the feature values ​​of each pixel in the region of interest at all sampling times are input into the damage detection model with the segmentation threshold confirmed in step S3). After post-processing, the damage detection and localization results are obtained.

[0075] Specifically, the post-processing process is as follows: after the image processed by the damage detection model is subjected to dilation and erosion operations in sequence, the maximum connected component extraction and hole filling operations are performed in sequence to achieve accurate segmentation. Finally, the damage of the apple under test is located using the minimum bounding rectangle to obtain the non-destructive testing result.

[0076] Specific embodiments of the present invention are as follows:

[0077] S1) Hyperspectral image acquisition. Multiple hyperspectral images M of apples are continuously acquired at a certain frequency using a hyperspectral imaging system;

[0078] The experiment used the same batch of Red Delicious apples purchased from the local fruit wholesale market as the experimental material. Before the experiment, the apples were inspected, and cracked, rotten, and misshapen fruits were removed to ensure they were undamaged. The apples were then placed at room temperature for 24 hours before the experiment to ensure their temperature matched the room temperature.

[0079] Apple damage simulation was conducted using a MY-A721 impact testing bench (Dongguan Mingyu Intelligent Technology Co., Ltd.). The impact hammer had a diameter of 80mm and a weight of 500g. The impact heights were 100mm, 150mm, and 200mm. Based on the impact height, the damaged samples were divided into three groups: H1, H2, and H3, with 20 damaged apple samples prepared at each height. Simultaneously, 20 undamaged samples were designated as group H0.

[0080] The hyperspectral imaging system is model SOC710 SWIR, with a spectral range of 917–1717 nm, a spectral resolution of 2.75 nm, a lens focal length of 35 mm, and a pixel resolution of 640 × 568 pixels. The SOC710 SWIR hyperspectral imaging system uses the accompanying HyperScanner software for image acquisition.

[0081] The characteristic value (average spectral reflectance) of "Red Delicious" fruit extracted from the mechanically damaged sensitive wavelength range of 1255-1314nm The frequency and number of hyperspectral images of apples can vary from 0 to 1.5 hours after damage. Therefore, the frequency and number of hyperspectral image acquisitions for apples need to be evaluated based on the apple harvest time or the expected damage time. In Example 1, hyperspectral image acquisition was performed immediately after damage to "Red Delicious" apples, with a time interval of 6 minutes, for a total of 16 acquisitions over 90 minutes.

[0082] S2) Hyperspectral image preprocessing. The hyperspectral image M is preprocessed, including black-and-white correction, image smoothing, and DN value normalization.

[0083] Images of the whiteboard were acquired using a hyperspectral imaging system. white (λ) and blackboard image R dark (λ), perform black and white correction using the accompanying SRAnal710TM software. The black and white correction formula is:

[0084]

[0085] In the formula: R xy (λ) — Raw image data; R white (λ) — Blackboard image data; R dark (λ) — Whiteboard image data; R(λ) — Corrected image data.

[0086] Select a wavelength of 1076nm to segment the background of the hyperspectral image in the 917–1717nm band.

[0087] The Savitzky-Golay smoothing method was used to smooth the spectral data to eliminate high-frequency random noise.

[0088] Normalization is performed to normalize the values ​​of each band of the spectral data to a range between 0 and 1, so that they can be compared and analyzed.

[0089] S3) Select apple damaged area ROI1 and undamaged area ROI2 of the same area from multiple preprocessed hyperspectral images, obtain the spectral reflectance of each pixel in the characteristic band of apple damaged area ROI1 and undamaged area ROI2 respectively, and input them into the damage detection model for processing to obtain the segmentation threshold.

[0090] 3.1) Statistical parameter processing: According to the above classification, the average spectral reflectance of each pixel in the damaged area ROI1 and the undamaged area ROI2 of the apple was obtained in the 1255-1314nm band. Average spectral reflectance Obtained using the following formula:

[0091]

[0092] In the formula: —The average spectral reflectance in the 1255~1314nm band; R(λ)—the spectral reflectance at wavelength λ; λ—wavelength.

[0093] Average spectral reflectance of each pixel The relationship with the image sampling time t is fitted to obtain a time-series fitting curve. The time-series fitting curve is obtained according to the following formula:

[0094]

[0095] In the formula: t—image sampling time; a—peak value of the curve; b—the corresponding abscissa of the peak value of the curve; c—standard deviation.

[0096] Differentiate the fitted curve to obtain the maximum slope K of the fitted curve for each pixel. max and minimum value K min And calculate the range K of the two according to the following formula. R :

[0097]

[0098] Where: K R —Slope range; K max —Maximum slope; K min —Minimum slope.

[0099] Finally, the range K R The distributed data is normalized and converted into a range grayscale image;

[0100] 3.3) Segmentation threshold acquisition: The Otsu threshold segmentation method is used to process the range grayscale image to select the optimal segmentation threshold T, and the optimal segmentation threshold T is used as the segmentation threshold of the damage detection model.

[0101] S4) Use the established damage detection model with defined segmentation thresholds to perform damage detection on the apple under test:

[0102] For the apple to be tested, hyperspectral data acquisition and preprocessing are performed according to steps S1) and S2); the average spectral reflectance of each pixel in the 1255-1314nm band within the apple detection area ROI3 is extracted. Average spectral reflectance of each pixel The relationship with the image sampling time t is fitted to obtain a fitted curve; the derivative of the fitted curve is calculated to obtain the maximum slope K of the fitted curve for each pixel. max and minimum value K min And calculate the range K of the two. R ; range K R The distributed data is normalized and converted into a range grayscale image; the range grayscale image is segmented using a segmentation threshold T; the damaged area is accurately segmented by using image dilation, erosion, maximum connected component extraction, and hole filling on the segmented image; and the damage is located using the minimum bounding rectangle.

[0103] The table below shows the results of non-destructive testing of initial damage to apples from impacts at different heights (100mm, 150mm, 200mm).

[0104]

[0105] The results show that the detection method in this embodiment has a 100% accuracy rate in detecting early-stage, undamaged apples, demonstrating good precision.

Claims

1. A non-destructive testing method for early damage in apples based on time-varying reflectance characteristics, characterized in that: The specific steps of the method include: S1) Acquisition of hyperspectral images: Multiple hyperspectral images of sample apples are continuously acquired at preset fixed time intervals using a hyperspectral imaging system. S2) Preprocessing of hyperspectral images; S3) After selecting the region of interest from multiple preprocessed hyperspectral images, the feature values ​​of each pixel in the region of interest at all sampling times are input into the damage detection model for processing, and the segmentation threshold of the damage detection model is determined. In step S3), the specific steps of the damage detection model are as follows: S3.1) Based on the feature values ​​of each pixel in the region of interest at each sampling time, obtain the time-series fitting curve between the feature values ​​of each pixel and the sampling time; Among them, the temporal fitting curve between the feature value of any pixel and the sampling time for: ； In the formula, t Sampling time, a The peak value of the feature value-sampling time curve for any pixel. b The peak value of the feature value-sampling time curve for any pixel a The corresponding sampling time, c Let be the standard deviation between the feature value of any pixel and the mean of the feature values ​​of all pixels. e is the base of the natural logarithm; S3.2) Obtain the slope range of each time series fitting curve, and after normalizing the slope range of all time series fitting curves, convert it into a range grayscale image. Among them, the slope range of any time-series fitted curve K R for: ； In the formula, K max K represents the maximum slope of any time-series fitted curve. min This represents the minimum slope of any time-series fitted curve; S3.3) The range grayscale image obtained in step S3.2) is segmented using a segmentation threshold; S4) Damage detection: According to steps S1)~S2), multiple preprocessed hyperspectral images of the apple to be tested are obtained. After selecting the region of interest from the multiple preprocessed hyperspectral images of the apple to be tested, the feature values ​​of each pixel in the region of interest at all sampling times are input into the damage detection model with the segmentation threshold confirmed in step S3). After post-processing, the damage detection and localization results are obtained.

2. The non-destructive testing method for early damage to apples based on time-varying reflectance characteristics according to claim 1, characterized in that: In step S3), the average spectral reflectance within the characteristic band is used as the feature value; the average spectral reflectance within the characteristic band of any pixel Specifically: ； In the formula, λ 1 The starting wavelength of the characteristic band. λ 2 The terminator wavelength of the characteristic band. For any pixel at wavelength λ Spectral reflectance at that location.

3. The non-destructive testing method for early damage to apples based on time-varying reflectance characteristics according to claim 2, characterized in that: The characteristic band is the mechanical damage sensitive band of apples, which is 1255-1314nm.

4. The non-destructive testing method for early damage to apples based on time-varying reflectance characteristics according to claim 1, characterized in that: In step S3), the process of determining the segmentation threshold of the damage detection model is as follows: the range grayscale image obtained in step S3.2) is processed using the Otsu threshold segmentation method to obtain the segmentation threshold of the damage detection model.

5. The non-destructive testing method for early damage to apples based on time-varying reflectance characteristics according to claim 1, characterized in that: In step S3), the process of selecting the region of interest is specifically as follows: When the preprocessed hyperspectral image is a preprocessed hyperspectral image of a sample apple, the damaged area and the undamaged area are selected as regions of interest from the preprocessed hyperspectral image, respectively. When the preprocessed hyperspectral image is the preprocessed hyperspectral image of the apple to be tested, the apple detection area is selected as the region of interest from the preprocessed hyperspectral image.

6. The non-destructive testing method for early damage to apples based on time-varying reflectance characteristics according to claim 1, characterized in that: In step S2), the preprocessing includes black and white correction, image smoothing, and DN value normalization.

7. The non-destructive testing method for early damage to apples based on time-varying reflectance characteristics according to claim 1, characterized in that: In step S4), the post-processing process specifically includes: After performing dilation and erosion operations on the image processed by the damage detection model, the maximum connected component extraction and hole filling operations are performed sequentially. Finally, the damage of the apple under test is located using the minimum bounding rectangle to obtain the non-destructive testing results.

Images