A free-fall multi-source information fusion method for nondestructive testing of internal quality of three plums
By using a free-fall multi-source information fusion method, combined with data collected by a spectrometer and camera, the problems of low efficiency and unstable accuracy in the internal quality detection of Sanhua plum were solved, and low-cost, high-precision automated grading was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- VEGETABLE RES INST GUANGDONG ACAD OF AGRI SERVICES
- Filing Date
- 2026-01-13
- Publication Date
- 2026-06-09
AI Technical Summary
Existing methods for internal quality testing of Sanhua plums are inefficient and costly. Traditional assembly line equipment is large in size and has high maintenance costs. Portable devices are inefficient and have unstable accuracy. Machine vision technology is not responsive enough to the chemical components of the fruit pulp. Single data source models are not robust enough in complex environments.
A free-fall multi-source information fusion method is adopted, which utilizes gravity to drive the fruit to fall naturally. Combined with photoelectric sensor to trigger the spectrometer and industrial camera to collect data synchronously, the method complements the internal chemical information of the spectrum with the external physical features of the image, and uses Savitzky-Golay smoothing, CARS algorithm and polynomial feature enhancement to construct a multi-source information prediction model.
It enables efficient and batch non-destructive testing, reduces equipment costs and maintenance requirements, improves testing accuracy, and provides a low-cost, high-precision automated solution for quality grading of spherical fruits such as Sanhua plum.
Smart Images
Figure CN122171464A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of non-destructive testing technology for agricultural products, and in particular to a free-fall multi-source information fusion method for non-destructive testing of the internal quality of Sanhua plums. Background Technology
[0002] Sanhua plum is a specialty fruit of southern my country. Its internal quality (such as sugar content, acidity, ripeness, and texture uniformity) directly affects the taste and commercial value of the fruit. Traditional testing methods, such as destructive physicochemical analysis and manual sensory evaluation, suffer from drawbacks such as low efficiency, high cost, and inability to perform batch testing. Non-destructive testing technology, due to its advantages of not damaging the integrity of the fruit, speed and efficiency, and the ability to perform batch testing, has become an important development direction for the quality testing of Sanhua plum.
[0003] Current methods for internal quality testing of Sanhua plums primarily rely on fruit dissection and sampling, which is inefficient, wasteful, and cannot meet the needs of industrial-scale grading. Traditional assembly line-style testing devices require long-distance conveyor belts, high-power motors, and complex mechanical structures, making them expensive, space-consuming, and requiring high ground flatness and three-phase power supply, making them difficult to deploy in hilly orchards or small processing plants. Portable devices, while highly flexible, rely on manual fruit-by-fruit handling, resulting in low testing efficiency and inaccuracy drift due to variations in contact force. While machine vision technology can quickly complete appearance grading, it can only capture surface RGB / grayscale information, lacking direct response to the chemical composition of the pulp, and has a high misjudgment rate when used alone. Near-infrared spectroscopy can reflect internal chemical information, but the single data source model lacks robustness in complex environments. Therefore, this invention proposes a free-fall, multi-source information fusion-based non-destructive testing method for the internal quality of Sanhua plums to address the problems existing in the prior art. Summary of the Invention
[0004] To address the aforementioned problems, the present invention aims to propose a free-fall, multi-source information fusion-based non-destructive testing method for the internal quality of Sanhua plums. This method utilizes gravity to drive the fruit to fall naturally, replacing the complex mechanical structures required by traditional production lines, significantly reducing equipment costs and maintenance expenses. Simultaneously, the compact, enclosed dark box platform avoids stray light interference, ensuring the stability of data acquisition. A photoelectric sensor triggers a spectrometer and an industrial camera to simultaneously acquire multi-source data. Combined with feature enhancement and fusion algorithms, the internal chemical information of the spectrum is complemented by the external physical features of the image, improving detection accuracy. Ultimately, this achieves efficient, batch-scale non-destructive testing, providing a low-cost, high-precision automated solution for the quality grading of spherical fruits such as Sanhua plums.
[0005] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion, comprising the following steps:
[0006] Step 1: Set up the testing platform. Build a free-fall device on the platform to allow the Sanhua plums to fall naturally.
[0007] Step 2: Data Acquisition and Parameter Optimization. After the fruit is released, it falls freely, and the photoelectric sensor triggers the spectrometer and industrial camera to collect data synchronously.
[0008] Step 3: Data preprocessing and feature extraction. The spectral data is preprocessed with Savitzky-Golay smoothing combined with standard normal variable correction, and the CARS algorithm is used to extract feature wavelengths. The visual image data is grayscaled and the statistical features of grayscale values are extracted.
[0009] Step 4: Multi-source information fusion modeling. The spectral features and image features are normalized by Z-score, enhanced by polynomial features and filtered by variance threshold. A multi-source information prediction model is constructed by linear weighted fusion to output the soluble solids content.
[0010] A further improvement is that the platform in step one includes a black box, a motor-driven wheel, a baffle, a spectrometer, an industrial camera, and a photoelectric sensor.
[0011] The further improvement is that in step two, the spectrometer covers a wavelength range of 400~1100nm, the white balance coefficient of the industrial camera is set to 2.242, and the exposure time is 1000ms.
[0012] The further improvement is that in step two, the motor speed is set to three levels: 6.6, 13.3, and 20 r / min; the integration time is 8, 11, and 14 ms; and the light spot diameter is 20 and 50 mm, with the light source illuminating the equatorial surface of the fruit.
[0013] A further improvement is made in that the CARS algorithm in step three includes the following steps:
[0014] S1. Initialization: Input the preprocessed spectral data, and set the number of sampling rounds and cross-validation folds;
[0015] S2. Weighted sampling: In each round of sampling, the regression coefficients of the PLSR model are calculated. The absolute value of the coefficients is used as the wavelength importance index. A weighted sampling probability distribution is constructed according to the importance ratio, and wavelengths with high importance have a higher probability of being selected.
[0016] S3, Exponential decay, uses an exponential function to dynamically reduce the number of wavelengths retained;
[0017] S4. Model evaluation: Establish a PLSR model for the wavelength subset obtained from each round of sampling, and calculate the root mean square error through cross-validation.
[0018] The S5 output results in a set of simplified characteristic wavelengths for subsequent modeling.
[0019] A further improvement lies in the following: the formula for the exponential function in the exponential decay is:
[0020]
[0021] Where, n k n is the number of wavelengths retained in the k-th round, n0 is the initial number of wavelengths, and μ is the attenuation coefficient.
[0022] A further improvement is made in step four, where the extracted spectral features and visual features are randomly divided into training and testing sets in a 7:3 ratio, and partial least squares regression is used to build the model.
[0023] The beneficial effects of this invention are as follows: By utilizing gravity to drive the fruit to fall naturally, this invention replaces the complex mechanical structure required by traditional production lines, significantly reducing equipment costs and maintenance costs. At the same time, the compact, enclosed dark box platform avoids stray light interference, ensuring the stability of data acquisition. By triggering a spectrometer and an industrial camera to synchronously acquire multi-source data through a photoelectric sensor, and combining feature enhancement and fusion algorithms, the internal chemical information of the spectrum and the external physical features of the image are complemented, improving detection accuracy. Ultimately, this invention achieves efficient, batch-scale non-destructive testing, providing a low-cost, high-precision automated solution for the quality grading of spherical fruits such as Sanhua plum. Attached Figure Description
[0024] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation
[0025] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.
[0026] The advancements in sensing, spectroscopy, and algorithms are propelling the non-destructive testing of agricultural product internal quality from the laboratory to the field and production line. Visible / near-infrared spectroscopy, electronic noses, and machine vision can instantly capture information on soluble solids content, acidity, and defects without damaging the fruit, becoming the core of modern grading equipment. Currently, the application of visible / near-infrared spectroscopy technology in online soluble solids detection mainly follows two routes: "assembly line" and "portable." However, assembly line equipment not only requires long-distance belt conveyors and turntables but also necessitates high-power motors, frequency converters, and multi-stage reduction mechanisms. It occupies a large area and imposes stringent requirements on foundation flatness and three-phase power supply. The complex mechanical structure is extremely expensive, and subsequent maintenance requires regular replacement of belts, bearings, and lubrication systems, resulting in high annual maintenance costs. Hilly orchards are often fragmented with narrow roads, making it difficult for large equipment to be installed. Small processing plants, limited by factory space and power capacity, are often forced to abandon automated grading. In contrast, while portable devices allow for individual operation, they rely on manual placement of each fruit, resulting in low average inspection efficiency. Furthermore, the varying contact pressure between the probe and the fruit surface exacerbates accuracy drift. While machine vision technology can complete appearance grading in milliseconds, its limitations are significant when used alone for internal quality inspection: it only analyzes surface RGB / grayscale information, lacking a direct response to the chemical composition of the fruit pulp; the correlation between peel color and soluble solids content is weak; and the misjudgment rate is high when surface defects do not correlate with internal quality. Dust, water film, abrasions, or changes in lighting can all cause feature drift, forcing frequent system recalibration. Fruit quality models relying solely on single image data exhibit large fluctuations in prediction errors, severely restricting industrial applications.
[0027] Example 1
[0028] according to Figure 1 As shown, this embodiment provides a non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion, including the following steps:
[0029] Step 1: Platform Setup. A free-fall device is installed on the platform to allow the plums to fall naturally. The platform includes a black box, a motor-driven wheel, a baffle, a spectrometer, an industrial camera, and photoelectric sensors. The main body of the platform is the black box, inside which is a motor-driven wheel that holds the fruit in place. When the motor starts, it drives the wheel to rotate, and the baffle opens instantly, allowing the plums to fall naturally. Two sets of photoelectric switch sensors are arranged along the fall path: the first set triggers the industrial camera (model MER2-160-249U3M-L-HS) to capture high-definition visual images; the second set triggers the spectrometer (model QE pro, covering the 400-1100 nm band) to simultaneously record the visible / near-infrared spectrum. The platform also includes components such as a light source, optical fiber, data cable, and host computer to ensure the synchronization and stability of data acquisition. This solves the problems of large footprint, high cost, and low efficiency of traditional assembly line equipment and portable devices. The free-fall method utilizes gravity drive, eliminating the need for complex mechanical structures, reducing equipment costs and maintenance requirements; the enclosed black box avoids stray light interference, ensuring data quality.
[0030] Step 2: Data Acquisition and Parameter Optimization. After the fruit is released, it falls freely. The photoelectric sensor triggers the spectrometer and industrial camera to collect data synchronously. The spectrometer covers a wavelength range of 400~1100nm, and the white balance coefficient of the industrial camera is set to 2.242 with an exposure time of 1000ms.
[0031] The motor speed was set to 13.3 r / min at three speeds, the integration time to 14 ms, and the light spot diameter to 20 mm, with the fruit's equatorial surface illuminated by the light source. Using 70 samples, the optimal settings were found by adjusting key parameters such as motor speed, integration time, light spot diameter, and fruit posture. During data acquisition, the equipment was calibrated first, and then the spectral and image data of each sample were recorded, repeated three times and the average value was taken. Based on the optimization results, data acquisition was performed on the remaining 90 samples. During parameter optimization, the optimal values were determined by comparing the average spectrum and modeling effect under different settings. Parameter optimization ensured the efficiency and accuracy of data acquisition: extending the integration time and using a small light spot enhanced the signal-to-noise ratio of the spectral signal and reduced scattering interference; fixing the optimal parameters improved data repeatability and provided stable input for modeling. Synchronous acquisition avoided timing errors, ensuring a one-to-one correspondence between spectral and image data, creating a prerequisite for multi-source fusion.
[0032] Step 3: Data Preprocessing and Feature Extraction. The collected raw data is processed, divided into spectral and visual image data. The spectral data undergoes Savitzky-Golay smoothing combined with standard normality correction. Savitzky-Golay smoothing filters out high-frequency fluctuations, and standard normality correction eliminates optical path differences. The CARS algorithm is then used to extract feature wavelengths. The visual image data is converted to grayscale, and statistical features of the grayscale values are extracted. Preprocessing eliminates errors caused by environmental factors such as dust or changes in lighting, improving data quality. Feature extraction reduces data dimensionality, focuses on key wavelengths, and avoids overfitting. The CARS algorithm includes the following steps:
[0033] S1. Initialization: Input the preprocessed spectral data, and set the number of sampling rounds and cross-validation folds;
[0034] S2. Weighted sampling: In each round of sampling, the regression coefficients of the PLSR model are calculated. The absolute value of the coefficients is used as the wavelength importance index. A weighted sampling probability distribution is constructed according to the importance ratio, and wavelengths with high importance have a higher probability of being selected.
[0035] S3, Exponential decay, uses an exponential function to dynamically reduce the number of wavelengths retained;
[0036] S4. Model evaluation: Establish a PLSR model for the wavelength subset obtained from each round of sampling, and calculate the root mean square error through cross-validation.
[0037] The S5 output results in a set of simplified characteristic wavelengths for subsequent modeling.
[0038] The formula for the exponential function in exponential decay is:
[0039]
[0040] Where, n k n is the number of wavelengths retained in the k-th round, n0 is the initial number of wavelengths, and μ is the attenuation coefficient.
[0041] Step 4: Multi-source information fusion modeling. Spectral and image features are Z-score normalized, multinomial feature enhancement is applied, and variance thresholding is used for selection. A multi-source information prediction model is constructed through linear weighted fusion, outputting the soluble solids content. The extracted spectral and visual features are randomly divided into training and test sets in a 7:3 ratio, and a partial least squares regression model is built. Z-score normalization (mean 0, standard deviation 1) is applied to the extracted features to eliminate dimensional differences. First-order multinomial feature enhancement is then used to generate interaction terms to uncover nonlinear relationships between features. Redundant features with low variance are removed, retaining 45 important features. These selected features are concatenated and input into the linear regression model. Weighted fusion of spectral and image features is achieved through weight learning. Finally, a multi-source information fusion model is established, outputting the predicted soluble solids content. Multi-source fusion overcomes the limitations of single data: spectra directly reflect internal chemical changes, while images capture external physical features. It achieves efficient and accurate non-destructive testing, providing a feasible solution for the automated grading of spherical fruits such as Sanhua plum.
[0042] The test results showed that the model's R² was 0.8389 and RMSE was 0.4976. The integration time was relatively high at 14 ms. The 20 mm spot diameter improved the focusing energy and signal-to-noise ratio. The appropriate motor speed of 13.3 r / min ensured efficient acquisition.
[0043] Example 2
[0044] according to Figure 1 As shown, this embodiment provides a non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion, including the following steps:
[0045] Step 1: Platform Setup. A free-fall device is installed on the platform to allow the plums to fall naturally. The platform includes a black box, a motor-driven wheel, a baffle, a spectrometer, an industrial camera, and photoelectric sensors. The main body of the platform is the black box, inside which is a motor-driven wheel that holds the fruit in place. When the motor starts, it drives the wheel to rotate, and the baffle opens instantly, allowing the plums to fall naturally. Two sets of photoelectric switch sensors are arranged along the fall path: the first set triggers the industrial camera (model MER2-160-249U3M-L-HS) to capture high-definition visual images; the second set triggers the spectrometer (model QE pro, covering the 400-1100 nm band) to simultaneously record the visible / near-infrared spectrum. The platform also includes components such as a light source, optical fiber, data cable, and host computer to ensure the synchronization and stability of data acquisition. This solves the problems of large footprint, high cost, and low efficiency of traditional assembly line equipment and portable devices. The free-fall method utilizes gravity drive, eliminating the need for complex mechanical structures, reducing equipment costs and maintenance requirements; the enclosed black box avoids stray light interference, ensuring data quality.
[0046] Step 2: Data Acquisition and Parameter Optimization. After the fruit is released, it falls freely. The photoelectric sensor triggers the spectrometer and industrial camera to collect data synchronously. The spectrometer covers a wavelength range of 400~1100nm, and the white balance coefficient of the industrial camera is set to 2.242 with an exposure time of 1000ms.
[0047] The motor speed was set to 6.6 r / min at three speeds, the integration time to 11 ms, and the light spot diameter to 20 mm, with the fruit's equatorial surface illuminated by the light source. Using 70 samples, the optimal settings were found by adjusting key parameters such as motor speed, integration time, light spot diameter, and fruit posture. During data acquisition, the equipment was calibrated first, and then the spectral and image data of each sample were recorded, repeated three times and averaged. Based on the optimization results, data was acquired for the remaining 90 samples. During parameter optimization, the optimal values were determined by comparing the average spectrum and modeling effect under different settings. Parameter optimization ensured the efficiency and accuracy of data acquisition: extending the integration time and using a small light spot enhanced the signal-to-noise ratio of the spectral signal and reduced scattering interference; fixing the optimal parameters improved data repeatability and provided stable input for modeling. Synchronous acquisition avoided timing errors, ensuring a one-to-one correspondence between spectral and image data, creating a prerequisite for multi-source fusion.
[0048] Step 3: Data Preprocessing and Feature Extraction. The collected raw data is processed, divided into spectral and visual image data. The spectral data undergoes Savitzky-Golay smoothing combined with standard normality correction. Savitzky-Golay smoothing filters out high-frequency fluctuations, and standard normality correction eliminates optical path differences. The CARS algorithm is then used to extract feature wavelengths. The visual image data is converted to grayscale, and statistical features of the grayscale values are extracted. Preprocessing eliminates errors caused by environmental factors such as dust or changes in lighting, improving data quality. Feature extraction reduces data dimensionality, focuses on key wavelengths, and avoids overfitting. The CARS algorithm includes the following steps:
[0049] S1. Initialization: Input the preprocessed spectral data, and set the number of sampling rounds and cross-validation folds;
[0050] S2. Weighted sampling: In each round of sampling, the regression coefficients of the PLSR model are calculated. The absolute value of the coefficients is used as the wavelength importance index. A weighted sampling probability distribution is constructed according to the importance ratio, and wavelengths with high importance have a higher probability of being selected.
[0051] S3, Exponential decay, uses an exponential function to dynamically reduce the number of wavelengths retained;
[0052] S4. Model evaluation: Establish a PLSR model for the wavelength subset obtained from each round of sampling, and calculate the root mean square error through cross-validation.
[0053] The S5 output results in a set of simplified characteristic wavelengths for subsequent modeling.
[0054] The formula for the exponential function in exponential decay is:
[0055]
[0056] Where, n k n is the number of wavelengths retained in the k-th round, n0 is the initial number of wavelengths, and μ is the attenuation coefficient.
[0057] Step 4: Multi-source information fusion modeling. Spectral and image features are Z-score normalized, multinomial feature enhancement is applied, and variance thresholding is used for selection. A multi-source information prediction model is constructed through linear weighted fusion, outputting the soluble solids content. The extracted spectral and visual features are randomly divided into training and test sets in a 7:3 ratio, and a partial least squares regression model is built. Z-score normalization (mean 0, standard deviation 1) is applied to the extracted features to eliminate dimensional differences. First-order multinomial feature enhancement is then used to generate interaction terms to uncover nonlinear relationships between features. Redundant features with low variance are removed, retaining 45 important features. These selected features are concatenated and input into the linear regression model. Weighted fusion of spectral and image features is achieved through weight learning. Finally, a multi-source information fusion model is established, outputting the predicted soluble solids content. Multi-source fusion overcomes the limitations of single data: spectra directly reflect internal chemical changes, while images capture external physical features. It achieves efficient and accurate non-destructive testing, providing a feasible solution for the automated grading of spherical fruits such as Sanhua plum.
[0058] The test results showed that the model's R² was 0.7114 and RMSE was 0.3309. The relatively high integration time of 11 ms, the spot diameter of 20 mm, and the improved focusing energy enhanced the signal-to-noise ratio. The motor speed of 6.6 r / min reduced the integration area and lowered noise interference.
[0059] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion, comprising the following steps: Step 1: Set up the testing platform. Build a free-fall device on the platform to allow the Sanhua plums to fall naturally. Step 2: Data Acquisition and Parameter Optimization. After the fruit is released, it falls freely, and the photoelectric sensor triggers the spectrometer and industrial camera to collect data synchronously. Step 3: Data preprocessing and feature extraction. The spectral data is preprocessed with Savitzky-Golay smoothing combined with standard normal variable correction, and the CARS algorithm is used to extract feature wavelengths. The visual image data is grayscaled and the statistical features of grayscale values are extracted. Step 4: Multi-source information fusion modeling. The spectral features and image features are normalized by Z-score, enhanced by polynomial features and filtered by variance threshold. A multi-source information prediction model is constructed by linear weighted fusion to output the soluble solids content.
2. The method for non-destructive testing of the internal quality of *Prunus triloba* based on free-fall multi-source information fusion according to claim 1, characterized in that: The platform in step one includes a black box, a motor-driven wheel, a baffle, a spectrometer, an industrial camera, and a photoelectric sensor.
3. The method for non-destructive testing of the internal quality of *Prunus triloba* based on free-fall multi-source information fusion according to claim 1, characterized in that: In step two, the spectrometer covers a wavelength range of 400~1100nm, the industrial camera's white balance coefficient is set to 2.242, and the exposure time is 1000ms.
4. The non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion according to claim 1, characterized in that: In step two, the motor speed is set to three levels: 6.6, 13.3, and 20 r / min; the integration time is 8, 11, and 14 ms; and the light spot diameter is 20 and 50 mm, respectively, so that the equatorial surface of the fruit is illuminated by the light source.
5. The non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion according to claim 1, characterized in that: The CARS algorithm in step three includes the following steps: S1. Initialization: Input the preprocessed spectral data, and set the number of sampling rounds and cross-validation folds; S2. Weighted sampling: In each round of sampling, the regression coefficients of the PLSR model are calculated. The absolute value of the coefficients is used as the wavelength importance index. A weighted sampling probability distribution is constructed according to the importance ratio, and wavelengths with high importance have a higher probability of being selected. S3, Exponential decay, uses an exponential function to dynamically reduce the number of wavelengths retained; S4. Model evaluation: Establish a PLSR model for the wavelength subset obtained from each round of sampling, and calculate the root mean square error through cross-validation. The S5 output results in a set of simplified characteristic wavelengths for subsequent modeling.
6. The method for non-destructive testing of the internal quality of *Prunus triloba* based on free-fall multi-source information fusion according to claim 5, characterized in that: The formula for the exponential function in the exponential decay is: Where, n k n is the number of wavelengths retained in the k-th round, n0 is the initial number of wavelengths, and μ is the attenuation coefficient.
7. The non-destructive testing method for the internal quality of *Prunus triloba* based on free-fall multi-source information fusion according to claim 1, characterized in that: In step four, the extracted spectral features and visual features are randomly divided into training and test sets in a 7:3 ratio, and a model is established using partial least squares regression.