Apple quality nondestructive testing method based on smart phone with miniature hyperspectral lens

Abstract

The invention relates to the technical field of agricultural product nondestructive testing, in particular to an apple quality nondestructive testing method based on a miniature hyperspectral lens smartphone, which comprises the following steps: optically coupling a miniature hyperspectral lens with a main camera of the smartphone to form a portable acquisition system; the method comprises the following steps: acquiring hyperspectral image data of an apple sample to obtain three-dimensional hyperspectral data; an apple surface region of interest is extracted from the three-dimensional data, an average reflection spectrum is calculated, and a feature selection algorithm is adopted to screen feature wavelengths related to quality indexes to form feature vectors; and finally inputting the feature vector into the quality prediction model to obtain a prediction result, and displaying the prediction result through the smart phone. According to the invention, portability and low cost of a hyperspectral detection technology are realized by using a smart phone platform, the technical problems of large size, high cost and dependence on a laboratory environment of existing equipment are effectively solved, and the system is suitable for rapid quality detection of field scenes such as orchards and packaging lines.

Description

Technical Field

[0001] This invention relates to the field of non-destructive testing technology for agricultural products, and in particular to a non-destructive testing method for the quality of apples based on a smartphone with a miniature hyperspectral lens. Background Technology

[0002] Existing hyperspectral detection technologies suffer from the following technical challenges: Firstly, existing hyperspectral imaging systems are bulky and heavy due to their core spectroscopic elements and complex optical structures, making them difficult to deploy and move around in orchards or on product packaging lines. Secondly, the high cost of these devices makes them unaffordable for small and medium-sized orchards or food processing enterprises. Thirdly, the systems are extremely sensitive to lighting conditions and mechanical stability, requiring a stable laboratory environment to obtain reliable data. They cannot adapt to the light fluctuations and vibrations present in field or workshop environments, ultimately preventing the application of advanced spectral detection technologies for rapid on-site quality screening and grading. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a non-destructive testing method for Apple products based on a miniature hyperspectral lens for smartphones. This method solves the technical problem that existing hyperspectral testing equipment is bulky, expensive, and dependent on laboratory environments, making it impossible to apply quickly and portablely in field scenarios such as orchards and packaging lines.

[0004] To solve the above-mentioned technical problems, the specific contents of the present invention are as follows:

[0005] The present invention provides a non-destructive testing method for Apple smartphones based on a miniature hyperspectral lens, comprising:

[0006] Step 1: Use a smartphone coupled with a miniature hyperspectral lens to acquire hyperspectral image data of an apple sample. Preprocess the hyperspectral image data to obtain corrected reflectance data. Perform image cube reconstruction on the corrected reflectance data to obtain three-dimensional data including spatial and spectral dimensions.

[0007] Step 2: Extract the region of interest (ROI) from the reconstructed 3D data of the apple surface, calculate the average reflectance spectrum within the ROI, and select the characteristic wavelengths associated with apple quality indicators from the average reflectance spectrum to form a feature vector.

[0008] Step 3: Input the feature vector into the quality prediction model to obtain the quality index prediction results for Apple, and display the quality index prediction results on a smartphone.

[0009] Furthermore, in the non-destructive testing method for Apple products based on a miniature hyperspectral lens smartphone described in this invention, step 1 includes:

[0010] Display a location guide box on the smartphone screen;

[0011] After placing the standard whiteboard at the center of the acquisition field of view according to the positioning guide frame, the hyperspectral image of the standard whiteboard is acquired by the coupled system and used as white reference data.

[0012] Dark current images were acquired when the lens was completely blocked, serving as dark reference data.

[0013] The preprocessing includes converting the digital brightness values ​​in the original data of the collected apple samples into reflectance data using the white reference data and dark reference data.

[0014] Furthermore, in the non-destructive testing method for Apple smartphones based on miniature hyperspectral lenses described in this invention, step 1 further includes:

[0015] Based on the pre-calibrated spectral-spatial mapping relationship of the miniature hyperspectral lens, the two-dimensional hybrid image corresponding to the corrected reflectance data is decoded;

[0016] The spatial images corresponding to each wavelength obtained after decoding are arranged in wavelength order to generate the three-dimensional data.

[0017] Furthermore, in the non-destructive testing method for Apple products based on a miniature hyperspectral lens for smartphones described in this invention, step 2 includes:

[0018] Feature bands are selected from the three-dimensional data to generate a grayscale image;

[0019] Adaptive thresholding is applied to grayscale images to initially distinguish the apple region from the background;

[0020] Edge detection is performed on the initially segmented image to locate the apple outline;

[0021] Define the pixel region within the outline as the region of interest.

[0022] Furthermore, in the non-destructive testing method for Apple smartphones based on miniature hyperspectral lenses described in this invention, step 3 includes: the selection of feature wavelengths is carried out by using a competitive adaptive reweighted sampling method, a continuous projection algorithm, or a random frog-jumping algorithm to identify the wavelength combination that contributes the most to the prediction of quality indicators from the average reflectance spectrum, so as to form a feature vector.

[0023] Furthermore, in the non-destructive testing method for Apple smartphones based on miniature hyperspectral lenses described in this invention, step 3 further includes:

[0024] Extract image texture features of the region of interest in a specific band from the three-dimensional data;

[0025] The image texture features are fused with the feature vector obtained from the average reflectance spectrum to generate a fused feature vector;

[0026] The input of the feature vector into the quality prediction model refers to inputting the fused feature vector into the quality prediction model.

[0027] Furthermore, the non-destructive testing method for Apple smartphones based on miniature hyperspectral lenses described in this invention includes the following steps in constructing the quality prediction model:

[0028] Collect a sample set of apples with known physicochemical properties;

[0029] Collect and process hyperspectral image data of apple sample sets to obtain the feature vector of each sample;

[0030] By using machine learning or deep learning algorithms, a mapping relationship between feature vectors and physicochemical index values ​​is established, thereby obtaining a quality prediction model.

[0031] Beneficial effects of this invention;

[0032] The technical solution provided by this invention achieves portability and cost reduction of existing hyperspectral detection systems through hardware coupling between smartphones and miniature hyperspectral lenses, enabling high-precision spectral analysis technology to be applied to on-site scenarios such as orchards and packaging lines. The system calibration and reflectivity correction methods effectively eliminate the influence of ambient light fluctuations and equipment noise, significantly improving the reliability of data acquisition in complex environments. Through image cube reconstruction and feature wavelength selection algorithms, high-dimensional spectral data is condensed into key feature vectors, preserving the spectral information most relevant to quality indicators while significantly reducing data processing complexity. Combining multi-source feature fusion technology with spectral and texture information enhances the model's comprehensive judgment ability on the internal and external quality of apples. Finally, through a cloud-collaborative intelligent algorithm model, rapid prediction and visualization of quality indicators are achieved, providing a complete technical solution for on-site quality testing of agricultural products. Attached Figure Description

[0033] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0034] Figure 1 This is a flowchart of the non-destructive testing method for Apple smartphones based on a miniature hyperspectral lens, according to the present invention. Detailed Implementation

[0035] To make the technical solution of the present invention clearer, the present invention will be clearly and completely described below with reference to specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The present invention provided by various embodiments will be described in detail below with reference to the accompanying drawings. To better understand the purpose of the present invention, the present invention will be described in further detail below.

[0036] Please see Figure 1 The present invention provides a non-destructive testing method for Apple smartphones based on a miniature hyperspectral lens, comprising:

[0037] Step 1: Use a smartphone coupled with a miniature hyperspectral lens to acquire hyperspectral image data of an apple sample. Preprocess the hyperspectral image data to obtain corrected reflectance data. Perform image cube reconstruction on the corrected reflectance data to obtain three-dimensional data including spatial and spectral dimensions.

[0038] Step 2: Extract the region of interest (ROI) from the reconstructed 3D data of the apple surface, calculate the average reflectance spectrum within the ROI, and select the characteristic wavelengths associated with apple quality indicators from the average reflectance spectrum to form a feature vector.

[0039] Step 3: Input the feature vector into the quality prediction model to obtain the quality index prediction results for Apple, and display the quality index prediction results on a smartphone.

[0040] In step 1, the system initiates the data acquisition process through hardware coupling between the smartphone and the miniature hyperspectral lens. The miniature hyperspectral lens disperses the incident light, forming a two-dimensional hybrid image data on the smartphone's main camera image sensor, which includes superimposed one-dimensional spatial and one-dimensional spectral information. The acquired raw hyperspectral image data needs to be preprocessed to eliminate environmental interference. The preprocessing first performs reflectivity correction, using a pre-acquired standard white board image as white reference data and a dark current image obtained after completely blocking the lens as dark reference data. The raw digital brightness values ​​are converted into reflectivity data that can reflect the true spectral characteristics of the apple surface through calculation. Subsequently, an image cube reconstruction operation is performed on the corrected reflectivity data. Based on the optical calibration parameters of the miniature hyperspectral lens, the two-dimensional hybrid image data is decoded and rearranged into a three-dimensional hyperspectral data cube with two spatial dimensions and one spectral dimension. This step transforms the raw sensor data into standard structured data that can be used for subsequent quantitative analysis.

[0041] Step 2 extracts key information for analysis from the reconstructed 3D hyperspectral data cube. First, image processing algorithms automatically identify the apple surface region. Specific feature bands are selected from the 3D data to generate a grayscale image. An adaptive threshold segmentation method is used to initially distinguish the apple from the background. Then, an edge detection algorithm is used to precisely locate the apple's outline, and the set of pixels within the outline is determined as the region of interest. Next, the average reflectance spectrum of all pixels within the region of interest is calculated. This spectral curve represents the overall spectral characteristics of the apple sample. To further focus on key information, a feature wavelength selection algorithm is used to screen the feature wavelengths most relevant to apple quality indicators such as sugar content and acidity from the full-band average reflectance spectrum. The reflectance values ​​at each feature wavelength are combined to form a feature vector characterizing apple quality.

[0042] Step 3 inputs the feature vector into a pre-trained quality prediction model for analysis and calculation. The quality prediction model is built by training a large number of apple samples with known physicochemical indicators using machine learning algorithms, enabling it to learn the complex mapping relationship between feature vectors and apple quality indicators. After processing the input feature vector, the model outputs the predicted quality indicators of the apples, including quantitative indicators such as sugar content and acidity, or classification results of internal diseases. Finally, a smartphone application displays the quality indicator prediction results on the screen in real time in the form of numerical values, grades, or visual charts, completing the entire detection process from data collection to result output. Each step is tightly linked through data flow, with the output of the previous step serving as the input for the next, forming a coherent automated processing chain.

[0043] Before acquiring hyperspectral image data of the apple sample, a system calibration step is also included: a positioning guide frame is displayed on the smartphone screen, which provides the user with a visual alignment reference, enabling the standard white board to accurately cover the acquisition field of view; after placing the standard white board at the center of the acquisition field of view according to the positioning guide frame, the hyperspectral image of the standard white board is acquired through the coupled system to obtain reference data under ideal reflection conditions as white reference data; dark current images are acquired with the lens completely blocked to measure the noise benchmark of the sensor itself as dark reference data; the preprocessing process specifically involves: using the white reference data and dark reference data to perform calibration calculations on the original data of the apple sample, converting the original digital brightness values ​​affected by ambient light into standardized reflectivity data, and eliminating errors introduced by device noise and ambient light fluctuations.

[0044] The image cube reconstruction process is specifically implemented as follows: based on the spectral-spatial mapping relationship pre-calibrated by the monochromator calibration device when the miniature hyperspectral lens leaves the factory, the two-dimensional mixed image after reflectance correction is decoded; the decoding process separates and restores the compressed two-dimensional image data by analyzing the spectral intensity distribution corresponding to each pixel row; the spatial images corresponding to each wavelength obtained after decoding are arranged and combined in order of wavelength from smallest to largest to generate a three-dimensional hyperspectral data cube including complete spatial information and continuous spectral information.

[0045] The process of extracting the region of interest (ROI) on the apple surface includes: selecting the band that best highlights the apple's surface features from the 3D data to generate a grayscale image; this band is typically chosen near the absorption peak of the apple's skin pigments; applying an adaptive threshold segmentation algorithm to the generated grayscale image, dynamically determining the segmentation threshold based on the local brightness characteristics of the image to achieve preliminary separation of the apple region from the background; applying the Canny edge detection operator to the preliminarily segmented image to accurately locate the apple's contour boundary coordinates; and mapping all pixel coordinates within the contour boundary back to the original 3D data to determine the final ROI used for spectral analysis.

[0046] Three feature selection algorithms can be used to screen characteristic wavelengths: the competitive adaptive reweighted sampling method dynamically optimizes the weight coefficients of wavelength variables through cyclic sampling and PLS regression analysis; the continuous projection algorithm gradually selects the combination of wavelength variables with the most information through vector projection operations; and the random frog jumping algorithm uses Markov chain Monte Carlo simulation to randomly search in the wavelength space and evaluate the predictive performance of different wavelength subsets. All three algorithms can identify the characteristic wavelength combination that contributes the most to the prediction of quality indicators from the full-band spectrum, forming a dimension-reduced feature vector.

[0047] After calculating the average reflectance spectrum, a feature fusion step can be added: extract image texture features of the region of interest in specific bands from the 3D data. Each band is usually selected in the visible light region to better characterize the surface texture. The texture feature extraction uses the gray-level co-occurrence matrix calculation method to obtain statistics such as contrast, correlation, and entropy. The extracted image texture features and spectral feature vectors are fused with multi-source data to generate a fused feature vector that includes both spectral and texture information. Finally, the fused feature vector is input into the quality prediction model to improve the detection accuracy of surface defects and internal quality of apples.

[0048] The process of constructing a quality prediction model includes: collecting a sample set of apples with known physicochemical index values, covering different origins, varieties, and maturity levels; collecting hyperspectral image data of the apple sample set and performing standardized preprocessing to obtain the spectral feature vector of each sample; using machine learning algorithms such as partial least squares regression to establish a quantitative relationship between spectral features and continuous quality indicators, or using support vector machine classification algorithms to establish a discriminant model between spectral features and discrete disease categories; optimizing model parameters through cross-validation to finally obtain a practically applicable quality prediction model.

[0049] In the specific implementation of this invention, the system hardware is first assembled, and the miniature hyperspectral lens is optically coupled to the main camera of a smartphone through a dedicated interface. The miniature hyperspectral lens adopts image plane diffraction grating technology, and its spectral range covers the visible light to near-infrared bands. In actual application in orchards, a portable auxiliary lighting unit is equipped to stabilize the lighting conditions.

[0050] Before starting the test, a system calibration procedure must be performed. The smartphone application interface displays a positioning guide; the operator follows the guide to place the standard whiteboard at the center of the acquisition field of view. The system acquires a hyperspectral image of the whiteboard as white reference data, and then acquires a dark current image as dark reference data with the lens completely blocked. This calibration process effectively eliminates the influence of ambient light fluctuations, ensuring the accuracy of subsequent data.

[0051] After calibration, apple sample detection can begin. The apple is placed in the detection area, and its hyperspectral image data is acquired through a coupling system. In the preprocessing stage, reflectance conversion is performed on the raw digital brightness values ​​using white and dark reference data to obtain standardized reflectance data. Subsequently, based on the pre-calibrated spectral-spatial mapping relationship of the lens, the two-dimensional mixed image data is decoded and reconstructed to generate a three-dimensional hyperspectral data cube containing complete spatial and spectral information.

[0052] Next, feature extraction is performed. Feature bands are selected from the 3D data to generate a grayscale image. An adaptive threshold segmentation algorithm is used to distinguish the apple region from the complex background, and edge detection is then used to accurately locate the apple's outline. After identifying the pixel region within the outline as the region of interest, the average reflectance spectrum curve of that region is calculated. A competitive adaptive reweighted sampling method is used to select the feature wavelengths most relevant to quality indicators such as sugar content and acidity from the full-band spectrum, forming a feature vector.

[0053] For surface defect detection, texture features of the region of interest in specific wavelength bands can be further extracted. Texture parameters such as contrast and correlation are calculated using the gray-level co-occurrence matrix, and the texture features are fused with spectral feature vectors to generate a multidimensional feature vector. This feature vector is then input into a pre-trained quality prediction model, which is built using partial least squares regression and can simultaneously output quantitative indicators such as sugar content and acidity, as well as disease classification results such as moldy core.

[0054] The final test results are displayed in real time via a smartphone application in numerical and visual chart formats, and historical data can be queried and exported. Through hardware integration and algorithm optimization, the system has transformed existing hyperspectral detection technology for field applications, effectively solving the technical challenges of using large equipment in orchards, packaging lines, and other similar settings.

Claims

1. A nondestructive detection method for apple quality based on a miniature hyperspectral lens smartphone, characterized in that, The application relates to a method for predicting the quality of apples by using a mobile phone, and belongs to the technical field of apple quality prediction. The method comprises the following steps: Step 1: coupling a micro-hyperspectral lens with a mobile phone to collect hyperspectral image data of an apple sample, pre-processing the hyperspectral image data to obtain corrected reflectance data, and reconstructing an image cube from the corrected reflectance data to obtain three-dimensional data comprising a spatial dimension and a spectral dimension; Step 2: extracting a region of interest on the surface of the apple from the reconstructed three-dimensional data, calculating an average reflectance spectrum in the region of interest, screening feature wavelengths associated with the quality indicators of the apple from the average reflectance spectrum to form a feature vector; 2. The method according to claim 1, wherein, Step 3: inputting the feature vector into a quality prediction model to obtain a quality indicator prediction result of the apple, and displaying the quality indicator prediction result on the mobile phone. The step 1 comprises the following steps: displaying a positioning guide frame on the screen of the mobile phone; placing a standard white board at the center of the collection field according to the positioning guide frame, and collecting hyperspectral images of the standard white board through the coupled system as white reference data; collecting a dark current image in a state that the lens is completely blocked as dark reference data; 3.The nondestructive testing method for apple quality based on micro-hyperspectral lens smart phone according to claim 2, wherein, the pre-processing comprises converting digital brightness values in the collected original data of the apple sample into reflectance data by using the white reference data and the dark reference data. The step 1 further comprises the following steps: decoding a two-dimensional mixed image corresponding to the corrected reflectance data according to a pre-calibrated spectral-spatial mapping relationship of the micro-hyperspectral lens; 4.The nondestructive testing method for apple quality based on micro-hyperspectral lens smart phone according to claim 3, characterized in that, arranging the spatial images corresponding to each wavelength obtained after decoding in a wavelength order to generate the three-dimensional data. The step 2 comprises the following steps: selecting a feature wavelength band from the three-dimensional data to generate a gray-scale image; performing adaptive threshold segmentation on the gray-scale image to preliminarily distinguish the apple region from the background; performing edge detection on the preliminarily segmented image to locate the outline of the apple; 5.The nondestructive testing method for apple quality based on micro-hyperspectral lens smart phone of claim 1, wherein, determining a pixel region in the outline as the region of interest. 6.The nondestructive testing method for apple quality based on micro-hyperspectral lens smart phone according to claim 1, wherein, The step 3 comprises the following step: the feature wavelengths are screened by using a competitive adaptive reweighted sampling method, a successive projections algorithm or a random frog algorithm to identify a wavelength combination having the largest contribution to the prediction of the quality indicators from the average reflectance spectrum to form the feature vector. The step 3 further comprises the following steps: extracting image texture features of the region of interest at a specific wavelength band from the three-dimensional data; fusing the image texture features with the feature vector obtained from the average reflectance spectrum to generate a fused feature vector; 7.The nondestructive testing method for apple quality based on micro-hyperspectral lens smart phone of claim 1, wherein, the feature vector is inputted into the quality prediction model by inputting the fused feature vector into the quality prediction model. The construction of the quality prediction model comprises the following steps: collecting an apple sample set with known physicochemical indicator values; collecting hyperspectral image data of the apple sample set and processing the same to obtain a feature vector of each sample; establishing a mapping relationship between the feature vector and the physicochemical indicator values by using a machine learning or deep learning algorithm to obtain the quality prediction model.

Images