Fruit quality detection method and system based on fruit quality detection model

Abstract

This invention discloses a fruit quality detection method and system based on a fruit quality detection model. First, the original hyperspectral image R of the fruit to be detected is acquired and preprocessed. Then, the preprocessed hyperspectral image R2 is input into the fruit quality detection model to obtain the surface defect results and internal quality index values ​​of the fruit. Compared with the prior art, this invention can achieve simultaneous detection of the internal and external quality of the fruit at low cost and high efficiency, and obtain the fruit quality grade result.

Description

Technical Field

[0001] This invention belongs to the field of fruit quality testing technology, and relates to a fruit quality testing method and system, specifically a fruit quality testing method and system based on hyperspectral imaging and a fruit quality testing model. Background Technology

[0002] Traditional fruit quality testing methods mainly rely on visual inspection (Reference 1), sensory evaluation (Reference 2), and single-index detection methods of physicochemical analysis (Reference 3) to determine the quality of fruit. However, these methods have problems such as low detection accuracy, high cost, and low efficiency, which limit their practicality in actual fruit quality monitoring.

[0003] [1] Pathmanaban P, Gnanavel BK, Anandan S S. Recent application of imaging techniques for fruit quality assessment[J]. Trends in Food Science&Technology, 2019, 94: 32-42.

[0004] [2]Bavay C, Symoneaux R, Maître I, et al. Importance of fruitvariability in the assessment of apple quality by sensory evaluation[J]. Postharvest biology and technology, 2013, 77: 67-74.

[0005] [3]Farina V, Lo Bianco R, Mazzaglia A. Evaluation of late-maturingpeach and nectarine fruit quality by chemical, physical, and sensory determinations[J]. Agriculture, 2019, 9(9): 189. Summary of the Invention

[0006] To address the technical problems of high cost and unsatisfactory detection accuracy in existing fruit quality testing technologies, a method and system for fruit quality testing based on hyperspectral imaging is provided.

[0007] The technical solution adopted by the method of the present invention is: a fruit quality detection method based on a fruit quality detection model, comprising the following steps:

[0008] Step 1: Obtain the original hyperspectral image R of the fruit to be detected and perform preprocessing;

[0009] Step 2: Input the preprocessed hyperspectral image R2 into the fruit quality detection model to obtain the results of fruit surface defects and internal quality index values;

[0010] Preferably, the preprocessing described in step 1 involves black-and-white correction of the original hyperspectral image R of the fruit to obtain a corrected hyperspectral image of the fruit. Where W and B represent the white calibration image and the black calibration image, respectively;

[0011] The standard deviation of the spectral reflectance value of each pixel in the corrected hyperspectral image of the fruit is calculated to create a feature matrix. Then, a threshold is set to binarize the feature matrix, and the binarized image is processed by opening and closing operations to detect the boundary of interest. Finally, the region of interest is extracted and the noise background is removed to obtain the preprocessed hyperspectral image of the fruit, R2.

[0012] Preferably, the fruit quality detection model in step 2 includes a normalization layer, a dimensionality reduction layer, a spatial spectral feature extraction layer, and a classification, regression, and convergence layer; the normalization layer is used to normalize the preprocessed hyperspectral image R2 of the fruit; the dimensionality reduction layer is used to perform principal component analysis on the normalized hyperspectral image. Dimensionality reduction is performed to remove redundant band information, resulting in a dimensionality-reduced hyperspectral image of the fruit, R3. The spatial-spectral feature extraction layer uses a normalized attention network to extract features from the dimensionality-reduced hyperspectral image R3, obtaining rich spatial and spectral characteristics, which are then fused with six texture features: mean, contrast, correlation, energy, homogeneity, and entropy. The classification, regression, and convergence layer uses the fused values ​​of spatial-spectral and texture features as input to a least-squares support vector machine for classification, regression, and convergence. Based on surface and internal quality indicators of the fruit, and combining spectral and spatial information, a fruit detection model is constructed using a least-squares support vector machine to distinguish whether there are defects on the fruit surface and predict the values ​​of internal quality indicators.

[0013] Preferably, the normalized attention network includes an encoding module, a residual module, a decoding module, a fusion module, and a normalized attention module;

[0014] The residual module is used to connect the input hyperspectral image X with the output feature map F3 of the convolution module C3 to ensure the transmission of information flow of the encoding module; the input hyperspectral image X is the dimension-reduced fruit hyperspectral image R3.

[0015] The decoding module includes three deconvolution modules D1, D2, and D3 arranged in series; each convolution module contains a deconvolution layer, and a normalization layer and an activation layer are added sequentially after each convolution layer; the kernel size of the convolution modules C1, C2, and C3 is 3, and the stride size is 2.

[0016] The fusion module is used to pass the feature maps F1, F2 and F3 output by the convolution modules C1, C2, C3 and C4 through a deconvolution layer so that the features are at the same scale as the deconvolution modules D1, D2 and D3 respectively, and to fuse the image spatial and spectral features in the order of D1, D2 and D3 respectively.

[0017] The normalized attention module is used to refine the feature map after the deconvolution module D3 and the convolution module C1 are fused, and a batch normalization scaling factor is used in the channel direction.

[0018] The normalized attention module is followed by an unfolded layer and a fully connected layer with a softmax function to classify the refined feature map;

[0019] The encoding module, consisting of four convolutional modules and one residual module, is used to compress the features of the dimensionality-reduced hyperspectral image R3 of the fruit. The input to convolutional module C1 is the dimensionality-reduced hyperspectral image R3 of the fruit. The input to convolutional module C2 is the output feature map F1 of convolutional module C1. The input to convolutional module C3 is the output feature map F2 of convolutional module C2. The input to convolutional module C4 is the result of a residual concatenation between the output feature map F3 of convolutional module C3 and the input. Then, a deconvolution operation is performed. The decoding module, consisting of three deconvolutional modules, three fusion modules, and one normalized attention module, is used to decode the output feature F5 of the encoding module. The input to the deconvolutional module D1 of the decoding module is the... The output feature map F5 of the convolution module C4; the input of the deconvolution module D2 of the decoding module is the feature map fused from the output feature map F6 of the deconvolution module D1 and the output feature map F3 of the convolution module C3; the input of the deconvolution module D3 of the decoding module is the feature map fused from the output feature map F7 of the deconvolution module D2 and the output feature map F2 of the convolution module C2; ​​the input of the normalization attention module of the decoding module is the feature map fused from the output feature map F8 of the deconvolution module D3 and the output feature map F1 of the convolution module C1; the normalization attention module of the decoding module, after the input feature map passes through this unit, each feature map will have a weight map, representing the importance of the channel direction feature map; the output of the decoding module is the fruit quality detection result Y input by the encoding module.

[0020] Preferably, the preprocessing described in step 1 involves black-and-white correction of the original hyperspectral image R of the fruit to obtain a corrected hyperspectral image of the fruit. Where W and B represent the white calibration image and the black calibration image, respectively;

[0021] The standard deviation of the spectral reflectance value of each pixel in the corrected hyperspectral image of the fruit is calculated to create a feature matrix. Then, a threshold is set to binarize the feature matrix, and the binarized image is processed by opening and closing operations to detect the boundary of interest. Finally, the region of interest is extracted and the noise background is removed to obtain the preprocessed hyperspectral image of the fruit, R2.

[0022] Preferably, in step 2, based on the surface characteristics of the pre-processed hyperspectral image R2 of the fruit, the fruit samples are divided into two categories: whole fruit and defective fruit, where whole fruit refers to fruit without surface defects; the internal quality index values ​​of different fruits are obtained, including four indicators: hardness, soluble solids, titratable acidity, and vitamin C; different fruit quality grade rules are set according to the surface defect results and internal quality index values ​​of different fruits.

[0023] Preferably, the fruit quality detection model is a pre-trained fruit quality detection model. After training, a validation set is used to validate the training results. The validation uses two criteria: the coefficient of determination of the prediction and the root mean square error of the prediction. During training, the fruit quality detection model uses cross-entropy as the loss function and the Adam optimizer is used to optimize the model training process to obtain a pre-trained fruit quality detection model. The encoding module of the pre-trained fruit quality detection model is used to extract the spatial spectral features of the input fruit hyperspectral image and, combined with texture features, is fed into a least squares support vector machine for classification.

[0024] The technical solution adopted by the system of the present invention is: a fruit quality detection system based on a fruit quality detection model, comprising the following modules:

[0025] The fruit hyperspectral image module is used to acquire the original hyperspectral image R of the fruit to be detected and to perform preprocessing.

[0026] The fruit quality detection module is used to input the preprocessed hyperspectral image R2 into the fruit quality detection model to obtain the results of fruit surface defects and internal quality index values.

[0027] Preferably, the fruit hyperspectral image module includes an imaging lens, a hyperspectral imager, a light source, a whiteboard, a mobile platform, and a server.

[0028] Preferably, the system further includes a terminal, a wireless sensing module, and a processor; the terminal is used to send instructions to the wireless sensing module and receive feedback information; the wireless sensing module is used for the exchange of information and instructions between the terminal and the processor; and the processor is used to process the execution instructions exchanged between the wireless sensors.

[0029] Compared with existing technologies, this invention achieves simultaneous detection of the internal and external quality of fruits at low cost and high efficiency, yielding fruit quality grade results. Firstly, the acquisition of hyperspectral images of fruits is susceptible to noise interference and contains redundant background information. Therefore, this invention eliminates noise background through preprocessing operations such as black-and-white correction and automatic segmentation, avoiding manual missegmentation and extracting regions of interest. Secondly, the acquired hyperspectral images of fruits contain hundreds of bands and a large amount of spectral information, which undoubtedly poses a challenge to computer hardware. This invention uses principal component analysis to perform dimensionality reduction on the preprocessed hyperspectral images of fruits to retain the main feature information, eliminate redundant bands, and reduce subsequent data processing costs. Traditional manual subjective evaluation of fruit quality detection mainly relies on the rich experience of the inspectors, making the detection time-consuming, labor-intensive, and inefficient. This invention establishes a standardized attention network based on convolutional neural networks, simultaneously extracting a large amount of feature information from the hyperspectral images of fruits from both spatial and channel dimensions. However, due to the limited number of fruit training samples, convolutional neural networks are prone to overfitting. Therefore, this invention introduces a residual module and a fusion module to fully utilize the information flow in the network model, deepening the network while mitigating overfitting. Furthermore, since traditional convolutional neural networks typically assume equal weights for each band, in reality, each band's features have different impacts on the model results. This invention utilizes a normalized attention module to reassign weights to different bands without requiring additional training parameters, thus highlighting features that contribute better to the model's classification results while suppressing unimportant feature bands. Finally, the trained normalized attention network's encoding part obtains discriminative spatial-spectral features from the fruit hyperspectral samples and acquires six texture features—mean, contrast, correlation, energy, homogeneity, and entropy—as supplementary information. The fused values ​​of the spatial-spectral and texture features are then input into a simple least-squares support vector machine classifier to achieve high-performance fruit quality grade classification. Attached Figure Description

[0030] The technical solutions described herein are further illustrated below using examples and specific implementation methods. Additionally, accompanying drawings are used in the description of the technical solutions. Those skilled in the art can, without any creative effort, obtain other drawings and the intent of the present invention based on these drawings.

[0031] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0032] Figure 2 This is a structural diagram of the fruit quality detection model according to an embodiment of the present invention;

[0033] Figure 3 This is a system structure diagram of an embodiment of the present invention. Detailed Implementation

[0034] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0035] Please see Figure 1 The present invention provides a fruit quality detection method based on a fruit quality detection model, comprising the following steps:

[0036] Step 1: The server receives the fruit quality detection command from the terminal;

[0037] Send a fruit quality testing request command to the server's wireless communication device (such as NB-IoT) via a terminal device such as a computer or mobile phone.

[0038] Step 2: Acquiring the original hyperspectral image of the fruit;

[0039] The fruit is fed into the storage chamber, where raw hyperspectral images are acquired and preprocessed. The hyperspectral images of the fruit collected by the hyperspectral data acquisition device contain noisy backgrounds, necessitating image segmentation. However, traditional manual segmentation of regions of interest is time-consuming, labor-intensive, and prone to human error, resulting in the loss of valuable information. This invention employs an automatic segmentation method combining standard deviation and threshold binarization to segment the regions of interest from the background.

[0040] (1) After the server receives the terminal's fruit quality detection instruction, it sends the fruit into a container equipped with a high-spectrum analyzer.

[0041] The imaging device consists of a housing. The hyperspectral imaging module mainly includes an imaging lens (HSIA-OL23), a hyperspectral imager (GaiaField-V10E), a light source (200W), a whiteboard (150×150), a mobile platform, and a computer loaded with hyperspectral data acquisition software (SpecView). The hyperspectral imaging device acquires wavelengths in the range of 420~1000nm.

[0042] (2) Obtain the original hyperspectral image R of the fruit using the hyperspectral imaging module, and perform black and white correction on the original hyperspectral image R of the fruit using the matching image processing software to obtain the corrected hyperspectral image R1 of the fruit. The mathematical formula is as follows:

[0043] ,

[0044] Where W and B represent the white calibration image and the black calibration image, respectively;

[0045] (3) Calculate the standard deviation of the spectral reflectance value of each pixel in the corrected hyperspectral image R1 of the fruit and create a feature matrix; then set an appropriate threshold to binarize the feature matrix, and use morphological operations such as opening and closing to process the binarized image and detect the boundary of interest; finally, extract the region of interest (ROI), remove the noise background, and obtain the preprocessed hyperspectral image R2 of the fruit.

[0046] Step 3: Obtaining the results of surface defects and internal quality index values ​​of the fruit;

[0047] The hyperspectral image to be detected is input into a pre-set detection model to obtain the surface defect results and internal quality index values ​​of the fruit. Hyperspectral imaging technology, as an image and spectrum fusion technology, has potential for wide-ranging applications in detection. However, fruit quality detection usually relies on the single detection of external defects or internal quality, and the problem of simultaneously detecting the internal and external quality of fruit has not been thoroughly studied. Therefore, this invention extracts the spectral and texture information of the hyperspectral image of the fruit based on the external and internal quality indicators using feature extraction technology, and fuses the spectral and texture information as input to a pre-set detection model to achieve simultaneous detection of internal defects and external quality of the fruit. In addition, to reduce the cost of fruit quality detection and improve efficiency, this invention uses a least-squares support vector machine to construct the detection model.

[0048] (1) Selection of external and internal quality indicators:

[0049] Based on the surface features of the preprocessed hyperspectral images of the fruit, the fruit samples can be divided into two categories: whole fruit and defective fruit. Whole fruit refers to fruit with no surface defects.

[0050] To obtain internal quality indicators for different fruits, four indicators were selected to characterize the internal quality of the fruits: firmness, soluble solids, titratable acidity, and vitamin C. The firmness of different types of fruits was measured using a hardness tester. Different fruits weighing 5.00 grams (including peel and pulp) were ground, and the juice was filtered. The filtered juice was then measured using a saccharimeter. The titratable acidity A can be obtained using the following formula:

[0051] ;

[0052] Where v (ml) and m (mol / L) are the dosage of sodium hydroxide solution, l is the conversion coefficient, a (g) represents the weight of different fruits, b (mL) is the total volume of fruit fluid in milliliters, and d (mL) represents the volume of sample solution used for titration in milliliters;

[0053] Vitamin C can be obtained using the following mathematical formula:

[0054] ;

[0055] Where h (mL) and h1 (mL) represent the number of milliliters of dye used in the titration sample and the number of milliliters of dye used in the blank titration, respectively; e (mg) is 1 mL of dye solution; f (mL) represents the number of milliliters of sample solution absorbed during the titration; q (mL) is the total volume of the diluted sample solution; and p (g) is the weight of the fruit.

[0056] Different fruit quality grading rules are set based on the surface defect results and internal quality index values ​​of different fruits.

[0057] (2) Construction of fruit quality testing:

[0058] Please see Figure 2 The fruit quality detection model in this embodiment includes a normalization layer, a dimensionality reduction layer, a spatial spectral feature extraction layer, and a classification, regression, and convergence layer. The normalization layer is used to normalize the preprocessed hyperspectral image R2 of the fruit. The dimensionality reduction layer is used to perform principal component analysis on the normalized hyperspectral image. Dimensionality reduction is performed to remove redundant band information, resulting in a dimensionality-reduced hyperspectral image of the fruit, R3. A spatial-spectral feature extraction layer is used to extract features from the dimensionality-reduced hyperspectral image R3 using a normalized attention network, obtaining rich spatial and spectral characteristics, which are then fused with six texture features: mean, contrast, correlation, energy, homogeneity, and entropy. A classification, regression, and convergence layer is used to input the fused values ​​of spatial-spectral and texture features into a least squares support vector machine (LSSVM) for classification, regression, and convergence. Based on surface and internal quality indicators of the fruit, and combining spectral and spatial information, a fruit detection model is constructed using least squares support vector machine to distinguish whether there are defects on the fruit surface and predict the values ​​of internal quality indicators.

[0059] In one implementation, the normalized attention network includes an encoding module, a residual module, a decoding module, a fusion module, and a normalized attention module;

[0060] The residual module is used to connect the input hyperspectral image X with the output feature map F3 of the convolution module C3 to ensure the transmission of information flow in the encoding module; the input hyperspectral image X is the dimension-reduced fruit hyperspectral image R3.

[0061] The decoding module includes three deconvolution modules D1, D2, and D3 arranged in series; each convolution module contains a deconvolution layer, and a normalization layer and an activation layer are added sequentially after each convolution layer; the kernel size of convolution modules C1, C2, and C3 is 3, and the stride size is 2.

[0062] The fusion module is used to pass the feature maps F1, F2 and F3 output by the C1, C2, C3 and C4 convolution modules through a deconvolution layer so that the features are at the same scale as the deconvolution modules D1, D2 and D3 respectively, and to fuse the image spatial and spectral features in the order of D1, D2 and D3 respectively.

[0063] The normalization attention module refines the feature map fused from the deconvolution module D3 and the convolution module C1. It uses a scaling factor based on batch normalization (BN) along the channel direction. This scaling factor measures the variance of each channel and indicates its importance. Its mathematical representation is as follows:

[0064]

[0065] in, The input feature map F is the value after normalization. To use the results of batch normalization in the channel direction, and Small batches The mean and standard deviation, and Let Y and Y represent the input and transformation factors, respectively, which are trainable transformation parameters. The resulting output feature Y can be expressed as:

[0066]

[0067] in, It is the scaling factor in the channel direction. This represents the weight obtained.

[0068] The normalized attention module is followed by an unfolded layer and a fully connected layer with a softmax function to classify the refined feature maps;

[0069] The encoding module, consisting of four convolutional modules and one residual module, is used to compress the features of the dimensionality-reduced hyperspectral image of the fruit, R3. The input to convolutional module C1 is the dimensionality-reduced hyperspectral image of the fruit, R3. The input to convolutional module C2 is the output feature map F1 of convolutional module C1. The input to convolutional module C3 is the output feature map F2 of convolutional module C2. The input to convolutional module C4 is the result of a residual concatenation between the output feature map F3 of convolutional module C3 and the input. Then, the deconvolution operation begins. The decoding module, consisting of three deconvolutional modules, three fusion modules, and one normalized attention module, is used to decode the output feature F5 of the encoding module. The input to the deconvolutional module D1 of the decoding module is the dimensionality-reduced hyperspectral image of the fruit, R3. The input to the deconvolution module D2 of the decoding module is the feature map resulting from the fusion of the output feature map F6 of the deconvolution module D1 and the output feature map F3 of the convolution module C3; the input to the deconvolution module D3 of the decoding module is the feature map resulting from the fusion of the output feature map F7 of the deconvolution module D2 and the output feature map F2 of the convolution module C2; ​​the input to the normalization attention module of the decoding module is the feature map resulting from the fusion of the output feature map F8 of the deconvolution module D3 and the output feature map F1 of the convolution module C1; the normalization attention module of the decoding module, after the input feature map passes through this unit, each feature map will have a weight map, representing the importance of the feature map in the channel direction; the output of the decoding module is the fruit quality detection result Y input to the encoding module.

[0070] In one embodiment, the encoding module includes four convolutional modules C1, C2, C3, and C4 arranged in series. Each convolutional module contains one convolutional layer, and a normalization layer and an activation layer are sequentially added after each convolutional layer. The kernel size of convolutional modules C1, C2, and C3 is 3, and the stride size is 2. The kernel size of convolutional module C4 is 3, and the stride size is 1.

[0071] In one implementation, the preprocessed hyperspectral image R2 of the fruit is normalized to shrink the data range to the (0, 1) interval, thereby accelerating the subsequent principal component analysis processing. The normalization formula is shown below:

[0072] ,

[0073] in, For the hyperspectral image after normalization, R mean R² represents the pixel average, and σ represents the standard deviation.

[0074] In one implementation, principal component analysis is used to process the normalized hyperspectral image. Dimensionality reduction is performed to remove redundant band information, resulting in a dimension-reduced hyperspectral image of the fruit, R3. This dimension-reduced hyperspectral image R3 is then used as spectral information.

[0075] In one implementation, six texture feature indices—mean, contrast, correlation, energy, homogeneity, and entropy—are obtained from the dimension-reduced hyperspectral image R3 of the fruit and used as texture features of the hyperspectral image of the fruit.

[0076] In one implementation, a least squares support vector machine (LSSVM) is used for classification, regression, and convergence. Based on fruit surface and internal quality indicators, and combined with spectral and texture information, a fruit detection model is constructed using LSSVM to distinguish whether there are defects on the fruit surface and predict the values ​​of internal quality indicators. LSSVM replaces the inequality constraints of slack variables in traditional support vector machines with equality constraints, thus allowing for a more efficient solution by solving a system of linear equations. Its discriminant function is shown below:

[0077] ,

[0078] in, x For the input vector, x i express x The target value is denoted by sgn(), which represents the step function, and ∑ is the summation operation. n To input the number of samples, a i For supporting values, For deviation, k ( x i ,x ) represents the radial basis function (RBF), with the following values:

[0079] ,

[0080] Where, σ 2 Here are the parameters for the RBF kernel function, and exp() represents an exponential function with the natural number e as the base.

[0081] (3) Input the preprocessed hyperspectral image R2 of the fruit into the preset detection model to obtain the surface defect results and internal quality index values ​​of the fruit.

[0082] Step 4: Obtaining Fruit Quality Grade Results: According to the preset fruit quality grading rules, obtain the fruit quality grade results and update the preset detection model.

[0083] (1) Input the pre-processed hyperspectral image of the fruit into the detection model to obtain the surface defect results, and the values ​​of internal quality indicators such as hardness, soluble solids, titratable acidity, and vitamin C.

[0084] (2) Based on the surface defects of the fruit, the internal quality index values ​​and the type of fruit, the quality grade of the fruit is given in accordance with the preset rules.

[0085] (3) Upload the fruit quality grade results to the terminal through wireless communication equipment (such as NB-IoT), and update the preset detection model using the fruit quality grade results predicted by the server.

[0086] Step 5: The terminal receives the fruit quality grade test results from the server, and the server feeds back the fruit quality grade results to the terminal.

[0087] Please see Figure 3 The present invention provides a fruit quality detection system based on a fruit quality detection model, comprising the following modules:

[0088] Terminal: mobile phone or computer, responsible for sending instructions to the wireless sensing module and receiving feedback information;

[0089] Wireless sensing module: Wireless communication device (such as NB-IoT), responsible for the transmission of information and instructions between the terminal and the processor;

[0090] Processor: Handles the execution instructions sent and received by the wireless sensors;

[0091] Fruit quality inspection module: Extracts spectral and texture information from the input hyperspectral image of fruit and outputs the fruit quality grade inspection results;

[0092] The hyperspectral image acquisition module mainly includes an imaging lens (HSIA-OL23), a hyperspectral imager (GaiaField-V10E), a light source (200W), a whiteboard (150×150), a mobile platform, and a server loaded with hyperspectral data acquisition software (SpecView). The acquisition wavelength range is 420~1000nm. It is responsible for acquiring hyperspectral images of fruits and performing black-and-white correction and region of interest extraction preprocessing operations.

[0093] The following uses apple quality inspection as an example to further illustrate the present invention. The method in this embodiment includes the following steps:

[0094] (1) Provide the server with raw hyperspectral images of apples through a hyperspectral image acquisition device, with a wavelength range of 420~1000 nm;

[0095] (2) Perform black and white correction and region of interest extraction preprocessing on the acquired original hyperspectral image of apple to obtain the preprocessed hyperspectral image of apple;

[0096] (3) Based on the surface characteristics of apples, apple samples can be divided into two categories: whole apples and defective apples. Among them, surface defects are divided into three types according to the degree of damage: minor defects, general defects, and severe defects. A whole apple refers to a fruit without surface defects. Four indicators, namely hardness, soluble solids, titratable acidity, and vitamin C, are selected to characterize the internal quality of the fruit.

[0097] (4) Normalize the preprocessed apple hyperspectral image and use principal component analysis to reduce the dimension of the normalized apple hyperspectral image. Then sort the principal components by contribution rate and select the top 10 principal components as the spectral information of the apple.

[0098] (5) Obtain the mean, contrast, correlation, energy, homogeneity and entropy from the dimension-reduced hyperspectral image, which respectively represent the average gray value in the window, the gray difference in the region, the linear correlation of gray, the uniformity of gray distribution, the local variation of gray and the disorder of gray distribution, as the texture information of the apple.

[0099] (6) Based on the surface and internal quality indicators of the fruit, and combined with spatial spectral and texture information, a fruit detection model is constructed using least squares support vector machine to distinguish whether the fruit surface has defects and predict the values ​​of internal quality indicators. To analyze the effectiveness of the proposed model, two discrimination criteria are introduced: the coefficient of certainty (R²) of the prediction. 2 ) and the root mean square error of the prediction (RMSEP).

[0100] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A fruit quality detection method based on a fruit quality detection model, characterized in that, Includes the following steps: Step 1: Obtain the original hyperspectral image R of the fruit to be detected and perform preprocessing; Step 2: Input the preprocessed hyperspectral image R2 into the fruit quality detection model to obtain the results of fruit surface defects and internal quality index values; The fruit quality detection model includes a normalization layer, a dimensionality reduction layer, a spatial spectral feature extraction layer, and a classification, regression, and convergence layer. The normalization layer is used to normalize the R² value of the preprocessed hyperspectral image of the fruit. The dimensionality reduction layer is used to perform principal component analysis on the normalized hyperspectral image. Dimensionality reduction is performed to remove redundant band information, resulting in a dimensionality-reduced hyperspectral image of the fruit, R3. The spatial-spectral feature extraction layer uses a normalized attention network to extract features from the dimensionality-reduced hyperspectral image R3, obtaining rich spatial and spectral characteristics, which are then fused with six texture features: mean, contrast, correlation, energy, homogeneity, and entropy. The classification, regression, and convergence layer uses the fused values ​​of spatial-spectral and texture features as input to a least-squares support vector machine for classification, regression, and convergence. Based on surface and internal quality indicators, and combining spectral and spatial information, it distinguishes whether there are defects on the fruit surface and predicts the values ​​of internal quality indicators. The normalized attention network includes an encoding module, a residual module, a decoding module, a fusion module, and a normalized attention module; The residual module is used to connect the input hyperspectral image X with the output feature map F3 of the convolution module C3 to ensure the transmission of information flow of the encoding module; the input hyperspectral image X is the dimension-reduced fruit hyperspectral image R3. The decoding module includes three deconvolution modules D1, D2 and D3 arranged in series; each convolution module contains a deconvolution layer, and a normalization layer and an activation layer are added sequentially after the convolution layer; the kernel size of convolution modules C1, C2 and C3 is 3, and the stride size is 2. The fusion module is used to pass the feature maps F1, F2 and F3 output by the convolution modules C1, C2, C3 and C4 through a deconvolution layer so that the features are at the same scale as the deconvolution modules D1, D2 and D3 respectively, and to fuse the image spatial and spectral features in the order of D1, D2 and D3 respectively. The normalized attention module is used to refine the feature map after the deconvolution module D3 and the convolution module C1 are fused, and a batch normalization scaling factor is used in the channel direction; the normalized attention module is followed by an unfolding layer and a fully connected layer with a softmax function to classify the refined feature map. The encoding module, consisting of four convolutional modules and one residual module, is used to compress the features of the dimensionality-reduced hyperspectral image R3 of the fruit. The input to convolutional module C1 is the dimensionality-reduced hyperspectral image R3 of the fruit. The input to convolutional module C2 is the output feature map F1 of convolutional module C1. The input to convolutional module C3 is the output feature map F2 of convolutional module C2. The input to convolutional module C4 is the result of a residual connection between the output feature map F3 of convolutional module C3 and the input. Then, a deconvolution operation is performed. The decoding module, consisting of three deconvolutional modules, three fusion modules, and one normalized attention module, is used to decode the output feature F5 of the encoding module. The input to the deconvolutional module D1 of the decoding module is the feature map F1 of the convolutional module C1. The output feature map F5 of block C4; the input of the deconvolution module D2 of the decoding module is the feature map fused from the output feature map F6 of the deconvolution module D1 and the output feature map F3 of the convolution module C3; the input of the deconvolution module D3 of the decoding module is the feature map fused from the output feature map F7 of the deconvolution module D2 and the output feature map F2 of the convolution module C2; ​​the input of the normalization attention module of the decoding module is the feature map fused from the output feature map F8 of the deconvolution module D3 and the output feature map F1 of the convolution module C1; the normalization attention module of the decoding module, after the input feature map passes through the normalization attention module, each feature map will have a weight map, representing the importance of the channel direction feature map; the output of the decoding module is the fruit quality detection result Y input to the encoding module.

2. The fruit quality detection method based on a fruit quality detection model according to claim 1, characterized in that: The preprocessing described in step 1 involves performing black-and-white correction on the original hyperspectral image R of the fruit to obtain a corrected hyperspectral image of the fruit. Where W and B represent the white calibration image and the black calibration image, respectively; The standard deviation of the spectral reflectance value of each pixel in the corrected hyperspectral image of the fruit is calculated to create a feature matrix. Then, a threshold is set to binarize the feature matrix, and the binarized image is processed by opening and closing operations to detect the boundary of interest. Finally, the region of interest is extracted and the noise background is removed to obtain the preprocessed hyperspectral image of the fruit, R2.

3. The fruit quality detection method based on the fruit quality detection model according to claim 1, characterized in that: In step 2, based on the surface features of the preprocessed hyperspectral image R2 of the fruit, the fruit samples are divided into two categories: intact fruit and defective fruit. Intact fruit refers to fruit with no surface defects. The internal quality index values ​​of different fruits are obtained. The internal quality index includes four indicators: firmness, soluble solids, titratable acidity, and vitamin C. Different fruit quality grading rules are set based on the surface defects and internal quality index values ​​of different fruits.

4. The fruit quality detection method based on a fruit quality detection model according to any one of claims 1-3, characterized in that: The fruit quality detection model is a pre-trained fruit quality detection model. After training, the training results are verified using a validation set. The verification uses two criteria: the coefficient of determination of the prediction and the root mean square error of the prediction. The fruit quality detection model uses cross-entropy as the loss function during training and employs the Adam optimizer to optimize the model training process, resulting in a well-trained fruit quality detection model. The encoding module of the well-trained fruit quality detection model is used to extract the spatial-spectral features of the input fruit hyperspectral image and, combined with texture features, feeds them into a least-squares support vector machine for classification.

5. A fruit quality detection system based on a fruit quality detection model, characterized in that: Includes the following modules: The fruit hyperspectral image module is used to acquire the original hyperspectral image R of the fruit to be detected and to perform preprocessing. The fruit quality detection module is used to input the preprocessed hyperspectral image R2 into the fruit quality detection model to obtain the results of fruit surface defects and internal quality index values. The fruit quality detection model includes a normalization layer, a dimensionality reduction layer, a spatial spectral feature extraction layer, and a classification, regression, and convergence layer. The normalization layer is used to normalize the R² value of the preprocessed hyperspectral image of the fruit. The dimensionality reduction layer is used to perform principal component analysis on the normalized hyperspectral image. Dimensionality reduction is performed to remove redundant band information, resulting in a dimensionality-reduced hyperspectral image of the fruit, R3. The spatial-spectral feature extraction layer uses a normalized attention network to extract features from the dimensionality-reduced hyperspectral image R3, obtaining rich spatial and spectral characteristics, which are then fused with six texture features: mean, contrast, correlation, energy, homogeneity, and entropy. The classification, regression, and convergence layer uses the fused values ​​of spatial-spectral and texture features as input to a least-squares support vector machine for classification, regression, and convergence. Based on surface and internal quality indicators, and combining spectral and spatial information, it distinguishes whether there are defects on the fruit surface and predicts the values ​​of internal quality indicators. The normalized attention network includes an encoding module, a residual module, a decoding module, a fusion module, and a normalized attention module; The residual module is used to connect the input hyperspectral image X with the output feature map F3 of the convolution module C3 to ensure the transmission of information flow of the encoding module; the input hyperspectral image X is the dimension-reduced fruit hyperspectral image R3. The decoding module includes three deconvolution modules D1, D2 and D3 arranged in series; each convolution module contains a deconvolution layer, and a normalization layer and an activation layer are added sequentially after the convolution layer; the kernel size of convolution modules C1, C2 and C3 is 3, and the stride size is 2. The fusion module is used to pass the feature maps F1, F2 and F3 output by the convolution modules C1, C2, C3 and C4 through a deconvolution layer so that the features are at the same scale as the deconvolution modules D1, D2 and D3 respectively, and to fuse the image spatial and spectral features in the order of D1, D2 and D3 respectively. The normalized attention module is used to refine the feature map after the deconvolution module D3 and the convolution module C1 are fused, and a batch normalization scaling factor is used in the channel direction; the normalized attention module is followed by an unfolding layer and a fully connected layer with a softmax function to classify the refined feature map. The encoding module, consisting of four convolutional modules and one residual module, is used to compress the features of the dimensionality-reduced hyperspectral image R3 of the fruit. The input to convolutional module C1 is the dimensionality-reduced hyperspectral image R3 of the fruit. The input to convolutional module C2 is the output feature map F1 of convolutional module C1. The input to convolutional module C3 is the output feature map F2 of convolutional module C2. The input to convolutional module C4 is the result of a residual connection between the output feature map F3 of convolutional module C3 and the input. Then, a deconvolution operation is performed. The decoding module, consisting of three deconvolutional modules, three fusion modules, and one normalized attention module, is used to decode the output feature F5 of the encoding module. The input to the deconvolutional module D1 of the decoding module is the feature map F1 of the convolutional module C1. The output feature map F5 of block C4; the input of the deconvolution module D2 of the decoding module is the feature map fused from the output feature map F6 of the deconvolution module D1 and the output feature map F3 of the convolution module C3; the input of the deconvolution module D3 of the decoding module is the feature map fused from the output feature map F7 of the deconvolution module D2 and the output feature map F2 of the convolution module C2; ​​the input of the normalization attention module of the decoding module is the feature map fused from the output feature map F8 of the deconvolution module D3 and the output feature map F1 of the convolution module C1; the normalization attention module of the decoding module, after the input feature map passes through the normalization attention module, each feature map will have a weight map, representing the importance of the channel direction feature map; the output of the decoding module is the fruit quality detection result Y input to the encoding module.

6. The fruit quality detection system based on the fruit quality detection model according to claim 5, characterized in that: The fruit hyperspectral imaging module includes an imaging lens, a hyperspectral imager, a light source, a whiteboard, a mobile platform, and a server.

7. The fruit quality detection system based on the fruit quality detection model according to claim 5, characterized in that: The system also includes a terminal, a wireless sensing module, and a processor; The terminal is used to send instructions to the wireless sensing module and receive feedback information; the wireless sensing module is used for the exchange of information and instructions between the terminal and the processor; the processor is used to process the execution instructions exchanged between the wireless sensors.

Images