Tomato shelf life prediction method based on hyperspectrum and RGB imaging technology

Abstract

The invention provides a tomato shelf life prediction method based on a hyperspectral and RGB imaging technology. The tomato shelf life prediction method comprises the following steps: S1, obtaining a hyperspectral image and an RGB image of a tomato sample; s2, preprocessing the hyperspectral image to obtain hyperspectral feature data; s3, extracting color feature data and texture feature data of the RGB image; s4, splicing the data obtained in the steps S2 and S3, performing standardization processing, and dividing the data into a training set, a test set and a verification set; s5, constructing an MLP model; screening the target feature set; s6, training the MLP model to obtain a tomato shelf life identification model; s6, obtaining a hyperspectral image and an RGB image of the detected tomato, performing preprocessing, and screening data according to the target feature set; and S7, splicing the screened color feature data, texture feature data and hyperspectral feature data of the detected tomatoes, inputting the spliced data into the tomato shelf life recognition model for recognition, and outputting the shelf life of the detected tomatoes. According to the invention, the accuracy of tomato shelf life identification is improved.

Description

Technical Field

[0001] This invention relates to a method for predicting shelf life, specifically a method for predicting the shelf life of tomatoes based on hyperspectral and RGB imaging technology. Background Technology

[0002] Tomatoes are one of my country's main fruits and vegetables, with a wide planting area and large yield. Accurate prediction of their post-harvest shelf life is a core aspect of agricultural product supply chain quality management, directly impacting logistics loss control, inventory management, and commodity value assurance. In recent years, hyperspectral imaging technology, with its advantage of non-destructively acquiring spatial image information and continuous band spectral information of the tested object simultaneously, has become an emerging core technology for quality inspection in the agricultural field. It has broken through the limitations of traditional testing technologies and has achieved widespread application and rapid development in areas such as the detection of physicochemical indicators of agricultural products, quality grading, and variety identification.

[0003] Hyperspectral imaging technology integrates knowledge from multiple disciplines such as optics, electronics, computer science, and spectroscopy. It uses a hyperspectral camera to acquire continuous spectral data of the object under test in the ultraviolet, visible, near-infrared, and short-wave infrared bands. Each pixel corresponds to a complete spectral curve, achieving a "picture-spectrum integration" detection effect. By analyzing the correlation between spectral characteristics and the chemical composition of substances, it enables non-destructive detection of the internal physicochemical indicators of agricultural products. Compared with traditional detection technologies, it has significant advantages such as non-destructiveness, speed, quantification, and integration, becoming an important technical means for modern agricultural testing.

[0004] In practical applications in agriculture, hyperspectral imaging technology has been successfully implemented in various quality testing scenarios for agricultural products and cash crops. In fruit and vegetable testing, it can accurately measure core physicochemical indicators such as soluble solids (SSC), fruit firmness, titratable acid content, and moisture content in fruits and vegetables like tomatoes, apples, and citrus, enabling maturity grading and quality assessment. In cash crops and traditional Chinese medicinal herbs, it can achieve origin traceability, variety classification, and authenticity identification through spectral feature difference analysis. It can also complete quality grading and component analysis for crops such as tea and tobacco. Furthermore, this technology can be applied to crop growth monitoring, early identification of pests and diseases, and quality control in agricultural product processing, providing crucial technical support for the refinement and intelligentization of agricultural production. However, in the field of tomato testing, hyperspectral technology currently focuses primarily on the quantitative detection of single physicochemical indicators, such as accurately measuring SSC and firmness of tomatoes by extracting characteristic spectral values ​​and combining them with machine learning models. It has not yet achieved a technological extension from physicochemical indicator detection to overall shelf-life prediction.

[0005] Existing technologies for predicting the shelf life of tomatoes have developed into specialized instrumentation methods such as high-performance liquid chromatography (HPLC) and electrochemical sensing. HPLC precisely analyzes post-harvest changes in the content of organic acids and sugars in tomatoes, reflecting the internal quality deterioration patterns of the fruit. Electrochemical sensing detects indicators such as cellular water loss and electrolyte permeation within fruits and vegetables, linking them to the freshness and shelf life of tomatoes. Compared to traditional physicochemical testing, these methods offer higher accuracy and greater quantification. However, they are expensive, require highly specialized operation, necessitate sample pretreatment, cannot achieve in-situ, non-destructive testing, and are time-consuming for single-sample analysis. Therefore, they are only suitable for precise laboratory analysis and are difficult to scale up for industrial applications. Summary of the Invention

[0006] The purpose of this invention is to provide a method for predicting the shelf life of tomatoes based on hyperspectral and RGB imaging technology, so as to solve the problems of long time consumption and high cost of existing technologies for detecting the shelf life of tomatoes.

[0007] The objective of this invention is achieved in this way:

[0008] A method for predicting the shelf life of tomatoes based on hyperspectral and RGB imaging technology includes the following steps:

[0009] S1. Select several tomato samples of the same variety as the tomato being tested, and obtain the hyperspectral image and RGB image of each tomato sample;

[0010] S2. Perform preprocessing on the hyperspectral image, including reflection removal, spectral value extraction and noise reduction, to obtain the hyperspectral feature data of the tomato sample;

[0011] S3. Perform preprocessing on the RGB image, including reflection removal and effective region segmentation. Extract color feature data and texture feature data from the preprocessed RGB image to obtain the color feature data and texture feature data of the tomato sample.

[0012] S4. The hyperspectral feature data of the tomato sample is concatenated with the extracted color feature data and texture feature data to obtain fused feature data; the fused feature data is standardized, and the standardized fused feature data is used as a sample, which is divided into training set, test set and validation set; each sample is the standardized fused feature data of one tomato sample.

[0013] S5. Construct an MLP model; use the training set and the MLP model to filter the target feature set;

[0014] S6. Based on the target feature set, the data of each sample in the training set, test set and validation set are filtered. The samples after data filtering in the training set, test set and validation set are input into the MLP model for training to obtain the tomato shelf life identification model.

[0015] S7. Acquire the hyperspectral image and RGB image of the tomato being tested, and perform the same processing as steps S2-S3 to obtain the hyperspectral feature data, color feature data, and texture feature data of the tomato being tested; extract the hyperspectral feature data, color feature data, and texture feature data of the tomato being tested according to the target feature set;

[0016] S8. The color feature data, texture feature data and hyperspectral feature data extracted from the tomato to be tested are spliced ​​together and input into the tomato shelf life recognition model. After the tomato shelf life recognition model recognizes the shelf life of the tomato to be tested, it outputs the shelf life of the tomato to be tested.

[0017] Furthermore, the color feature data includes the average values ​​of the three major color spaces: RGB, HSV, and LAB.

[0018] Furthermore, the texture features include energy, entropy, contrast, inverse difference moment, correlation, and homogeneity.

[0019] Furthermore, the MLP model includes an input layer, a first fully connected hidden layer, a second fully connected hidden layer, a third fully connected hidden layer, and an output layer, with cross-layer residual connections used between the input layer and the third fully connected hidden layer.

[0020] Furthermore, both the first and second fully connected hidden layers process the input data in the following way: the input data is processed by feature mapping through the fully connected layer, and then sequentially processed by batch normalization, activation function activation, and discard regularization before output.

[0021] Furthermore, the third fully connected hidden layer processes the input data as follows: the input data first passes through the fully connected layer for feature mapping, then passes through the batch normalization layer for normalization, and finally passes through the activation function for output.

[0022] Furthermore, the specific method for filtering the target feature set in step S5 is as follows:

[0023] S5a-1. Use UVE to perform feature optimization on the training set at least 50 times, and each optimization yields a corresponding subset of candidate features;

[0024] S5a-2. Input the data in the training set corresponding to each candidate feature subset into the MLP model, and calculate the target evaluation index corresponding to each candidate feature subset;

[0025] S5a-3. Compare the target evaluation metrics for each candidate feature subset to determine the target feature set.

[0026] Furthermore, the specific method for feature optimization of the training set in step S5a-1 is as follows:

[0027] S5a-1-1. Convert the training set into an original feature matrix; the column of the original feature matrix represents the number of samples in the training set, and the column represents the features of the samples.

[0028] S5a-1-2. Use the UVE algorithm to add noise to the original feature matrix to obtain the concatenated matrix, and set the stability percentile threshold;

[0029] S5a-1-3. Use PLRS for modeling, set the number of principal components, determine the contribution weight of each feature to the predicted value based on the number of principal components, and select a subset of candidate features based on the contribution weight of each feature to the predicted value.

[0030] S5a-1-4. The range of principal component number is 5-20, and the range of stability percentile threshold is 90-99.5. Based on the ranges corresponding to the principal component number and stability percentile threshold, repeat steps S5a-1-1-S5a-1-2 at least 50 times, and obtain the corresponding candidate feature subset each time.

[0031] Furthermore, the target evaluation index includes the average coefficient of determination and the root mean square error, and the features in the candidate feature subset with the largest average coefficient of determination and the smallest root mean square error are taken as the target feature set.

[0032] This invention preprocesses tomato samples of the same variety as the tested tomatoes to obtain hyperspectral data, color feature data, and texture feature data. It then uses UVE and MLP models to calculate the feature data, determining the target feature set and achieving adaptive feature extraction. Subsequent MLP model training reduces the amount of feature data processing required. Through MLP model training, a tomato shelf-life identification model is obtained, allowing the same tomato shelf-life to be used for the same variety, thus reducing the prediction time and cost of tomato shelf-life.

[0033] This invention innovatively utilizes a residual connection structure in MLP models that directly connects the input layer to the third hidden layer. This differs from the conventional design of residual connections, which often uses adjacent layer connections, breaking the inherent limitations of "layer-by-layer propagation and gradient reduction." This structure directly superimposes and fuses the hyperspectral and RGB fusion features of the input layer with the features processed by the three hidden layers. This preserves the core effective information in the original fusion features while fully integrating the abstract features extracted by the deep network, achieving bidirectional complementarity between the original and deep features. This cross-layer design effectively alleviates the gradient vanishing and gradient exploding problems that easily occur during deep MLP training, accelerates model training convergence, and enhances the model's ability to capture and learn the fusion information of hyperspectral physicochemical features and RGB appearance features. This allows the model to more accurately uncover the intrinsic relationship between fusion features and tomato shelf life, further improving prediction accuracy and model generalization ability, and adapting to the feature learning requirements of the specific task of tomato shelf life prediction. Attached Figure Description

[0034] Figure 1 This is a flowchart of the present invention.

[0035] Figure 2 This is the original spectral curve of a tomato.

[0036] Figure 3 This is the original spectral curve of the tomato after SNV preprocessing.

[0037] Figure 4 This is a schematic diagram of the MLP structure.

[0038] Figure 5 This is a schematic diagram of the MI-VIF feature screening bands.

[0039] Figure 6 These are the loss curves for the training and validation sets.

[0040] Figure 7 This is a test scatter plot. Detailed Implementation

[0041] The invention will now be described in further detail with reference to the accompanying drawings.

[0042] like Figure 1 As shown, the present invention includes the following steps:

[0043] S1. Select several tomato samples of the same type as the tomato being tested, and obtain the hyperspectral image and RGB image of each tomato sample.

[0044] This invention selected 78 tomato samples of the same variety as the tested tomatoes, which were free from mechanical damage, pests, and diseases, and had a regular shape. After removing surface impurities, the samples were placed at room temperature for 2 hours to ensure surface stability and avoid image acquisition deviations caused by temperature differences. Images of this batch of tomatoes were collected every afternoon at 2 PM, with a total of 1021 valid data points collected over a 25-day cycle. The firmness of the tomatoes was measured using a Pfizer GY-4 tester. When the firmness was soft to around 10N, the day of the photograph was recorded as the shelf life of the tomato at 0 days.

[0045] like Figure 2 As shown, a ColorSpectrum FS-23 hyperspectral camera was used in a darkroom environment with 50W halogen lamps at 12V on both sides as the light source. The camera lens was kept perpendicular to the tomato sample surface, and the acquisition distance was consistent. Hyperspectral images were acquired, obtaining spectral information in 300 bands. The hyperspectral camera used in this invention has a spectral range of 400nm-1000nm, a spectral resolution of 2.5nm, and a spatial combining number of 4.

[0046] When acquiring RGB images of tomato samples, the same darkroom and light source environment as the hyperspectral acquisition were used, along with the same acquisition angle and distance. This ensured a one-to-one correspondence between the hyperspectral and RGB images of the tomato samples.

[0047] Each hyperspectral and RGB image is labeled with its shelf life in days. For example, if a tomato is soft to 10N, the shelf life of the hyperspectral and RGB images taken on that day is 0 days. The shelf life of the hyperspectral and RGB images taken the previous day is 1 day, and so on. This way, the shelf life of the hyperspectral and RGB images taken each day for each tomato sample can be determined.

[0048] S2. Perform preprocessing on the hyperspectral image, including reflection removal, spectral value extraction and noise reduction, to obtain the hyperspectral feature data of the tomato sample.

[0049] like Figure 3As shown, after acquiring the hyperspectral image, it undergoes preprocessing. First, reflective areas are identified through band animation. This invention selects a grayscale image of a specific band of 567.11nm and uses a threshold segmentation method to identify and remove reflective areas from the tomato surface, eliminating interference from reflections on the spectral data and retaining the hyperspectral image of the effective tomato area. Then, for the hyperspectral image with reflective areas removed, the spectral values ​​of each pixel are extracted. In this invention, each pixel has spectral values ​​of 300 bands, i.e., the original hyperspectral feature data. That is, each pixel has spectral values ​​corresponding to several bands. The average value of the spectral values ​​for the same band is taken to obtain the hyperspectral feature data corresponding to the hyperspectral image. Finally, the Standard Normal Variation (SNV) method is used to process the original hyperspectral feature data, eliminating noise interference such as baseline drift and scattering effects, thus improving the quality of the spectral data.

[0050] S3. Perform preprocessing on the RGB image, including reflection removal and effective region segmentation. Extract color feature data and texture feature data from the preprocessed RGB image to obtain the RGB features of the tomato sample, namely color feature data and texture feature data.

[0051] This invention first preprocesses the original RGB image using a threshold segmentation method to identify and remove reflective areas on the tomato surface, eliminating reflective interference. Then, the de-reflected RGB image is converted to the HSV color space, and the effective pixel area of ​​the tomato is accurately segmented using the HSV method, removing invalid areas such as background and shadows to ensure targeted feature extraction. Finally, the mean values ​​of the RGB, HSV, and LAB color spaces are calculated for the segmented effective pixel area of ​​the tomato, obtaining 9-dimensional color feature data. This allows for the extraction of the tomato's core color feature data, reflecting changes in the tomato's appearance and color, providing an appearance-based basis for predicting shelf life.

[0052] Finally, six texture feature data were extracted from the RGB images of the tomato samples: Energy, Entropy, Contrast, Inverse Moment (IDM), Homogeneity, and Homogeneity.

[0053] Determine the proportion p(i,j) of two adjacent identical gray values ​​in an RGB image, and determine the energy based on this proportion.

[0054]

[0055] Where i and j both represent grayscale values, N g This represents the number of gray levels, typically 256.

[0056] Energy can represent the smoothness and uniformity of a tomato's skin. When the energy is high, there are few grayscale variations in the image, the pixels look similar, and the texture is simple, neat, and smooth. When the energy is low, the grayscale in the image is messy and jumps around everywhere, and the texture is complex, chaotic, and rough.

[0057]

[0058] Entropy represents the "disorder", "complexity" and "irregularity" of image texture. When entropy is low, the image grayscale is uniform, with few changes and a simple structure, and is not messy and is very smooth. When entropy is high, the image grayscale fluctuates, has many changes, and has a fragmented structure, and is particularly messy, complex and coarse.

[0059]

[0060] Contrast represents the degree of difference, depth, and texture of gray levels in an image. When the contrast is low, the brightness and darkness of pixels are roughly the same, the transition is smooth, and it looks flat, soft, and unobtrusive. When the contrast is high, some areas are particularly bright and some are particularly dark, the contrast between light and dark is strong, and it looks distinct, with a strong sense of depth and texture.

[0061]

[0062] Homogeneity can measure the smoothness, uniformity, and delicacy of image texture. When homogeneity is high, the gray levels between pixels are very similar, the transitions are smooth, and the surface is smooth, uniform, and delicate. When homogeneity is low, the differences in pixel brightness are large and the changes are drastic, and the surface is rough, messy, and uneven.

[0063]

[0064] Correlation measures the regularity and linearity of pixel grayscale changes in an image, essentially indicating the neatness and orderliness of the texture arrangement. High correlation indicates regular, orderly, and stable pixel changes, rather than a chaotic pattern; low correlation indicates disordered, irregular, and loosely structured pixel brightness variations.

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] Where, μ x μ yσ x and σ y It is a homogeneous characteristic.

[0071] Thus, we can obtain the RGB mean, HSV mean, LAB mean, and 6 texture features of each RGB image, totaling 15 dimensions of feature data.

[0072] S4. The hyperspectral feature data of the tomato sample is concatenated with the extracted color feature data and texture feature data to obtain fused feature data; the fused feature data is standardized, and the standardized fused feature data is used as samples, and the samples are divided into training set, test set and validation set.

[0073] One sample is the standardized fusion feature data of a tomato sample.

[0074] The 300-dimensional hyperspectral feature data, preprocessed from hyperspectral images of the same tomato sample taken on the same day, is stitched together with the color and texture feature data extracted from the RGB images. Dimension alignment is performed in the order of hyperspectral features first, followed by RGB features. Feature dimension verification is then performed to ensure no sample misalignment or feature loss, ultimately forming fused feature data.

[0075] This invention uses only z-score standardization (StandardScaler) to process the fused feature data, transforming the data into a distribution with a mean of 0 and a variance of 1. This effectively eliminates the differences in the dimensions and ranges of different features, avoids bias in model training, and improves the adaptability between data and model.

[0076] The standardized fused feature data is used as samples and divided into training, validation, and test sets. This invention uses the training set for feature selection. In this invention, a sample includes 315-dimensional feature data.

[0077] S5. Construct an MLP model; based on the training set and the MLP model, filter the target feature set.

[0078] like Figure 4 As shown, the MLP model consists of an input layer, three fully connected hidden layers, and an output layer. A cross-layer residual connection is used between the input layer and the third fully connected hidden layer. The three fully connected hidden layers consist of a first fully connected hidden layer, a second fully connected hidden layer, and a third fully connected hidden layer.

[0079] The input layer receives the standardized fused feature data, which is then fed into the first fully connected hidden layer. The first fully connected hidden layer outputs 128-dimensional data, which is then fed into the second fully connected hidden layer. The second fully connected hidden layer outputs 64-dimensional data, which is then fed into the third fully connected hidden layer. The third fully connected hidden layer outputs 32-dimensional data. The data output from the third fully connected hidden layer is then residually concatenated with the input data of the MLP model to obtain 32-dimensional data, which is finally fed into the fully connected layer. The output layer outputs a one-dimensional predicted shelf life value for tomatoes.

[0080] The first and second fully connected hidden layers process the input data as follows: the input data passes through the fully connected layer, then undergoes batch normalization, activation function activation, and dropout regularization in sequence. The input data in the third fully connected layer passes through the fully connected layer, then undergoes batch normalization and activation function activation in sequence, and is finally output.

[0081] S5a-1. Use UVE to perform feature optimization at least 50 times after the data has been filtered and processed, and obtain the corresponding candidate feature subset in each optimization.

[0082] like Figure 5 As shown, this invention uses Unified Noise Filtering (UVE) for feature selection. The core idea of ​​UVE is to introduce artificial noise as a benchmark. It artificially adds a randomly generated noise matrix to the original feature matrix, and then uses the Partial Least Squares Regression (PLSR) algorithm to model the mixed data. By comparing the stability of the original features and the artificial noise features in the model, those original features whose stability is even worse than pure noise are judged as "uninformative variables" and removed. Figure 5 The horizontal axis represents the bands of the hyperspectral feature data, the vertical axis represents the data after SNV processing of the original hyperspectral feature data, and the points represent the features selected by UVE.

[0083] S5a-1-1. Convert the training set into the original feature matrix.

[0084] The original feature matrix is ​​defined as the number of samples in the training set, listed as a feature of the sample. The original feature matrix is ​​generated by converting the n standardized fused feature data from the training set into a single original feature matrix. The original feature matrix is ​​defined as the number of data points in each row, listed as a feature of the data. Each row in this invention contains 315 feature data points for a tomato sample, including 300 wavelengths (i.e., 300-dimensional hyperspectral feature data), 9-dimensional color feature data, and 6-dimensional texture feature data.

[0085] S5a-1-2. Use the UVE algorithm to add noise to the original feature matrix to obtain the spliced ​​matrix, and set the stability percentile threshold.

[0086] Artificially adding noise involves adding noise of the same dimension to the original feature matrix, which then forms the concatenated matrix of the UVE. This is equivalent to adding 315 dimensions of irrelevant noise after the original 315-dimensional matrix. This matrix is ​​n. 630.

[0087] The core control variables of the UVE algorithm are the principal component count and the stability percentile threshold. The search interval for the principal component count is set to [5, 20], and the search interval for the stability percentile threshold is set to [90, 99.5]. The search intervals for the principal component count and the stability percentile threshold are jointly defined as a two-dimensional dynamic search space S.

[0088] S5a-1-3. Use PLRS for modeling, set the number of principal components, determine the contribution weight of each feature to the predicted value based on the number of principal components, and select a subset of candidate features based on the contribution weight of each feature to the predicted value.

[0089] This invention uses a PLSR model for modeling, with the number of principal components set to t. For example, the number of principal components can be set to 5. Then, the 630-dimensional fused feature data is divided into 5-dimensional comprehensive variables: t1, t2, t3, t4, and t5.

[0090] t1=w 11 x1+w 12 x2+…+w 1630 x 630

[0091] t2=w 21 x1+w 22 x2+…+w 2630 x 630

[0092] t3=w 31 x1+…+w 3630 x 630

[0093] t4=w 41 x1+…+w 4630 x 630

[0094] t5=w 51 x1+…+w 5630 x 630

[0095] Here, x is a feature, and during calculation, the feature values ​​corresponding to x in the training set are calculated, and w is a coefficient.

[0096] y = c1t1 + c2t2 + c3t3 + c3t3 + c5t5

[0097] Where c is the coefficient and y is the predicted value.

[0098] Secondly, back substitution to combine like terms:

[0099] y=c1×(w 11 x1+w 12 x2+…+w 1630 x 630 )+c2×(w 21 x1+w 22 x2+…+w 2630 x 630 )+…+c5×(w 51 x1+…+w 5630 x 630 )

[0100] Then, multiply by and combine like terms.

[0101] y=(c1w 11 +c2w 21 +…+c5w 51 )x1+(c1w 12 +c2w 22 +…+c2w 52 )x+…+(c1w 1630 +…+c5w 5630 )x 630

[0102] available:

[0103] b1=c1w 11 +c2w 21 +…+c5w 51

[0104] b2=c1w 12 +c2w 22 +…+c5w 52

[0105] The final formula for PLSR can be obtained as follows:

[0106] y≈b1x1+b2x2+…+b 630 x 630

[0107] Where b is the contribution weight of the feature calculated by modeling to the predicted value.

[0108] And calculate b i Corresponding stability.

[0109] After multiple training iterations of PLSR, select the features with high values ​​for b (i.e., b must be greater than the noise x corresponding to the feature x). 噪声The features x corresponding to the contribution weights and b with little fluctuation (i.e., stability greater than the stability percentage threshold) are selected. These features x have relatively good stability and are the features selected in this optimization. The features x are candidate features, and the set of all selected features x is a subset of candidate features. The original features with even lower stability than pure noise (i.e., the remaining features) are judged as "uninformative variables" and removed.

[0110] This invention involves adding 315 dimensions of noise to a 315-dimensional original feature matrix, resulting in a 630-dimensional concatenated matrix. The 315-dimensional original feature matrix corresponds to the 315-dimensional noise matrix. During optimization, features are selected from the first 315 dimensions of the 630-dimensional matrix. In the concatenated matrix, the noise in the 316th dimension corresponds to the feature in the 1st dimension, the noise in the 317th dimension corresponds to the feature in the 2nd dimension, and so on. Each feature has a corresponding noise level. Finally, both the calculated features and the noise have corresponding contribution weights. Taking the third-dimensional feature x3 as an example... 318 For noise corresponding to feature x3, when the contribution weight b3 corresponding to x3 is greater than x 318 The corresponding contribution weight b 318 At that time, and b 318 When the stability is greater than the stability percentage threshold, x3 is the feature selected through optimization.

[0111] S5a-1-4. The range of principal component number is 5-20, and the range of stability percentile threshold is 90-99.5. Based on the ranges corresponding to the principal component number and stability percentile threshold, repeat steps S5a-1-S5a-2 at least 50 times, and obtain the corresponding candidate feature subset each time.

[0112] By performing random sampling (or grid search) within the two-dimensional dynamic search space S, the UVE algorithm is driven to adaptively remove redundant information, performing the same processing steps as S5a-1-2-S5a-1-3. During each optimization, the principal component count and stability percentile threshold are changed, and each round of optimization selects corresponding features to form a feature subset F. i At least 50 optimizations are performed. This invention performs 50 optimizations to generate 50 candidate feature subsets {F1, F2, ..., F...} with different feature dimensions and information densities. 50}

[0113] S5a-2. The data in the training set corresponding to each candidate feature subset is input into the MLP model to calculate the target evaluation index corresponding to each candidate feature subset.

[0114] Select each candidate feature subset F i The corresponding data in the training set, i.e., the candidate feature subset Fi The data corresponding to each candidate feature in the training set are sequentially input into the MLP model for prediction performance evaluation.

[0115] In each iteration of the optimization process, the training set is further divided into a training set and a validation set. A five-fold cross-validation (5-FoldCV) mechanism is used to rigorously validate the selection results under the current UVE parameter configuration. The average coefficient of determination (R²) of this feature subset on the validation set is then calculated. 2 ) and root mean square error (RMSE) are used as target evaluation indicators:

[0116]

[0117]

[0118] Among them, y i It is the true value corresponding to the i-th feature. R is the predicted value corresponding to the i-th feature. 2 The RMSE values ​​are the results of a 50-fold cross-validation.

[0119] The average coefficient of determination and root mean square error corresponding to each candidate feature subset can be obtained.

[0120] S5a-3. Compare the target evaluation metrics for each candidate feature subset to determine the target feature set.

[0121] Global Optimal Feature Locking: The system constructs an automatic recording and comparison mechanism. After completing N rounds of preset optimization experiments, it globally compares the evaluation metrics of all candidate feature subsets. Finally, it locks and outputs the feature that enables the MLP model to achieve the highest prediction accuracy (i.e., R0). 2 When the MLP model approaches maximum accuracy and RMSE approaches minimum accuracy, the features in the subset of candidate features that enable the MLP model to achieve the highest prediction accuracy are used as the target feature set to complete the adaptive extraction of feature-level fusion.

[0122] When performing feature screening on other tomatoes of the same variety as the tomato sample, no calculation is required; the same set of target features can be used directly.

[0123] The cross-layer residual connection design is specifically adapted to the fusion dimension of hyperspectral core features (27 dimensions) + RGB features (15 dimensions), avoiding information loss due to differences in feature dimensions. Training with added noise (mean 0, variance 0.2) specifically addresses the issue of tomato spectral data being susceptible to ambient light interference, improving the model's robustness to interference by 25% in practical applications. The AdamW optimizer is used with a learning rate of 0.001 and weight decay of 0.2; the HuberLoss loss function, delta=1.0, balances the advantages of mean squared error and mean absolute error, and is robust to outliers; the ReduceLROnPlateau learning rate scheduler is used to dynamically adjust the learning rate based on the validation set loss, improving training performance.

[0124] S6. Based on the target feature set, perform data filtering on the data of each sample in the training set, test set, and validation set; input the data-filtered samples from the training set, test set, and validation set into the MLP model for training to obtain the tomato shelf life identification model.

[0125] The data of the target feature set corresponding to each sample in the training set, test set and validation set are extracted, that is, data that does not belong to the target feature set are removed. The present invention reduces the 315-dimensional data of the sample to 124-dimensional data (i.e. 120-dimensional hyperspectral features and 4-dimensional RGB features). The extracted data is then input into the MLP model for training to obtain the tomato shelf life identification model.

[0126] For example, the target feature set includes feature a, feature c, and feature d, and the data in the sample includes feature a, feature b, feature c, feature d, feature e, and feature f. The data of the target feature set corresponding to the sample is extracted, and the extracted data is the data corresponding to each of feature a, feature c, and feature d in the sample.

[0127] When predicting the shelf life of other tomatoes of the same variety as the tomato sample, the tomato shelf life identification model can be used directly.

[0128] S7. Acquire the hyperspectral image and RGB image of the tomato being tested, and perform the same processing as steps S2-S3 to obtain the hyperspectral feature data, color feature data and texture feature data of the tomato being tested; extract the hyperspectral feature data of the tomato being tested according to the target feature set.

[0129] The variety of the tomato being tested is the same as that of the tomato sample.

[0130] Hyperspectral images of the tomatoes to be tested were acquired using a hyperspectral camera, and RGB images of the tomatoes were also acquired under the same conditions. The hyperspectral images of the tomatoes were preprocessed in step S2, including reflection removal, spectral value extraction, and noise reduction, to obtain the hyperspectral feature data of the tomatoes. Then, the RGB images of the tomatoes were preprocessed in step S3, including reflection removal and effective region segmentation. Finally, color feature data and texture feature data were extracted from the preprocessed RGB images of the tomatoes.

[0131] Extract the data corresponding to the target feature set from the hyperspectral feature data, color feature data, and texture feature data of the tested tomatoes.

[0132] S8. The color feature data, texture feature data and hyperspectral feature data extracted from the tomato to be tested are spliced ​​together and input into the tomato shelf life recognition model. After the tomato shelf life recognition model recognizes the shelf life of the tomato to be tested, it outputs the shelf life of the tomato to be tested.

[0133] The extracted hyperspectral feature data is spliced ​​together with the extracted color feature data and texture feature data. The splicing method of the tested tomato is the same as that of the tomato sample. The data is then input into the tomato shelf life identification model obtained in step S6 to obtain the shelf life of the tested tomato.

[0134] This fusion is not a simple, indiscriminate stacking, but a deep fusion based on high-quality core spectral features and full-dimensional appearance features. It retains the key information of hyperspectral data reflecting the internal physiological changes of tomatoes, and integrates external information of RGB data representing changes in skin color and texture. This comprehensive characterization of physical and chemical quality and appearance quality significantly improves the prediction accuracy and stability of subsequent models.

[0135] like Figure 6 As shown, the effectiveness of the present invention is verified through experiments. Figure 6 As can be seen, the model training process converged well, and no overfitting occurred. Train_Loss is the loss curve for the training set, and Val_Loss is the loss curve for the validation set.

[0136] Table 1. Evaluation Indicators for Tomato Shelf Life Prediction Using MLP

[0137]

[0138] like Figure 7 As shown in Table 1, the horizontal axis represents the correct values, and the vertical axis represents the predicted values. This invention also selects the coefficient of determination (R²), mean squared error (MSE), and root mean square error (RMSE) as core evaluation indicators to verify the actual performance of this invention in predicting the shelf life of tomatoes. Figure 6Table 1 shows that R² = 0.8114, MSE = 5.30, and RMSE = 2.30. The test results demonstrate that the MLP model designed in this invention possesses excellent predictive stability and generalization ability. The predicted values ​​output by the model are strongly correlated with the actual shelf life of tomatoes, accurately capturing the intrinsic relationship between fused features and tomato shelf life. The MSE and RMSE values ​​are within a low and reasonable range, indicating that the model has small prediction errors, stable and reliable output results, and excellent overall prediction accuracy. These results fully demonstrate that this invention, through deep fusion of hyperspectral and RGB images, targeted image preprocessing, and structurally optimized MLP model, effectively solves many problems of traditional detection techniques. The established model exhibits high stability, high accuracy, and strong robustness in predicting the shelf life of unknown tomato samples, meeting the practical application requirements for non-destructive detection of tomato shelf life.

Claims

1. A method for predicting the shelf life of tomatoes based on hyperspectral and RGB imaging technology, characterized in that, Includes the following steps: S1. Select several tomato samples of the same variety as the tomato being tested, and obtain the hyperspectral image and RGB image of each tomato sample; S2. Perform preprocessing on the hyperspectral image, including reflection removal, spectral value extraction and noise reduction, to obtain the hyperspectral feature data of the tomato sample; S3. Perform preprocessing on the RGB image, including reflection removal and effective region segmentation. Extract color feature data and texture feature data from the preprocessed RGB image to obtain the color feature data and texture feature data of the tomato sample. S4. The hyperspectral feature data of the tomato sample is concatenated with the extracted color feature data and texture feature data to obtain fused feature data; the fused feature data is standardized, and the standardized fused feature data is used as a sample, which is divided into training set, test set and validation set; each sample is the standardized fused feature data of one tomato sample. S5. Construct an MLP model; use the training set and the MLP model to filter the target feature set; S6. Based on the target feature set, the data of each sample in the training set, test set and validation set are filtered. The samples after data filtering in the training set, test set and validation set are input into the MLP model for training to obtain the tomato shelf life identification model. S7. Acquire the hyperspectral image and RGB image of the tomato being tested, and perform the same processing as steps S2-S3 to obtain the hyperspectral feature data, color feature data, and texture feature data of the tomato being tested; extract the hyperspectral feature data, color feature data, and texture feature data of the tomato being tested according to the target feature set; S8. The color feature data, texture feature data and hyperspectral feature data extracted from the tomato to be tested are spliced ​​together and input into the tomato shelf life recognition model. After the tomato shelf life recognition model recognizes the shelf life of the tomato to be tested, it outputs the shelf life of the tomato to be tested.

2. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 1, characterized in that, The color feature data includes the average values ​​of the three major color spaces: RGB, HSV, and LAB.

3. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 1, characterized in that, The texture features include energy, entropy, contrast, inverse difference moment, correlation, and homogeneity.

4. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 1, characterized in that, The MLP model includes an input layer, a first fully connected hidden layer, a second fully connected hidden layer, a third fully connected hidden layer, and an output layer, with cross-layer residual connections used between the input layer and the third fully connected hidden layer.

5. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 4, characterized in that, The first and second fully connected hidden layers both process the input data in the following way: the input data is processed by feature mapping through the fully connected layer, and then successively processed by batch normalization, activation function activation, and discard regularization before output.

6. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 4, characterized in that, The third fully connected hidden layer processes the input data as follows: the input data first passes through the fully connected layer for feature mapping, then through the batch normalization layer for normalization, and finally through the activation function for output.

7. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 1, characterized in that, The specific method for filtering the target feature set in step S5 is as follows: S5a-1. Use UVE to perform feature optimization on the training set at least 50 times, and each optimization yields a corresponding subset of candidate features; S5a-2. Input the data in the training set corresponding to each candidate feature subset into the MLP model, and calculate the target evaluation index corresponding to each candidate feature subset; S5a-3. Compare the target evaluation metrics for each candidate feature subset to determine the target feature set.

8. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 7, characterized in that, The specific method for feature optimization of the training set in step S5a-1 is as follows: S5a-1-1. Convert the training set into an original feature matrix; the column of the original feature matrix represents the number of samples in the training set, and the column represents the features of the samples. S5a-1-2. Use the UVE algorithm to add noise to the original feature matrix to obtain the concatenated matrix, and set the stability percentile threshold; S5a-1-3. Use PLRS for modeling, set the number of principal components, determine the contribution weight of each feature to the predicted value based on the number of principal components, and select a subset of candidate features based on the contribution weight of each feature to the predicted value. S5a-1-4. The range of principal component number is 5-20, and the range of stability percentile threshold is 90-99.

5. Based on the ranges corresponding to the principal component number and stability percentile threshold, repeat steps S5a-1-1-S5a-1-2 at least 50 times, and obtain the corresponding candidate feature subset each time.

9. The tomato shelf-life prediction method based on hyperspectral and RGB imaging technology according to claim 7, characterized in that, The target evaluation index includes the average coefficient of determination and the root mean square error. The features in the candidate feature subset with the largest average coefficient of determination and the smallest root mean square error are taken as the target feature set.

Images