Method and system for detecting maturity of red bayberry based on hyperspectrum

Abstract

The embodiment of the invention discloses a red bayberry maturity detection method and system based on hyperspectrum, and the method comprises the steps: obtaining a plurality of sample red bayberries including a plurality of varieties and red bayberries to be detected, and carrying out the hyperspectral image collection to obtain an original hyperspectral image; obtaining corresponding filtering parameters based on the noise characteristic parameters of each original hyperspectral image, and filtering the original hyperspectral images based on the filtering parameters to obtain processed hyperspectral images; obtaining chemical detection data corresponding to each sample waxberry; based on the chemical detection data corresponding to each sample waxberry and the processed hyperspectral image, training the to-be-trained model to obtain a prediction model; and taking the processed hyperspectral image of the to-be-detected waxberry as the input of the prediction model to obtain prediction data for measuring the maturity of the to-be-detected waxberry. Rapid, accurate and convenient waxberry maturity detection is realized, and the problem of poor variety adaptability in detection is solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of fruit ripeness detection, and in particular to the optimization of a waxberry ripeness detection method. BACKGROUND

[0002] Due to the short ripening period, the lack of outer skin covering, and the vulnerability of the fruit, accurate detection of the ripeness of waxberries is directly related to the picking quality, preservation ability, and market competitiveness. Traditional methods of judgment relying on human experience are highly subjective, have non-uniform standards, and can easily cause fruit loss. Although destructive physicochemical detection methods have high precision, they cannot meet the needs of large-scale non-destructive detection. Therefore, developing efficient and non-destructive ripeness detection technology is of great significance to the sustainable development of the waxberry industry.

[0003] Existing waxberry ripeness detection technologies are mainly based on optical imaging methods. Some methods use RGB color images to extract color features through color conversion (such as RGB to LAB space). Other methods use hyperspectral imaging technology to capture spectral information of the fruit to invert internal quality parameters. In addition, some studies combine machine learning models (such as convolutional neural networks) to process image data, achieving automatic classification or prediction of ripeness.

[0004] However, existing technologies have significant defects in handling differences between different varieties of waxberries. Different varieties (such as the Euryale ferox and Dongkui varieties) have inherent differences in fruit size, shape, and surface hair density. These differences lead to unstable optical response signals, limiting the precision and robustness of existing methods and making it difficult to achieve reliable non-destructive detection in practical applications. SUMMARY

[0005] Embodiments of the present specification provide a hyperspectral-based waxberry ripeness detection method and system. By acquiring hyperspectral images of waxberry fruits, adaptively matching image filtering parameters, and constructing a waxberry ripeness detection model, the problem of poor variety adaptability in non-destructive detection of waxberry ripeness is overcome, achieving a high-precision, low-loss, fast, and multi-variety applicable detection method.

[0006] The technical solution is as follows:

[0007] In a first aspect, embodiments of the present specification provide a hyperspectral-based waxberry ripeness detection method, comprising the following steps:

[0008] Obtain multiple sample waxberries of multiple varieties and a to-be-detected waxberry belonging to any of the varieties, and perform hyperspectral image acquisition to obtain original hyperspectral images corresponding to each of the to-be-detected waxberry and the sample waxberries;

[0009] The noise characteristic parameter of each original hyperspectral image is used to obtain the corresponding filtering parameter, and the original hyperspectral image is filtered based on the filtering parameter to obtain the processed hyperspectral image corresponding to each sample bayberry and each sample bayberry, and the filtering parameter includes a filtering window size and a polynomial order.

[0010] The chemical detection data corresponding to each sample bayberry is obtained.

[0011] The training model is trained based on the chemical detection data corresponding to each sample bayberry and the processed hyperspectral image to obtain a prediction model.

[0012] The processed hyperspectral image of the bayberry to be detected is used as the input of the prediction model to obtain prediction data for measuring the maturity of the bayberry to be detected.

[0013] As a preferred scheme, the noise characteristic parameter of each original hyperspectral image is used to obtain the corresponding filtering parameter, including:

[0014] Each original hyperspectral image is averaged and smoothed based on a preset window size to obtain a smoothed image;

[0015] The spectral noise variance corresponding to each original hyperspectral image is obtained as the noise characteristic parameter based on each original hyperspectral image and the corresponding smoothed image thereof;

[0016] The spectral noise variance corresponding to each original hyperspectral image is obtained as the noise characteristic parameter based on each original hyperspectral image and the corresponding smoothed image thereof;

[0017] As a preferred scheme, the noise characteristic parameter of each original hyperspectral image is used to obtain the corresponding filtering parameter, including:

[0018] The spectral noise variance threshold interval corresponding to each sample bayberry of each variety is obtained based on the spectral noise variance corresponding to each original hyperspectral image of the same variety of all sample bayberries;

[0019] The boundary value between each variety is obtained based on the spectral noise variance threshold interval corresponding to each sample bayberry of each variety;

[0020] The filtering parameter-spectral noise variance segmentation function is obtained based on the boundary value between each variety;

[0021] The filtering parameter corresponding to each original hyperspectral image is obtained based on the filtering parameter-spectral noise variance segmentation function and the spectral noise variance corresponding to each original hyperspectral image.

[0022] As a preferred scheme, the boundary value between each variety is obtained based on the spectral noise variance threshold interval corresponding to each sample bayberry of each variety, including:

[0023] Based on the spectral noise variance corresponding to each of the original hyperspectral images of the samples of each variety of bayberry, obtain the overall spectral noise variance corresponding to each of the samples of each variety of bayberry;

[0024] Based on the overall spectral noise variance corresponding to each of the samples of each variety of bayberry and the spectral noise variance threshold interval, obtain the boundary value between each variety.

[0025] As a preferred scheme, the filtering parameter-based filtering processing of the original hyperspectral image to obtain the processed hyperspectral image corresponding to each of the to-be-detected bayberry and the sample bayberry includes:

[0026] Filtering parameter-based filtering processing of the original hyperspectral image to obtain a filtered hyperspectral image;

[0027] Based on the image enhancement strategy, the filtered hyperspectral image is enhanced to obtain the processed hyperspectral image corresponding to each of the to-be-detected bayberry and the sample bayberry;

[0028] The determination of the image enhancement strategy includes:

[0029] Based on a plurality of different candidate strategies, the processed hyperspectral image is enhanced, and a to-be-trained model is trained to obtain a corresponding prediction model;

[0030] Based on the prediction accuracy of each prediction model, the image enhancement strategy is determined.

[0031] Each of the candidate strategies is at least one of differential transformation, logarithmic transformation, and reciprocal transformation.

[0032] As a preferred scheme, the to-be-trained model is a UNet convolutional neural network model including a three-dimensional convolutional kernel.

[0033] As a preferred scheme, the UNet convolutional neural network model further includes a deep network structure and a residual structure arranged between layers of the deep network structure.

[0034] In a second aspect, the embodiments of the present specification provide a hyperspectral-based bayberry maturity detection system, including an image acquisition unit, an image processing unit, a calibration data acquisition unit, a model training unit, and a model prediction unit:

[0035] The image acquisition unit acquires a plurality of sample bayberries including multiple varieties and a to-be-detected bayberry belonging to any of the varieties, and performs hyperspectral image acquisition to obtain original hyperspectral images corresponding to each of the to-be-detected bayberry and the sample bayberry;

[0036] The image processing unit obtains a respective filtering parameter based on a noise characteristic parameter of each original hyperspectral image, and performs filtering processing on the original hyperspectral image based on the filtering parameter to obtain a respective processed hyperspectral image corresponding to each sample bayberry and the bayberry to be detected, wherein the filtering parameter includes a filtering window size and a polynomial order.

[0037] The calibration data acquisition unit obtains respective chemical detection data of each sample bayberry.

[0038] The model training unit trains the to-be-trained model based on the respective chemical detection data and the processed hyperspectral image of each sample bayberry to obtain a prediction model.

[0039] The model prediction unit takes the processed hyperspectral image of the bayberry to be detected as an input of the prediction model to obtain prediction data for measuring the maturity of the bayberry to be detected.

[0040] In a third aspect, an electronic device is provided, including a processor and a memory; the processor is connected with the memory; the memory is used for storing executable program codes; the processor runs a program corresponding to the executable program codes by reading the executable program codes stored in the memory, so as to execute the steps of the first aspect of the above-mentioned embodiments.

[0041] In a fourth aspect, a computer storage medium is provided, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the first aspect of the above-mentioned embodiments.

[0042] The technical solutions provided by some embodiments of the present specification have at least the following beneficial effects:

[0043] The present application solves the problems of difficulty in implementation and large loss in nondestructive detection of bayberry fruit maturity, proposes a filtering parameter self-adaptive adjustment optimization method while using hyperspectral imaging technology, and constructs a convolutional neural network model based on deep learning, which overcomes the problem of poor variety adaptability in nondestructive detection of bayberry maturity, realizes rapid, accurate and convenient detection of bayberry maturity, and has important significance for guaranteeing the quality of bayberry picking, improving the efficiency of later sorting and reducing labor costs, and improving the market competitiveness of bayberry in China.

[0044] 1. The hyperspectral imaging technology is used instead of the traditional imaging technology to shoot bayberry fruit images, fuse spatial and spectral dimension information, and obtain chemical detection data of bayberry fruits to form a data set.

[0045] 2. An adaptive adjustment and optimization method for filtering parameters is used to preprocess hyperspectral images to ensure that the model obtains the most effective information from the data.

[0046] 3. By combining image enhancement strategies including differential transformation, reciprocal transformation and logarithmic transformation, the spectral feature information of bayberry fruit is deeply mined from multiple dimensions.

[0047] 4. Based on the UNet model, we improved it by introducing three-dimensional convolution and residual structure to build a lossless, fast and accurate model for detecting the ripeness of bayberry fruits. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic flowchart of a method for detecting the maturity of bayberry based on hyperspectral imaging, provided in the embodiments of this specification.

[0050] Figure 2 This is a schematic diagram of the improved UNet convolutional neural network model structure in a hyperspectral-based method for detecting the maturity of bayberries provided in the embodiments of this specification.

[0051] Figure 3 This is a schematic diagram of the structure of a single-layer module in the UNet convolutional neural network model.

[0052] Figure 4 This is a schematic diagram of a hyperspectral-based waxberry maturity detection system provided in the embodiments of this specification.

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0054] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0055] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0056] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0057] Unlike other fruits and vegetables, the waxberry fruit lacks an outer skin; its surface is covered with a dense fuzzy structure. This fuzz induces strong spectral scattering and noise effects. In hyperspectral imaging, the fuzz causes multiple reflections and scatterings of incident light, resulting in random fluctuations in the spectral curve and obscuring intrinsic characteristics related to maturity. Furthermore, the density and distribution patterns of the fuzz vary depending on the variety, making it difficult to apply uniform testing standards. Therefore, waxberry maturity detection faces unique challenges. Hence, this application is submitted.

[0058] Reference Figure 1 As shown, Figure 1 A flowchart illustrating a method for detecting the maturity of bayberry based on hyperspectral imaging, provided as an embodiment of this specification, may include at least the following steps:

[0059] Step 102: Obtain multiple samples of bayberries of various varieties and bayberries belonging to any of the varieties to be tested, and perform hyperspectral image acquisition to obtain the original hyperspectral images of the bayberries to be tested and each sample of bayberries.

[0060] Step 104: Obtain the corresponding filtering parameters based on the noise feature parameters of each original hyperspectral image, and perform filtering processing on the original hyperspectral image based on the filtering parameters to obtain the bayberry to be detected and the processed hyperspectral image corresponding to each sample bayberry. The filtering parameters include the filtering window size and the polynomial order.

[0061] Step 106: Obtain the chemical test data corresponding to each sample of bayberry;

[0062] Step 108: Based on the chemical detection data and processed hyperspectral image corresponding to each sample of bayberry, train the model to be trained to obtain the prediction model.

[0063] Step 110: Use the processed hyperspectral image of the bayberry to be tested as input to the prediction model to obtain prediction data that measures the maturity of the bayberry to be tested.

[0064] For illustrative purposes, this embodiment uses two bayberry varieties, *Eriocheir sinensis* and *Dongkui*, for illustration. Bayberry fruits from different regions were harvested throughout their entire growth period, averaging once a week as sample bayberries. The harvested sample bayberries were numbered, weighed, and their shape and size were measured. Then, hyperspectral images were acquired in a dark environment. First, bayberry fruits and a standard white board were placed horizontally on a platform covered with a black background cloth. The hyperspectral imager was positioned vertically downwards, 30 cm directly above the bayberry fruits. Two cold-light halogen lamps were placed on either side at the same height as the hyperspectral imager lens, at a 45° angle to the horizontal plane. In this dark environment, hyperspectral images were taken of each bayberry fruit individually. Raw hyperspectral images were obtained. The raw hyperspectral images of the collected sample bayberries were then preprocessed and the dataset was divided. Two problems exist in the preprocessing of the original hyperspectral images of collected bayberry samples: First, the strong spectral scattering effect caused by the dense pubescence on the surface of the bayberry fruit leads to random noise interference in the spectral curve, masking the characteristics related to the fruit's internal quality. Second, different bayberry varieties have inherent differences in the density of epidermal pubescence and the peel structure, resulting in a significant variety-specific distribution of spectral noise. The commonly used Savitzky-Golay (SG) spectral smoothing and noise reduction method, which uses a fixed window size and polynomial order, is difficult to adapt to these differences. For high-noise varieties, fixed parameters easily lead to incomplete noise reduction, with residual noise interfering with subsequent feature extraction; for low-noise varieties, excessive smoothing will cause the loss of spectral details. To address these issues, an adaptive parameter adjustment optimization method is proposed.

[0065] Specifically: Before smoothing filtering, the noise feature parameters of the original hyperspectral image are obtained to distinguish the bayberry varieties, and a suitable filter window size and polynomial order are matched for filtering to obtain the processed hyperspectral image. This achieves a precise match between the variety and the spectral smoothing requirements, preserves the feature details in the spectral curve to the maximum extent, and provides a more reliable input for the subsequent maturity prediction model.

[0066] In some embodiments, reflectance correction can be performed on the acquired raw hyperspectral image before smoothing filtering. The purpose is to eliminate interfering factors that are not inherent properties of the target itself. The formula is as follows:

[0067]

[0068] Wherein, DN is the original gray value of the pixel in the original hyperspectral image, and DNmin / DNmax are the minimum and maximum values ​​of that band in the original hyperspectral image.

[0069] Furthermore, various indicators of the collected bayberry samples were measured using chemical detection methods and correlated one-to-one with their original hyperspectral images as calibration data for measuring maturity.

[0070] In one embodiment of this application, the chemical detection data includes total sugar content, total acid content, and soluble solids.

[0071] Interpretive methods select indicators representative of bayberry fruit maturity for research, providing a comprehensive maturity assessment that outperforms single-indicator methods. This ensures the objectivity and quantifiability of the prediction results, providing highly reliable labels for model training. Specifically:

[0072] 1. The total sugar content of bayberry fruit was measured using a kit. Specifically, the bayberry fruit was first pitted, then the pulp was cooled and ground into powder using liquid nitrogen. After weighing and counting, 1.5 ml of distilled water and 1 ml of reagent one (hydrochloric acid solution) were added. After incubating in a 95°C water bath for 30 min, 1 ml of reagent two (NaOH solution) was added, and distilled water was added to the solution to bring the volume to 10 ml. Then, the solution was centrifuged at 8000 g at 25°C for 10 min. After centrifugation, 12 μl of the supernatant was collected, and 12 μl of the supernatant was added to the solution. Distilled water and 18 μl of Reagent III (Solution A: phenol, NaOH, and sodium bisulfite solution; Solution B: potassium sodium tartrate, NaOH, and 3,5-dinitrosalicylic acid solution. Reagent III is obtained by mixing solutions A and B) were incubated in a 95°C water bath for 10 minutes. Then, 258 μl of distilled water was added to the solution, and after thorough mixing, 200 μl of the solution was transferred to an ELISA plate. A blank tube was set up as a control, containing 24 μl of distilled water (12 μl of distilled water replacing 12 μl of sample supernatant) and 18 μl of Reagent III. Finally, the absorbance was measured at 540 nm using an ELISA reader. Each bayberry fruit sample was tested three times in total. The absorbance was calculated using the following formula:

[0073]

[0074] Where S represents the total sugar content (mg / g); The absorbance of the test tube is minus the absorbance of the blank tube; W is the sample mass; p is the dilution factor.

[0075] 2. The total acid content of bayberry fruit was measured using an acid-base indicator titration method. First, the bayberry fruit was pitted, then the pulp was cooled with liquid nitrogen and ground into powder. The powder was weighed, and distilled water was added to 20 ml. The mixture was then placed in a water bath at 80°C for 30 minutes and cooled. Next, distilled water was added to 50 ml for dilution. The solution was filtered using an Erlenmeyer flask, funnel, and filter paper. 10 ml of the solution was taken, and 10 ml of distilled water was added, followed by titration with 5 drops of phenolphthalein reagent. Finally, NaOH reagent was added for acid-base neutralization until the solution turned light red. The acid content was then counted. Each bayberry fruit sample was tested three times. The calculation formula is as follows:

[0076]

[0077] Where A is the total acid content (g / kg); C is the molar concentration of NaOH standard solution (0.1); V1 is the volume of NaOH standard solution consumed during titration; V0 is the volume of sample solution used for titration; and m(v) is the sample mass or volume.

[0078] 3. The soluble solids content of bayberry fruit was measured using the refractometer method.

[0079] After measuring the total sugar, total acid content, and soluble solids, outliers were removed, the experiment was repeated three times and the average value was taken. The processed hyperspectral images were matched one-to-one with the measured data, and a database of bayberry fruit maturity parameters was constructed by combining the hyperspectral images.

[0080] Explanatory, based on the physiological and biochemical laws governing the ripening process of bayberry fruit: total sugar content increases with maturity, total acid content decreases due to the degradation of organic acids, and soluble solids comprehensively reflect changes in the fruit's intrinsic quality. Single indicators are easily affected by environmental or varietal factors, while total sugar, total acid, and soluble solids characterize maturity from three dimensions: sweetness, acidity, and solid matter concentration, respectively. The complementary nature of multiple indicators can offset random errors. This multi-indicator synergistic calibration method provides a more comprehensive and objective scientific basis for maturity assessment and enhances the robustness of maturity inversion.

[0081] Furthermore, the chemical analysis data also includes the ratio of total sugar to total acid, which can more sensitively reflect subtle changes in maturity. For example, high sugar content may stem from varietal characteristics rather than maturity, but by combining it with low acid and high soluble solids data, true maturity can be accurately distinguished from varietal differences. This comprehensive strategy enables the model to learn more complex feature maps, avoiding misjudgments caused by fluctuations in a single feature.

[0082] Based on the acquired chemical detection data and processed hyperspectral images, a dataset is formed corresponding to multiple sets of data for each sample of bayberry. This dataset is then divided. For example, the entire dataset is divided in a 7:1.5:1.5 ratio, with the training set accounting for 70%, the validation set for 15%, and the test set for 15%. Model parameters such as the learning rate and number of iterations are adjusted based on the model training results to obtain the prediction model.

[0083] Preferably, the number of sample bayberries for each variety is the same. Maintaining variety sample balance is enforced during dataset partitioning to reduce the interference of variety differences on the model. Furthermore, the number of samples for each variety in the test set, validation set, and training set remains consistent. Given the inherent differences between bayberry varieties, such as the significant differences between the *Eriocheir sinensis* and *Eriocheir yunnanensis* varieties in fruit size, pubescence density, and spectral characteristics, a conventional random partitioning method might lead to over-representation of some varieties in the training set and under-representation of others in the test set, causing model training bias. The forced balancing strategy ensures a uniform distribution of each variety across different datasets, effectively preventing overfitting or underfitting of specific varieties. During model training, a balanced variety distribution allows the model to fully learn the characteristics of different varieties, rather than being dominated by varieties with larger sample sizes. Secondly, during model validation, a balanced validation set can more accurately evaluate the model's generalization performance across varieties and promptly identify model recognition deficiencies for certain varieties. Most importantly, during model testing, a balanced test set objectively reflects the model's true performance in real-world applications.

[0084] Finally, the processed hyperspectral image of the bayberry to be tested is used as the input of the prediction model to predict the total sugar, total acid content and soluble solids of the bayberry fruit, thereby measuring the maturity of the bayberry to be tested.

[0085] This embodiment overcomes the problem of poor variety adaptability in non-destructive testing of bayberry maturity, and realizes a high-precision, low-loss, rapid testing method applicable to multiple varieties.

[0086] In one embodiment of this specification, the corresponding filtering parameters are obtained based on the noise feature parameters of each original hyperspectral image, including:

[0087] Each original hyperspectral image is averaged and smoothed based on a preset window size to obtain a smoothed image;

[0088] The spectral noise variance of each original hyperspectral image and its corresponding smoothed image are obtained as noise feature parameters.

[0089] The corresponding filtering parameters are obtained based on the spectral noise variance of each original hyperspectral image.

[0090] Illustratively, the noise variance σ of the original hyperspectral image is calculated based on the original hyperspectral image and the smoothed image after average smoothing. 2 This serves as the corresponding noise characteristic parameter. The calculation formula is as follows:

[0091]

[0092] in, This represents the original reflectance of the original hyperspectral image at wavelength λ. The reflectance at wavelength λ after averaging and smoothing the original hyperspectral image; K is the half-width of the moving window (i.e., half the preset window size); L is the total number of spectral bands; (LK) is the number of effective calculated bands after removing edge bands.

[0093] A linear functional relationship between noise variance and filtering parameters is established to achieve adaptive noise reduction. It's easy to understand that varieties with relatively high noise levels are given a larger window and a lower-order polynomial to prioritize noise reduction; varieties with medium noise levels are given a medium window and a polynomial to balance noise reduction and detail preservation; and varieties with relatively low noise levels are given a smaller window and a polynomial to prioritize the preservation of feature details. Dynamically linking the spectral noise quantification index of a variety with the filtering parameters avoids the limitations of traditional fixed parameters, making the preprocessing effect more closely match the actual spectral characteristics of different bayberry varieties, maximizing the preservation of feature details in the spectral curves, and providing a more reliable input for the subsequent maturity detection model.

[0094] However, it's important to note that, on the one hand, because each newly collected sample requires independent calculation of the optimal filtering parameters, the computational load in the filtering process increases significantly, severely limiting the real-time performance of the detection. This computational burden is particularly problematic in high-throughput detection scenarios (such as processing thousands of fruits per hour), where it leads to a sharp decline in detection throughput. On the other hand, determining the coefficients of the linear function between noise variance and filtering parameters relies heavily on extensive experimental data, making it difficult to achieve a consistent filtering effect in the absence of objective standards. Inaccurate functional relationships can also lead to the loss of varietal characteristics, affecting the prediction accuracy of subsequent prediction models.

[0095] Therefore, in one embodiment of this specification, obtaining the corresponding filtering parameters based on the spectral noise variance of each original hyperspectral image includes:

[0096] Based on the original hyperspectral images of all samples of the same variety of bayberry, the spectral noise variance threshold range for each variety of bayberry sample is obtained.

[0097] The boundary values ​​between varieties are obtained based on the spectral noise variance threshold range corresponding to each variety of bayberry sample.

[0098] The filter parameters and spectral noise variance piecewise function are obtained based on the boundary values ​​between different varieties.

[0099] Based on the filter parameter-spectral noise variance piecewise function and the spectral noise variance corresponding to each original hyperspectral image, the filter parameters corresponding to each original hyperspectral image are obtained.

[0100] Interpretive analysis was conducted by statistically analyzing the spectral noise variance of all samples of different varieties of bayberry to obtain the spectral noise variance threshold intervals for each variety. Boundary values ​​were calculated based on the magnitude relationship of each spectral noise variance threshold interval to segment the data, and corresponding filtering parameters were set for each segment to obtain the filter parameter-spectral noise variance piecewise function. The filter parameters were set using the locally optimal values ​​of the filtering effect for different varieties obtained experimentally, to take into account the characteristics of each variety and avoid insufficient noise reduction or loss of feature details.

[0101] For example, if the sample bayberry includes three varieties, and the noise variance boundary values ​​between varieties are a and b, then the filter parameter-spectral noise variance piecewise function is:

[0102]

[0103] Where M is the size of the filter window; n is the order of the fitted polynomial.

[0104] After determining the filter parameters—the piecewise function of spectral noise variance—then the spectral noise variance σ of the sample bayberry and the bayberry to be tested is used as the basis for further analysis. 2 The comparison is performed with pre-set inter-variety noise variance boundary values, directly matching the corresponding filter parameters, including the filter window size and polynomial order, without any calculation. For example, after inputting the bayberry to be detected, the spectral noise variance σ of the corresponding original hyperspectral image is calculated. 2 The system automatically matches filter parameters and calls the mapping library to match the corresponding noise variance level. For products with high noise variance, a 15-point window + 2nd-order polynomial is used; for products with medium noise variance, an 11-point window + 3rd-order polynomial is used; and for products with low noise variance, a 9-point window + 3rd-order polynomial is used.

[0105] In some embodiments, obtaining the boundary values ​​between varieties based on the spectral noise variance threshold range corresponding to each variety of bayberry sample includes:

[0106] Based on the spectral noise variance corresponding to the original hyperspectral images of all samples of the same variety of bayberry, the overall spectral noise variance corresponding to each sample of bayberry variety is obtained.

[0107] The boundary values ​​between varieties are obtained based on the overall spectral noise variance and the spectral noise variance threshold range corresponding to each variety of bayberry sample.

[0108] The straightforward approach is to classify and calculate the spectral noise variance threshold intervals for different varieties of bayberry (c) based on the original hyperspectral image and the smoothed image after average smoothing. The spectral noise variance of all bayberry samples for any given variety constitutes the threshold interval for that variety. The spectral noise variance σ calculated using this method... 2 The values ​​are very small. Due to the significant differences in the dense pubescence characteristics on the surface of different varieties of bayberry fruit, the spectral noise variance threshold intervals corresponding to each variety usually do not overlap. Therefore, any value can be arbitrarily selected at the intervals between intervals as the boundary value for dividing the spectral noise variance of different varieties. However, for cases where the differences between varieties are not significant or due to random errors in a single sample, the spectral noise variance threshold intervals corresponding to different varieties may overlap, leading to difficulties in defining the boundary values ​​for the spectral noise variance of different varieties.

[0109] Therefore, the overall spectral noise variance for each variety c is calculated, and the overall spectral noise variance for any variety is calculated. The calculation formula is as follows:

[0110]

[0111] Where N is the number of samples for that variety; Let be the original reflectance of the original hyperspectral image of the i-th sample at wavelength λ; Let be the reflectance at wavelength λ after averaging and smoothing the original hyperspectral image of the i-th sample.

[0112] The spectral noise variance threshold ranges and overall spectral noise variances of the two intersecting varieties can be used as the basis for defining the inter-variety noise variance boundary values. For example, variety A has a spectral noise variance threshold range of [0.005, 0.008] and an overall spectral noise variance of 0.007; variety B has a spectral noise variance threshold range of [0.003, 0.007] and an overall spectral noise variance of 0.004. Therefore, the inter-variety noise variance boundary value for varieties A and B can be set as the mean of the two overall spectral noise variances, i.e., (0.007 + 0.004) / 2 = 0.0055. Other specific calculation methods for this concept of using the overall spectral noise variances of different varieties c to set the inter-variety noise variance boundary values ​​are all within the scope of this embodiment.

[0113] By averaging the spectral noise variance threshold intervals for each sample of the same variety of bayberry, the concentration value of the noise variance for each variety is determined, thus distinguishing the random errors of a single sample. The boundary value calculation method between varieties described above significantly reduces the probability of misjudging and matching incorrect filter parameters between two varieties with correlated boundary values, avoiding insufficient noise reduction or loss of feature details.

[0114] In one embodiment of this specification, filtering the original hyperspectral image based on filtering parameters yields the processed hyperspectral image of the bayberry to be detected and each sample bayberry, including:

[0115] The original hyperspectral image is filtered based on the filtering parameters to obtain the filtered hyperspectral image;

[0116] Based on the image enhancement strategy, the filtered hyperspectral image is enhanced to obtain the bayberry to be detected and the corresponding processed hyperspectral image of each sample bayberry.

[0117] The determination of image enhancement strategies includes:

[0118] The hyperspectral images were enhanced using a variety of different candidate strategies, and the training models were trained to obtain their respective prediction models.

[0119] Image enhancement strategies are determined based on the prediction accuracy of each prediction model;

[0120] Each candidate strategy is at least one of the differential transformation, logarithmic transformation, and reciprocal transformation.

[0121] Illustratively, a preprocessing combination strategy is introduced, employing one or more of the following: differential transformation, reciprocal transformation, and logarithmic transformation, to comprehensively enhance feature representation, outperforming single transformation methods. Building upon matched adaptive filtering parameters to denoise and ensure data quality, an image enhancement strategy is further employed to avoid transformation conflicts (such as noise amplification) in traditional methods, thus improving feature separability. This multi-dimensional collaborative strategy not only overcomes the limitations of traditional single transformations but also significantly improves the ability to invert the intrinsic qualities of bayberries (such as sugar and acid content).

[0122] This study aims to determine the optimal preprocessing method for predicting the ripeness of bayberry fruit, and thus, to identify the optimal image enhancement strategy. Differential transformation is combined with its reciprocal and logarithmic transformations. The differential transformation further includes first-order and second-order differentials, resulting in five preprocessing combinations (i.e., candidate strategies): first-order differential + reciprocal, first-order differential + logarithmic, second-order differential + reciprocal, second-order differential + logarithmic, and logarithmic + reciprocal. Each of these preprocessing combinations is then applied to a sample dataset. Finally, the detection accuracy of the sample datasets with different preprocessing combinations in different prediction models is compared to determine the final preprocessing combination (i.e., the image enhancement strategy).

[0123] The differential transform is primarily used to highlight and enhance the characteristic information in the spectral curve. Its principle is to perform a derivative operation on the original spectral reflectance curve. The spectral differential transform can highlight steeply changing bands in the spectral curve, making the effective features more prominent. This invention uses first-order and second-order differential transforms, with the following formulas:

[0124] First-order differential:

[0125]

[0126] Second-order differential:

[0127]

[0128] Where ρ' is the spectral value obtained by the first-order differential transform, ρ'' is the spectral value obtained by the second-order differential transform, i represents the number of spectral bands i=1,2…m, and 2Δλ represents the number of band intervals.

[0129] The principle of logarithmic transformation is to compress the dynamic range of the data by obtaining the logarithm of the data, while maintaining the relative relationships between the data. Reciprocal transformation is performed by taking the reciprocal of the original spectral reflectance. Both methods can make changes in spectral data more apparent, providing a basis for detecting the maturity of bayberries.

[0130] First-order and second-order derivatives highlight the abrupt changes in the spectral curve, reciprocal transformations compress the dynamic range, and logarithmic transformations enhance the contrast of details. The combination of these three methods allows for the mining of spectral features from multiple dimensions, including gradient, scale, and distribution, thus enhancing their complementarity.

[0131] In one embodiment of this specification, the model to be trained is a UNet convolutional neural network model including three-dimensional convolutional kernels.

[0132] Illustratively, the UNet model is a convolutional neural network model applied to image segmentation. Because traditional image data has a two-dimensional spatial structure (height × width), two-dimensional convolutional kernels are typically used, with the most prominent feature being its U-shaped symmetric structure. First, higher-level, more abstract features are captured through convolutional layers and downsampling. Second, the spatial dimensions of the feature map are gradually restored through upsampling and convolutional layers, improving the model's accuracy. This embodiment is an improvement on the UNet convolutional neural network model, constructing a model suitable for detecting the ripeness of bayberries.

[0133] Explaining this, in the processing of hyperspectral images of bayberry trees based on the UNet model, hyperspectral images are essentially three-dimensional data cubes of space and spectrum. Two-dimensional convolution can only slide the convolution kernel along the spatial dimension, disrupting the continuity of the spectral dimension and exhibiting significant technical adaptation defects. It fails to capture the "cooperative features of multi-band spectra at a certain spatial location," making it difficult for the model to utilize the three-dimensional intrinsic correlations of hyperspectral data. Therefore, the two-dimensional convolution in UNet is replaced with three-dimensional convolution. This allows the UNet model to fully adapt to the three-dimensional data of hyperspectral images. This structure gives the model two major advantages:

[0134] 1. Spatial-spectral feature co-capture: The three-dimensional convolution kernel can slide along both the spatial and spectral dimensions simultaneously. While extracting spatial features such as fruit outline and pubescence distribution, it can simultaneously capture the spectral response patterns between different bands, thus realizing the correlation modeling of "spatial location-spectral signal".

[0135] 2. Preservation of 3D Information Integrity: Compared with the "flattening" of the spectral dimension by 2D convolution, 3D convolution can completely preserve the structural characteristics of the hyperspectral data cube, avoiding information loss of spectral features during preprocessing. It is especially suitable for extracting key information such as "spectral distribution differences of velvet scattering noise" and "specific characteristic bands of different varieties" in the hyperspectral data of bayberry.

[0136] Adaptive preprocessing provides clean input to the model, and the 3D UNet model deeply mines features, ultimately achieving non-destructive, fast, and high-precision detection.

[0137] In one embodiment of this specification, the UNet convolutional neural network model further includes a deep network structure and residual structures disposed between the layers of the deep network structure.

[0138] To illustrate, to fully explore the deep spatial-spectral correlation features in the hyperspectral data of bayberry, it is necessary to further deepen the network layers. However, deep networks are prone to the problems of vanishing or exploding gradients, where gradients continuously decay or amplify after undergoing multiple nonlinear transformations and parameter updates during backpropagation. This makes it difficult to effectively optimize the parameters of shallow networks, ultimately affecting the model's ability to learn subtle features. To address this issue, this embodiment introduces a residual structure into the 3D convolutional UNet model. The input features of the convolutional layer are directly fused with the output features after nonlinear transformations (convolution, activation). The core logic can be expressed as: F(x) + x (where x is the input feature and F(x) is the feature after nonlinear transformation).

[0139] For example, this invention improves the UNet network model. The specific operations are as follows:

[0140] Reference Figure 2 , Figure 3 ,Figure 2 This is a schematic diagram of the improved UNet convolutional neural network model structure in a hyperspectral-based method for detecting the maturity of bayberries provided in the embodiments of this specification. Figure 3 This is a schematic diagram of a single-layer module in the UNet convolutional neural network model. The improved backbone network consists of 9 stages. Stages 1 to 4 use 3D convolution and max pooling downsampling for operations, while the remaining stages use 3D convolution and upsampling. Stages 1 to 4 all contain 3*3*3 convolutional layers and max pooling downsampling layers, but the number of convolutional kernels varies in each stage: 64, 128, 256, and 512, respectively. Stages 5 to 8 all contain 3*3*3 convolutional layers, 1*1*1 convolutional layers, and upsampling layers. At the end of each stage, the upsampling layer adjusts the spatial size of the output feature map to be the same as the corresponding stage and concatenates it with the feature map of the corresponding stage (here, stage 6 corresponds to stage 4, stage 7 corresponds to stage 3, stage 8 corresponds to stage 2, and stage 9 corresponds to stage 1). Stage 9 consists of 3x3x3 convolutional layers and 1x1x1 convolutional layers. The number of convolutional kernels in stages 5 through 9 are 512, 512, 256, 128, and 64, respectively. In the first eight stages, each stage connects the input and output feature maps via residual connections before performing downsampling or upsampling operations. After stage 9, the result is output through three fully connected layers. Through residual structure and parameter optimization, the model's computational cost is reduced, supporting deployment on embedded devices.

[0141] The dual transformation of 3D convolution and residual structure not only solves the limitation of traditional UNet in making it difficult to utilize 3D information, but also breaks through the bottleneck of gradient propagation in deep networks, enabling the model to learn the complex features in the hyperspectral data of bayberry more efficiently, and providing more reliable model support for subsequent tasks such as variety identification and quality inversion.

[0142] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0143] Please refer to the following. Figure 4 , Figure 4 A schematic diagram of a hyperspectral-based waxberry maturity detection system provided in an embodiment of this specification is shown.

[0144] The maturity detection system 400 includes an image acquisition unit 401, an image processing unit 402, a calibration data acquisition unit 403, a model training unit 404, and a model prediction unit 405.

[0145] Image acquisition unit 401 acquires multiple sample bayberries of various varieties and bayberries to be tested belonging to any of the varieties, and performs hyperspectral image acquisition to obtain the original hyperspectral image corresponding to the bayberry to be tested and each sample bayberry.

[0146] The image processing unit 402 obtains the corresponding filtering parameters based on the noise feature parameters of each original hyperspectral image, and performs filtering processing on the original hyperspectral image based on the filtering parameters to obtain the bayberry to be detected and the processed hyperspectral image corresponding to each sample bayberry. The filtering parameters include the filtering window size and the polynomial order.

[0147] The calibration data acquisition unit 403 acquires the chemical detection data corresponding to each sample of bayberry.

[0148] The model training unit 404 trains the model to be trained based on the chemical detection data and processed hyperspectral image corresponding to each sample of bayberry to obtain the prediction model.

[0149] The model prediction unit 405 takes the processed hyperspectral image of the bayberry to be detected as the input of the prediction model to obtain prediction data that measures the maturity of the bayberry to be detected.

[0150] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiment of the waxberry maturity detection system is basically similar to the embodiment of the waxberry maturity detection method, so the description is relatively simple; relevant parts can be referred to in the description of the waxberry maturity detection method embodiment.

[0151] Please see Figure 5 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0152] like Figure 5 As shown, the electronic device 500 may include at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0153] The communication bus 502 can be used to realize the connection and communication of the above components.

[0154] The user interface 503 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0155] The network interface 504 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0156] The processor 501 may include one or more processing cores. The processor 501 connects to various parts within the electronic device 500 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by calling data stored in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of DSP, FPGA, or PLC. The processor 501 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0157] The memory 505 may include RAM or ROM. Optionally, the memory 505 may include a non-transitory computer-readable medium. The memory 505 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. As a computer storage medium, the memory 505 may include an operating system, a network communication module, a user interface module, and a bayberry maturity detection application. The processor 501 may be used to call the bayberry maturity detection application stored in the memory 505 and execute the steps of the bayberry maturity detection method mentioned in the foregoing embodiments.

[0158] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the embodiments of the above-described bayberry maturity detection method. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0159] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this specification is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0161] The above embodiments are merely preferred embodiments described in this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A method for detecting the maturity of red bayberry based on hyperspectral, characterized in that, The method comprises the following steps: obtaining a plurality of sample red bayberries of multiple varieties and a red bayberry to be detected belonging to any one of the varieties, and performing hyperspectral image acquisition to obtain original hyperspectral images corresponding to the red bayberry to be detected and each sample red bayberry respectively; obtaining filter parameters corresponding to each original hyperspectral image based on noise characteristic parameters of the original hyperspectral image, and performing filter processing on the original hyperspectral image based on the filter parameters to obtain processed hyperspectral images corresponding to the red bayberry to be detected and each sample red bayberry respectively, wherein the filter parameters comprise a filter window size and a polynomial order; obtaining chemical detection data corresponding to each sample red bayberry respectively; training a prediction model based on the chemical detection data corresponding to each sample red bayberry respectively and the processed hyperspectral images to obtain the prediction model; taking the processed hyperspectral image of the red bayberry to be detected as an input of the prediction model to obtain prediction data for measuring the maturity of the red bayberry to be detected.

2. The method for detecting the maturity of waxberry based on hyperspectral according to claim 1, characterized in that, The method comprises the following steps: performing average smoothing on each original hyperspectral image based on a preset window size to obtain a smoothed image; obtaining spectral noise variances corresponding to each original hyperspectral image as noise characteristic parameters based on the original hyperspectral image and the smoothed image corresponding to the original hyperspectral image respectively; obtaining filter parameters corresponding to each original hyperspectral image based on the spectral noise variances corresponding to the original hyperspectral image respectively.

3. The method for detecting the maturity of waxberry based on hyperspectral according to claim 2, characterized in that, The method comprises the following steps: obtaining spectral noise variance threshold intervals corresponding to each variety of sample red bayberry based on the spectral noise variances corresponding to the original hyperspectral images of all sample red bayberries of the same variety respectively; obtaining boundary values between varieties based on the spectral noise variance threshold intervals corresponding to each variety of sample red bayberry respectively; obtaining a filter parameter-spectral noise variance segmentation function based on the boundary values between varieties; obtaining filter parameters corresponding to each original hyperspectral image based on the filter parameter-spectral noise variance segmentation function and the spectral noise variances corresponding to the original hyperspectral image respectively.

4. The method for detecting the maturity of waxberry based on hyperspectral according to claim 3, characterized in that, The method comprises the following steps: obtaining overall spectral noise variances corresponding to each variety of sample red bayberry based on the spectral noise variances corresponding to the original hyperspectral images of all sample red bayberries of the same variety respectively; obtaining boundary values between varieties based on the overall spectral noise variances corresponding to each variety of sample red bayberry respectively and the spectral noise variance threshold intervals.

5. The method for detecting the maturity of waxberry based on hyperspectral according to claim 1, characterized in that, The method comprises the following steps: performing filter processing on the original hyperspectral image based on the filter parameters to obtain a filtered hyperspectral image; performing enhancement processing on the filtered hyperspectral image based on an image enhancement strategy to obtain the processed hyperspectral images corresponding to the red bayberry to be detected and each sample red bayberry respectively; The method comprises the following steps: The processed hyperspectral images are enhanced based on different candidate strategies, and a to-be-trained model is trained to obtain a corresponding prediction model; The image enhancement strategy is determined based on the prediction accuracy of each prediction model; Each of the candidate strategies is at least one of differential transformation, logarithmic transformation, and reciprocal transformation.

6. The method for detecting the maturity of waxberry based on hyperspectral according to claim 1, characterized in that, The to-be-trained model is a UNet convolutional neural network model including a three-dimensional convolution kernel.

7. The method for detecting the maturity of waxberry based on hyperspectral according to claim 6, characterized in that, The UNet convolutional neural network model further includes a deep network structure and a residual structure arranged between layers of the deep network structure.

8. A hyperspectral-based detection system for the ripeness of waxberries, characterized by, The system includes an image acquisition unit, an image processing unit, a calibration data acquisition unit, a model training unit, and a model prediction unit. The image acquisition unit acquires multiple sample red bayberries of multiple varieties and a to-be-detected red bayberry belonging to any of the varieties, and performs hyperspectral image acquisition to obtain original hyperspectral images corresponding to the to-be-detected red bayberry and each sample red bayberry; The image processing unit obtains a filter parameter corresponding to each original hyperspectral image based on a noise feature parameter of the original hyperspectral image, and performs filter processing on the original hyperspectral image based on the filter parameter to obtain a processed hyperspectral image corresponding to the to-be-detected red bayberry and each sample red bayberry, the filter parameter including a filter window size and a polynomial order; The calibration data acquisition unit acquires chemical detection data corresponding to each sample red bayberry; The model training unit trains the to-be-trained model based on the chemical detection data and the processed hyperspectral image corresponding to each sample red bayberry to obtain a prediction model; The model prediction unit inputs the processed hyperspectral image of the to-be-detected red bayberry into the prediction model to obtain prediction data for measuring the maturity of the to-be-detected red bayberry.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, the computer-readable storage medium having instructions stored therein, which, when executed on a computer or processor, cause the computer or processor to perform the steps of the method of any one of claims 1-7.

Images