A method and system for predicting sugar content of fruits based on multispectral images

Abstract

The application provides a fruit sugar content prediction method and system based on a multispectral image, comprising the following steps: S1, constructing a multi-band original image group; S2, obtaining a multi-band reflectivity image group through white-dark reference correction and spatial registration processing and performing target segmentation to obtain a complete fruit mask; S3, obtaining a safe region of interest through boundary inward shrinking processing and removing corresponding pixels according to pixel reflectivity distribution to obtain a robust sampling region; S4, extracting a multispectral robust feature vector based on the robust sampling region; and S5, inputting the multispectral robust feature vector into a sugar content prediction model after standardization processing to output a sugar content prediction value. The application can construct a safe region of interest which can effectively avoid fruit edges, background mixing, high light and shadow interference from a multispectral fruit image under limited waveband and small sample conditions, and extract a multispectral statistical feature with robustness to realize accurate fruit sugar content prediction.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and more specifically, to a method and system for predicting the sugar content of fruits based on multispectral images. Background Technology

[0002] Fruit sugar content is a key indicator for measuring its maturity and flavor quality, and it has significant application value in post-harvest sorting, storage, and sales. Currently, non-destructive testing methods based on near-infrared spectroscopy, hyperspectral imaging, and multispectral imaging are gradually becoming a research hotspot. However, existing fruit sugar content prediction technologies based on spectral or multispectral images still have the following shortcomings: Firstly, regarding the extraction of regions of interest, existing methods often directly use the entire fruit region or a fruit mask obtained from simple segmentation as the feature extraction object. Because the fruit surface has a curved structure, the fruit edge region is prone to abrupt changes in reflection angle, dark edges, and background pixel intrusion; simultaneously, local highlights and shadows on the peel can cause abnormal fluctuations in reflectivity across certain wavelengths. If the mean or simple statistics are directly calculated over the entire fruit region, the resulting features are easily affected by these unstable pixels, leading to significant fluctuations in the sugar content prediction model under small sample conditions and insufficient prediction stability.

[0003] Secondly, in multispectral imaging scenarios with limited bands, existing technologies typically focus on selecting characteristic wavelengths and extracting average spectra from continuous hyperspectral bands. For portable multispectral detection systems with a limited number of bands, the effects of highlights, dark spots, and edge noise in a single band image are more significant. If only the mean, standard deviation, or intensity of a single band is used as modeling input, the model is highly susceptible to the influence of acquisition posture, illumination intensity, and batch-specific differences in samples, resulting in poor cross-batch generalization ability and making it difficult to meet the engineering requirements of rapid on-site detection.

[0004] Furthermore, the collection of fruit sugar content data is often limited by factors such as sample size, harvest batch, and measurement time, making it difficult to obtain large-scale, highly balanced training data. Existing regression models (such as support vector regression and partial least squares regression) cannot effectively reduce the interference of fruit edges, background blending, local shadows, and fruit peel highlights during the feature extraction stage. Even with complex nonlinear regression algorithms, it is difficult to guarantee stable and accurate prediction results under small sample conditions.

[0005] In summary, how to construct a safe region of interest from multispectral fruit images under limited band and small sample conditions, effectively avoiding interference from fruit edges, background intrusion, highlights, and shadows, and extract robust multispectral statistical features to accurately predict fruit sugar content, is a technical problem that urgently needs to be solved in the field. Summary of the Invention

[0006] The technical problem to be solved by this invention is how to construct a safe region of interest from multispectral fruit images under limited band and small sample conditions, which can effectively avoid interference from fruit edges, background blending, highlights and shadows, and extract robust multispectral statistical features to achieve accurate prediction of fruit sugar content. In order to overcome the defects of the above-mentioned prior art (or related art), this invention provides a method and system for predicting fruit sugar content based on multispectral images.

[0007] This invention provides a method for predicting the sugar content of fruits based on multispectral images, comprising the following steps: Step S1: Collect original images of the fruit to be tested in multiple narrowband bands to construct a multiband original image group; Step S2: Perform white-dark reference correction and spatial registration on the multi-band original image group to obtain a multi-band reflectance image group in the same spatial coordinate system and perform target segmentation to obtain a complete fruit mask representing the complete main body area of ​​the fruit to be tested. Step S3: Perform boundary shrinkage processing on the complete fruit mask to obtain a safe region of interest, and remove a preset proportion of bright and low-bright pixels within the safe region of interest according to the pixel reflectance distribution to obtain a robust sampling region; Step S4: Based on the robust sampling region, extract the median and interquartile range of reflectance for each band, the band ratio feature between any two bands, and the normalized difference index feature to form a multispectral robust feature vector. Step S5: After standardizing the multispectral robust feature vector, input it into the pre-trained sugar content prediction model to output the sugar content prediction value of the fruit to be tested.

[0008] Compared with existing technologies, the fruit sugar content prediction method based on multispectral images of this invention has the following advantages: This invention constructs a safe region of interest and removes abnormal bright and dark pixels. Combined with boundary shrinkage processing, it avoids unstable reflective regions, allowing subsequent feature extraction to focus more on stable areas of the fruit body. This effectively avoids interference from fruit edges, background intrusion, peel highlights, and local shadows on spectral features. Furthermore, it extracts median, interquartile range, band ratio features, and normalized difference index features. These features not only reflect the relative spectral responses related to sugar content across different bands but also mitigate the impact of light intensity variations, local noise, and sample pose differences. This helps overcome the shortcomings of traditional mean features, which are easily affected by sampling pose and light fluctuations under limited band and small sample conditions. Finally, by combining standardized multispectral robust feature vectors with a sugar content prediction model, it achieves rapid and accurate prediction of fruit sugar content, significantly improving generalization ability across samples and batches.

[0009] In one possible implementation, step S2, before performing white-dark reference correction, further includes acquiring dark reference images and white reference images corresponding to each of the narrowband bands. The white-dark reference correction process includes: The first The original image of each band at pixel location The gray value at the position minus the first Each band of dark reference image at pixel location The difference between the gray values ​​at each position is used as the numerator value, and the value of the first position is used as the numerator value. The original image of each band at pixel location The gray value at the position minus the first The white reference image for each band is located at the pixel position. The difference between the gray values ​​at each point is added to a preset constant to obtain the denominator value. Then, the numerator value is divided by the denominator value to obtain the first... Each band at pixel position Corrected reflectivity at that location.

[0010] Compared with existing technologies, the above-mentioned technical solution can eliminate camera dark current and environmental background noise by acquiring dark reference images, correct the differences in light source intensity and camera response in different bands by acquiring white reference images, and obtain the corrected reflectance by calculating the difference ratio. This can effectively eliminate the nonlinear effects of hardware system and ambient light, so that the corrected reflectance data can more realistically reflect the spectral response characteristics of the surface of the fruit under test, and lay a reliable physical quantity foundation for subsequent multispectral robust feature vector extraction.

[0011] In one possible implementation, the boundary shrinkage process in step S3 employs morphological erosion, boundary distance constraint, or proportional shrinkage to ensure that the safe region of interest avoids the unstable reflective regions, background blending regions, highlight regions of the fruit peel, and local shadow regions at the fruit edge.

[0012] Compared with existing technologies, the above-mentioned technical solution can actively avoid unstable reflection areas, background blending areas, highlight areas of the peel, and local shadow areas of the safe region of interest by using a flexible boundary shrinkage processing strategy. This avoids the problems of dark edges, occlusion, and background pixel blending caused by curved surface structures. Compared with traditional full fruit region feature extraction methods, it significantly reduces the interference of edge artifacts on sugar content prediction and improves the robustness of feature extraction.

[0013] In one possible implementation, the preset ratio in step S3 is 5%, and the pixels in the safe region of interest are sorted from high to low brightness, and the bright pixels in the top 5% and the low-brightness pixels in the bottom 5% are removed.

[0014] Compared with existing technologies, the above technical solution can effectively filter out extreme outliers such as local highlights, dark spots and noise pixels on the fruit peel, while retaining enough effective pixels for statistical feature calculation. It improves feature stability while avoiding the problem of the sampling area being too small and the representativeness decreasing due to the removal of too many pixels, thus achieving a good balance between anti-interference ability and information retention.

[0015] In one possible implementation, when removing a preset proportion of the bright pixels and the low-brightness pixels according to the pixel reflectance distribution in step S3, the bright pixels and the low-brightness pixels are sorted according to the single-band reflectance distribution, the multi-band average reflectance distribution, and the main band reflectance distribution, and then the pixels located at both ends after sorting are removed.

[0016] Compared with existing technologies, the above-mentioned technical solution can provide a variety of flexible abnormal pixel identification strategies. The most suitable sorting method can be selected according to the actual imaging conditions and band configuration, so that the generation of robust sampling regions does not depend on the accidental fluctuations of specific bands, further enhancing the adaptability and robustness of feature extraction.

[0017] In one possible implementation, in step S4, the interquartile range is obtained by calculating the difference between the upper and lower quartiles of the reflectance of the corresponding band, the band ratio feature is obtained by calculating the ratio of the median reflectance of any two bands, and the normalized difference index feature is obtained by calculating the ratio of the difference between the median reflectance of any two bands to the sum of the median reflectance.

[0018] Compared with existing technologies, the above-mentioned technical solution can reflect the dispersion of reflectance within a robust sampling area through interquartile range and is insensitive to outliers; while the band ratio feature and normalized difference index feature can eliminate the influence of the proportional changes in light source intensity and camera response on a global scale, highlighting the relative spectral differences related to sugar content between different bands; the combination of the three types of features not only preserves single-band information but also enhances the expressive power of nonlinear spectral relationships, effectively reducing the interference of changes in light intensity and acquisition conditions on sugar content prediction values.

[0019] In one possible implementation, the sugar content prediction model in step S5 adopts one of the following: support vector regression model, kernel ridge regression model, partial least squares regression model, and random forest regression model, and the sugar content prediction model is trained using the measured sugar content values ​​of the fruit as supervision labels.

[0020] Compared with existing technologies, the above technical solution can provide multiple application options for the sugar content prediction model, adapting to different application scenarios. Using the measured sugar content value as a supervision label for training ensures the physical interpretability of the sugar content prediction model output. Users can flexibly select the optimal model according to the actual amount of data and accuracy requirements, thus improving engineering adaptability.

[0021] In one possible implementation, when pre-training the sugar content prediction model before performing step S1, for the same sample of tested fruit, multi-angle image acquisition is performed according to different sides, and the image acquisition results of each side are subjected to side-level averaging or fruit-level fusion processing to obtain fusion features. Then, the fusion features are input into the sugar content prediction model for training.

[0022] Compared with existing technologies, the above technical solution takes into account the potential spatial non-uniformity of sugar content distribution on the fruit surface. Single-sided sampling can easily introduce pose deviations. Therefore, by averaging or fusing image features from different sides of the same fruit, sampling errors caused by differences in fruit orientation and curvature can be eliminated, resulting in higher consistency between training labels and input features, thereby improving the prediction accuracy and repeatability of the sugar content prediction model.

[0023] In one possible implementation, after performing step S5, the method further includes: When executed on a PC, a corresponding sugar content heatmap is generated based on the predicted sugar content value and in conjunction with local sliding window or dense image patch inference methods; or When executed on the embedded end, the predicted sugar content, maximum sugar content, minimum sugar content, and average sugar content are output in a lightweight result display mode, while the sample number of the fruit to be tested, the image acquisition time, and the multi-band original image group are saved.

[0024] Compared with existing technologies, the above-mentioned technical solution can take into account both detailed laboratory analysis and rapid on-site detection. On the PC, the sugar content distribution heatmap can be used for scientific research or quality grading strategy formulation; on the embedded device, the lightweight results show that it meets the rapid detection needs of portable devices in fields, warehouses and other scenarios. This dual-mode design significantly improves the coverage of application scenarios and engineering practical value.

[0025] This invention also provides a fruit sugar content prediction system based on multispectral images, which applies the above-mentioned fruit sugar content prediction method based on multispectral images, including: The image acquisition module is used to acquire raw images of the fruit under test in multiple narrowband bands to construct a multiband raw image group; The region segmentation module, connected to the image acquisition module, is used to perform white-dark reference correction and spatial registration on the multi-band original image group to obtain a multi-band reflectance image group in the same spatial coordinate system and perform target segmentation to obtain a complete fruit mask representing the complete main body region of the fruit to be tested. The region extraction module, connected to the region segmentation module, is used to perform boundary shrinking processing on the complete fruit mask to obtain a safe region of interest, and to remove a preset proportion of bright and low-bright pixels within the safe region of interest based on the pixel reflectance distribution to obtain a robust sampling region. The feature extraction module, connected to the region extraction module, is used to extract the median and interquartile range of reflectance of each band, the band ratio feature between any two bands, and the normalized difference index feature based on the robust sampling region, to form a multispectral robust feature vector. The sugar content prediction module, connected to the feature extraction module, is used to standardize the multispectral robust feature vector and then input it into the pre-trained sugar content prediction model to output the sugar content prediction value of the fruit to be tested.

[0026] Compared with existing technologies, the fruit sugar content prediction system based on multispectral images of the present invention has the following advantages: This invention constructs a safe region of interest and removes abnormal bright and dark pixels. Combined with boundary shrinkage processing, it avoids unstable reflective regions, allowing subsequent feature extraction to focus more on stable areas of the fruit body. This effectively avoids interference from fruit edges, background intrusion, peel highlights, and local shadows on spectral features. Furthermore, it extracts median, interquartile range, band ratio features, and normalized difference index features. These features not only reflect the relative spectral responses related to sugar content across different bands but also mitigate the impact of light intensity variations, local noise, and sample pose differences. This helps overcome the shortcomings of traditional mean features, which are easily affected by sampling pose and light fluctuations under limited band and small sample conditions. Finally, by combining standardized multispectral robust feature vectors with a sugar content prediction model, it achieves rapid and accurate prediction of fruit sugar content, significantly improving generalization ability across samples and batches. Attached Figure Description

[0027] Figure 1 This is a flowchart of the method steps of the present invention; Figure 2 This is a schematic diagram of the system structure of the present invention; Figure 3 This is a schematic diagram of the image acquisition module of the present invention. Figure 3 (a) in the diagram represents a schematic diagram of a six-eye multispectral camera. Figure 3 (b) in the diagram represents a schematic diagram of the structure of the six narrowband imaging units in a six-eye multispectral camera; Figure 4This is a schematic diagram illustrating the division of the complete fruit mask, safe region of interest (ROI), and robust sampling region in this invention. The blue line represents the original fruit surface outline, i.e., the complete fruit mask; the green line represents the robust ROI, i.e., the safe region of interest; and the red line represents the filtered effective area, i.e., the robust sampling region. Detailed Implementation

[0028] First, those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0030] See Figure 1 This invention discloses a method for predicting the sugar content of fruits based on multispectral images, comprising the following steps: Step S1: Collect original images of the fruit to be tested in multiple narrowband bands to construct a multiband original image group; Step S2: Perform white-dark reference correction and spatial registration on the multi-band original image group to obtain a multi-band reflectance image group in the same spatial coordinate system and perform target segmentation to obtain a complete fruit mask representing the complete main body area of ​​the fruit to be tested. Step S3: Perform boundary shrinkage processing on the complete fruit mask to obtain a safe region of interest, and remove a preset proportion of bright and low-bright pixels within the safe region of interest according to the pixel reflectance distribution to obtain a robust sampling region; Step S4: Based on the robust sampling region, extract the median and interquartile range of reflectance of each band, the band ratio feature between any two bands, and the normalized difference index feature to form a multispectral robust feature vector. Step S5: After standardizing the multispectral robust feature vector, input it into the pre-trained sugar content prediction model to output the sugar content prediction value of the fruit to be tested.

[0031] See Figure 2 This invention also discloses a fruit sugar content prediction system based on multispectral images, which applies the above-mentioned fruit sugar content prediction method based on multispectral images, including: The image acquisition module is used to acquire raw images of the fruit under test in multiple narrowband bands to construct a multiband raw image group; The region segmentation module, connected to the image acquisition module, is used to perform white-dark reference correction and spatial registration on the multi-band original image group to obtain a multi-band reflectance image group in the same spatial coordinate system and perform target segmentation to obtain a complete fruit mask representing the complete main body region of the fruit to be tested. The region extraction module, connected to the region segmentation module, is used to perform boundary shrinking processing on the complete fruit mask to obtain a safe region of interest, and to remove a preset proportion of bright and low-bright pixels within the safe region of interest based on the pixel reflectance distribution to obtain a robust sampling region. The feature extraction module, connected to the region extraction module, is used to extract the median and interquartile range of reflectance of each band, the band ratio feature between any two bands, and the normalized difference index feature based on the robust sampling region, forming a multispectral robust feature vector. The sugar content prediction module, connected to the feature extraction module, is used to standardize the multispectral robust feature vector and then input it into the pre-trained sugar content prediction model to output the sugar content prediction value of the fruit to be tested.

[0032] In this embodiment of the invention, the image acquisition module can use a six-lens multispectral camera (such as a 6-channel snapshot multispectral camera) or a black-and-white camera with multiple narrowband filters for time-division imaging. The region segmentation module, region extraction module, and feature extraction module can run on an ARM processor on a PC or embedded device. The sugar content prediction module is a pre-trained model file (such as libsvm format). Data is transferred between modules sequentially through memory sharing or file interfaces. The system also integrates an exposure control module for automatically adjusting the exposure time of each band, a light source module (broadband halogen lamp or LED array), and an interactive interface (PC graphical interface or embedded touch screen). The system can generate a sugar content distribution heatmap for model optimization in laboratory mode, and can also achieve rapid detection of single fruit in seconds in field mode, meeting the needs of different application scenarios.

[0033] See Figure 3 In this embodiment of the invention, apples are used as the fruit to be tested, but the method is also applicable to any variety of fruit such as blueberries, pears, and peaches. The multispectral image acquisition module uses a six-lens multispectral camera, which includes six independently imaging narrowband imaging units. Figure 3 Image (a) shows a six-eye multispectral camera. Figure 3(b) shows a narrowband imaging unit. Each narrowband imaging unit corresponds to a narrowband band, such as center wavelengths of 450nm, 550nm, 650nm, 750nm, 850nm and 950nm, which correspond to response regions sensitive to sugar content, such as blue light, green light, red light and near-infrared light. The system maps logical numbers to physical numbers for each narrowband imaging unit and independently adjusts exposure parameters according to the light response intensity of different bands. During acquisition, the fruit to be tested is placed in a dark box with uniform illumination, and each narrowband imaging unit is triggered sequentially. The original image, exposure parameters and image acquisition time of each narrowband band are recorded synchronously. For the same sample of fruit, multiple angle acquisitions are performed according to the left side (angle with the optical axis of about -30°), the middle side (0°) and the right side (+30°). Each side is acquired three times to reduce the error caused by the difference in shooting posture in a single shot.

[0034] In this embodiment of the invention, before performing white and dark reference correction in step S2, dark reference images and white reference images corresponding to each narrowband band are first acquired. The method for acquiring the dark reference image is as follows: turn off all light sources and cover the lens cap, and acquire the dark current and ambient background noise of the six-eye multispectral camera. The method for acquiring the white reference image is as follows: place a standard diffuse reflection white board (such as a polytetrafluoroethylene white board) in the same position and posture as the fruit to be tested, turn on all light sources, and acquire the image as the white reference image.

[0035] In this embodiment of the invention, the specific process of white-dark reference correction in step S2 is as follows: The first... The original image of each band at pixel location The gray value at that location is denoted as , No. Each band of dark reference image at pixel location The gray value at that location is denoted as , No. The white reference image for each band is located at the pixel position. The gray value at that location is denoted as Then the first Each band at pixel position Corrected reflectivity at It is obtained through the following calculation formula: ; in, This represents a preset constant used to prevent the denominator from being zero.

[0036] In this embodiment of the invention, after completing the white and dark reference correction in step S2, the multi-band registration parameters (including rotation, translation, and scaling matrices) obtained in advance through a checkerboard calibration board are used to map the reflectance images of all bands to the same spatial coordinate system through affine transformation or homography transformation, resulting in a group of registered multi-band reflectance images. Subsequently, a registered main band image (such as the 850 nm band, which has strong penetration into fruit pulp tissue and a high signal-to-noise ratio) is selected as input, and a deep learning-based instance segmentation network (such as Mask R-CNN) or a traditional threshold segmentation method (such as OTSU global thresholding combined with morphological opening operation) is used to extract the complete contour of the fruit to be tested, generating a binarized image such as... Figure 4 The blue line indicates the full fruit mask. Within the full fruit mask, a pixel value of 1 represents the fruit area, and 0 represents the background.

[0037] In this embodiment of the invention, in step S3, the boundary shrinkage processing is implemented using morphological erosion. Specifically, a circular or rectangular structural element (e.g., a disk-shaped structural element with a radius of 15 pixels) is defined, and an erosion operation is performed on the complete fruit mask. The number of erosion iterations is set according to the fruit size (usually 3-5 times), causing the boundary of the complete fruit mask to shrink inward, thereby avoiding dark edges, occlusion, and abrupt changes in reflection angle at the fruit edge. Alternatively, boundary distance constraints (retaining areas greater than a certain pixel threshold from the fruit outline) or proportional shrinkage (scaling the complete fruit mask to 80%-90% of its original area) can be used. The area obtained through the above processing is as follows: Figure 4 The safe region of interest (ROI) indicated by the green line mainly covers the flat area in the center of the fruit, effectively avoiding interference from background intrusion, fruit peel highlights, and local shadows. Within the ROI, the average reflectance of each pixel across all wavelengths (or the reflectance value of the main wavelength band) is calculated. All pixels are then sorted from smallest to largest according to this average value. Pixels in the top 5% and bottom 5% after sorting are removed. The remaining pixels constitute the following... Figure 4 The robust sampling region (trimregion) shown by the red line can also be removed according to the reflectance distribution of a single band (such as 550 nm) if the imaging quality is good. The preset ratio can be adjusted in the range of 3% to 10% according to the quality of the sample integrated image. In this embodiment, 5% is preferred, which achieves a good balance between filtering out extreme outliers and retaining effective information.

[0038] In this embodiment of the invention, in step S4, there are M bands (M=6 in this embodiment), and the robust sampling region contains N effective pixels. For the first... For each of the N bands, obtain the set of reflectance values ​​for these N pixels, and then calculate the i-th value using the following formula. Interquartile range of each band : ; in, Indicates the first reflectivity of each band The upper quartile, Indicates the first reflectivity of each band The lower quartiles; For any two of the... The first band and the first Band ratio characteristics between bands It is obtained through the following calculation formula: ; in, Indicates the first The median reflectance of each band within the robust sampling region. Indicates the first The median reflectance of each band within the robust sampling region. Indicates a preset constant; For any two of the... The first band and the first Normalized difference index characteristics between bands It is obtained through the following calculation formula: ; Then, all the single-band medians, single-band interquartile ranges, pairwise combined band ratio features, and normalized difference index features are concatenated into a multispectral robust feature vector. When M=6, the total number of features is 6 + 6 + 15 + 15 = 42 dimensions. This multispectral robust feature vector not only includes the central tendency and dispersion of each band, but also eliminates the influence of global changes in light source intensity and acquisition conditions through ratios and normalized differences.

[0039] In this embodiment of the invention, in step S5, the normalization process employs the Z-score method: for each dimension of the multispectral robust feature vector... Using this feature from the training set mean and standard deviation The standardized features are calculated using the following formula. : ; All standardized feature dimensions have zero mean and unit variance, eliminating the influence of differences in dimensions and numerical ranges on the sugar content prediction model. The sugar content prediction model uses a support vector regression (SVR) model with a radial basis function (RBF) kernel function. Hyperparameters are determined through grid search and five-fold cross-validation, for example, the penalty coefficient C=100 and gamma=0.01. During model training, the measured sugar content of the tested fruit (Brix value, obtained by juicing the tested fruit with a handheld refractometer) is used as the supervision label. In the inference stage, the standardized feature vectors obtained by processing the fruit to be tested in steps S1~S4 are input into the trained support vector regression (SVR) model. The output of the support vector regression (SVR) model is the predicted sugar content value (unit: % Brix) of the fruit to be tested.

[0040] In this embodiment of the invention, the sugar content prediction model in step S5 adopts one of the following: support vector regression, kernel ridge regression, partial least squares regression, and random forest regression. The sugar content prediction model is trained using the measured sugar content values ​​of the tested fruits as supervisory labels. During the model training phase, a training set containing 200-500 tested fruit samples is constructed. For each tested fruit sample, multispectral robust feature vectors are extracted according to steps S1-S4, and the corresponding measured sugar content values ​​from the refractometer are recorded. For the support vector regression model, both the input features and output labels are standardized. For the random forest regression model, the number of trees can be set to 100, and the maximum depth to 10. After the model training is completed, the generalization performance is evaluated using an independent test set (samples not involved in training). The evaluation metrics include the coefficient of determination (COP). The root mean square error (RMSE) and relative analysis error (RPD) of the multispectral robust feature vector extracted in this invention are experimentally verified to achieve high performance on the Apple test set. , The RPD > 2.5, which is significantly better than the traditional method of directly extracting the mean spectrum from the whole fruit area.

[0041] In this embodiment of the invention, when pre-training the sugar content prediction model before executing step S1, for the same sample of tested fruit, multi-angle images are acquired from different sides, and the image acquisition results of each side are averaged at the side level or fused at the fruit level to obtain fusion features. Then, the fusion features are input into the sugar content prediction model for training. Specifically, for each sample of tested fruit in the training set, multispectral images of its left side, middle side, and right side are acquired, and each angle is acquired three times. Steps S1 to S4 are executed for each angle to obtain the multispectral robust feature vector of that angle. Then, the multispectral robust feature vectors of the same sample of tested fruit are averaged dimension by dimension to obtain the fusion feature vector of the sample of tested fruit. At the same time, the measured sugar content of the tested fruit is the Brix value measured after extracting the mixed juice of the whole fruit. By adopting this multi-angle fusion strategy, the feature fluctuations caused by the uneven distribution of sugar content on the fruit surface and the difference in placement posture can be effectively eliminated, making the trained sugar content prediction model more robust.

[0042] In this embodiment of the invention, after performing step S5, the method further includes: When executed on a PC, a corresponding heatmap of sugar content distribution is generated based on the predicted sugar content value, combined with inference methods using local sliding windows or dense image patches; or When executed on the embedded end, the predicted sugar content, maximum sugar content, minimum sugar content, and average sugar content are output in a lightweight result display mode. At the same time, the sample number of the fruit to be tested, the image acquisition time, and the original multi-band image group are saved.

[0043] In this embodiment of the invention, on the PC, to achieve detailed analysis, the registered multi-band reflectance image is divided into several overlapping image blocks (e.g., block size 50×50 pixels, sliding step size 10 pixels). Each image block independently executes steps S3~S5 to obtain the sugar content prediction value of that local area. The sugar content prediction values ​​of all image blocks are stitched together according to their spatial positions, and a sugar content distribution heatmap covering the entire fruit surface is generated using pseudo-color mapping (e.g., red high, blue low). Simultaneously, the average sugar content, maximum sugar content, and minimum sugar content of the entire fruit are calculated. On the embedded end (e.g., a portable detection device based on an ARM processor), due to limited computing power and memory, a sugar content distribution heatmap is not generated. Instead, the overall sugar content prediction value is obtained directly through a one-time inference of the entire fruit and displayed on the OLED screen in a concise format: "Sugar content: 12.3 °Brix, maximum: 13.1, minimum: 11.5, average: 12.3”, and at the same time, the sample number, image acquisition time, image file path of each band, model version and prediction results are saved to the SD card in JSON format to support subsequent data traceability and batch statistics.

[0044] In the description of this invention, the references to "one embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0045] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting fruit sugar content based on multispectral images, characterized in that, Includes the following steps: Step S1: Collect original images of the fruit to be tested in multiple narrowband bands to construct a multiband original image group; Step S2: Perform white-dark reference correction and spatial registration on the multi-band original image group to obtain a multi-band reflectance image group in the same spatial coordinate system and perform target segmentation to obtain a complete fruit mask representing the complete main body area of ​​the fruit to be tested. Step S3: Perform boundary shrinkage processing on the complete fruit mask to obtain a safe region of interest, and remove a preset proportion of bright and low-bright pixels within the safe region of interest according to the pixel reflectance distribution to obtain a robust sampling region; Step S4: Based on the robust sampling region, extract the median and interquartile range of reflectance for each band, the band ratio feature between any two bands, and the normalized difference index feature to form a multispectral robust feature vector. Step S5: After standardizing the multispectral robust feature vector, input it into the pre-trained sugar content prediction model to output the sugar content prediction value of the fruit to be tested.

2. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, Step S2, before performing white-dark reference correction, also includes acquiring dark reference images and white reference images corresponding to each of the narrowband bands. The white-dark reference correction process includes: The first The original image of each band at pixel location The gray value at the position minus the first Each band of dark reference image at pixel location The difference between the gray values ​​at each position is used as the numerator value, and the value of the first position is used as the numerator value. The original image of each band at pixel location The gray value at the position minus the first The white reference image for each band is located at the pixel position. The difference between the gray values ​​at each point is added to a very small constant to obtain the denominator value. Then, the numerator value is divided by the denominator value to obtain the first... Each band at pixel position Corrected reflectivity at that location.

3. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, The boundary shrinkage process in step S3 employs morphological erosion, boundary distance constraint, or proportional shrinkage to ensure that the safe region of interest avoids the unstable reflective areas, background blending areas, highlight areas of the fruit peel, and local shadow areas at the fruit edge.

4. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, The preset ratio in step S3 is 5%. The pixels in the safe region of interest are sorted from high to low brightness, and the bright pixels in the top 5% and the low pixels in the bottom 5% are removed.

5. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, In step S3, when removing a preset proportion of bright pixels and low-brightness pixels according to the pixel reflectance distribution, the bright pixels and low-brightness pixels are sorted according to the single-band reflectance distribution, the multi-band average reflectance distribution, and the main band reflectance distribution, and then the pixels located at both ends after sorting are removed.

6. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, In step S4, the interquartile range is obtained by calculating the difference between the upper and lower quartiles of the reflectance of the corresponding band; the band ratio feature is obtained by calculating the ratio of the median reflectance of any two bands; and the normalized difference index feature is obtained by calculating the ratio of the difference between the median reflectance of any two bands to the sum of the median reflectance.

7. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, The sugar content prediction model in step S5 adopts one of the following: support vector regression model, kernel ridge regression model, partial least squares regression model, and random forest regression model, and the sugar content prediction model is trained using the measured sugar content values ​​of the fruit as supervision labels.

8. The fruit sugar content prediction method based on multispectral images according to claim 7, characterized in that, Before performing step S1, when pre-training the sugar content prediction model, for the same sample of tested fruit, multi-angle image acquisition is performed according to different sides, and the image acquisition results of each side are subjected to side-level averaging or fruit-level fusion processing to obtain fusion features. Then, the fusion features are input into the sugar content prediction model for training.

9. The fruit sugar content prediction method based on multispectral images according to claim 1, characterized in that, After performing step S5, the method further includes: When executed on a PC, a corresponding sugar content heatmap is generated based on the predicted sugar content value and in conjunction with local sliding window or dense image patch inference methods; or When executed on the embedded end, the predicted sugar content, maximum sugar content, minimum sugar content, and average sugar content are output in a lightweight result display mode, while the sample number of the fruit to be tested, the image acquisition time, and the multi-band original image group are saved.

10. A fruit sugar content prediction system based on multispectral images, characterized in that, The method for predicting fruit sugar content based on multispectral images as described in any one of claims 1-9 includes: The image acquisition module is used to acquire raw images of the fruit under test in multiple narrowband bands to construct a multiband raw image group; The region segmentation module, connected to the image acquisition module, is used to perform white-dark reference correction and spatial registration on the multi-band original image group to obtain a multi-band reflectance image group in the same spatial coordinate system and perform target segmentation to obtain a complete fruit mask representing the complete main body region of the fruit to be tested. The region extraction module, connected to the region segmentation module, is used to perform boundary shrinking processing on the complete fruit mask to obtain a safe region of interest, and to remove a preset proportion of bright and low-bright pixels within the safe region of interest based on the pixel reflectance distribution to obtain a robust sampling region. The feature extraction module, connected to the region extraction module, is used to extract the median and interquartile range of reflectance of each band, the band ratio feature between any two bands, and the normalized difference index feature based on the robust sampling region, to form a multispectral robust feature vector. The sugar content prediction module, connected to the feature extraction module, is used to standardize the multispectral robust feature vector and then input it into the pre-trained sugar content prediction model to output the sugar content prediction value of the fruit to be tested.

Images