Citrus sugar content prediction model training method based on lightweight rgb reconstruction hsi

Abstract

The application provides a citrus sugar content prediction model training method based on light-weight RGB reconstruction HSI, comprising: acquiring citrus RGB image sample data and citrus HSI image sample data subjected to systematic preprocessing and data enhancement operation, and acquiring sugar content measured data of the citrus; determining a key prediction sub-spectrum based on the citrus HSI image sample data and the sugar content measured data; constructing an initial citrus sugar content prediction model; training the initial citrus sugar content prediction model based on the key prediction sub-spectrum and the citrus RGB image sample data, and outputting a citrus sugar content prediction model meeting prediction requirements after multiple rounds of training. The method provided in the application significantly surpasses traditional RGB and HSI methods in the case of only using RGB image input, and provides a practical high-precision low-cost scheme for fruit nondestructive sugar content detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent agricultural product quality detection and machine vision technology, and in particular to a citrus sugar content prediction model training method based on light-weight RGB reconstruction HSI. BACKGROUND

[0002] Non-destructive detection of fruit sugar content usually relies on near-infrared or hyperspectral equipment, however, such equipment is often costly and bulky, which is not conducive to flexible deployment on production lines. In contrast, although using only RGB (Red, Green, Blue) images for sugar content regression prediction is less costly, the prediction accuracy is often difficult to meet the requirements due to the limited information dimension. If a regression model is directly trained on a hyperspectral image, although the prediction accuracy is higher, it cannot be used without hyperspectral hardware support during inference, and is accompanied by higher computing and storage costs. SUMMARY

[0003] The present application provides a citrus sugar content prediction model training method based on light-weight RGB reconstruction HSI, in order to solve the above technical problems, the present application adopts the following technical method:

[0004] The present application provides a citrus sugar content prediction model training method based on light-weight RGB reconstruction HSI, comprising:

[0005] Obtain citrus RGB image sample data and citrus HSI image sample data subjected to systematic preprocessing and data enhancement operations, and obtain sugar content measurement data of the citrus;

[0006] Determine a key prediction sub-spectrum based on the citrus HSI image sample data and the sugar content measurement data;

[0007] Construct an initial citrus sugar content prediction model;

[0008] Train the initial citrus sugar content prediction model based on the key prediction sub-spectrum and the citrus RGB image sample data, and output a citrus sugar content prediction model meeting the prediction requirements after multiple rounds of training.

[0009] Optionally, the initial citrus sugar content prediction model comprises a low-resolution pixel score map generation module, a key pixel selection module, a key pixel spectrum reconstruction module and a context-aware regression module connected in sequence; the citrus sugar content prediction model outputs a prediction result of the sugar content of the citrus according to an input citrus RGB image;

[0010] The citrus RGB image sample data is preprocessed, specifically:

[0011] The citrus RGB image sample data is subjected to a geometric cropping operation;

[0012] The citrus HSI image sample data were preprocessed as follows:

[0013] The RGB image of the citrus fruit is subjected to spectral smoothing, geometric cropping, and threshold-based foreground mask generation operations in sequence.

[0014] Optionally, determining the key prediction sub-spectrum based on the citrus HSI image sample data and the measured sugar content data includes:

[0015] By employing cross-sample permutation importance, the key prediction sub-spectrum is determined by calculating the HSI image sample data of the citrus fruit and the measured sugar content data.

[0016] Optionally, the method of employing cross-sample permutation importance to calculate the key prediction sub-spectrum for the citrus HSI image sample data and the measured sugar content data includes:

[0017] Based on the citrus HSI image sample data, a first number of candidate bands for the hyperspectral image of citrus were determined;

[0018] Construct a citrus sugar content estimation model based on convolutional neural networks;

[0019] The citrus HSI image sample data is divided into a citrus HSI image sample verification set, a citrus HSI image sample training set, and a citrus HSI image sample test set according to a preset ratio.

[0020] Using the citrus HSI image sample training set, the citrus sugar content estimation model is trained in multiple rounds to determine the optimal weights after the citrus sugar content estimation model is trained.

[0021] The optimal weights are fixed, and the baseline loss of each first number of candidate bands is calculated on the citrus HSI image sample validation set. After each candidate band is randomly rearranged across samples within a batch, the permutation loss is obtained.

[0022] Based on the difference between the baseline loss and the permutation loss, the candidate bands are arranged in descending order to obtain the priority of the candidate sub-spectrums;

[0023] Based on the priority of the candidate sub-spectrums and the measured sugar content data, the determination coefficient of the sugar content prediction performance of the candidate bands is determined;

[0024] Based on the determination coefficients of the sugar content prediction performance of the candidate bands, a second number of key prediction sub-spectrums are determined, and evaluated and reported on the test set of the citrus HSI image samples.

[0025] Optionally, training the initial citrus sugar content prediction model based on the key prediction sub-spectrum and the citrus RGB image sample data includes:

[0026] Step S301: input the citrus RGB image sample data into the low-resolution pixel score map generation module for generation operation to generate a low-resolution pixel score map;

[0027] Step S302: input the low-resolution pixel score map into the key pixel selection module for selection processing to determine the pixel position of the selected key pixel;

[0028] Step S303: input the pixel position of the selected key pixel into the key pixel spectrum reconstruction module for reconstruction processing to reconstruct the key prediction sub-spectrum at the pixel position;

[0029] Step S304: input the key prediction sub-spectrum into the context-aware regression module for prediction processing to output the sugar content prediction result of the citrus.

[0030] Optionally, the specific operation of step S301 is:

[0031] input the citrus RGB image sample data into a lightweight backbone network to output multi-scale features;

[0032] align and concatenate the multi-scale features in the channel dimension through pooling, and obtain a fused feature map after convolution projection;

[0033] perform convolution processing on the fused feature map to output a low-resolution pixel score map.

[0034] Optionally, the specific operation of step S302 is:

[0035] flatten the low-resolution pixel score map into a pixel importance score vector;

[0036] add independent and identically distributed Gumbel noise to each element of the pixel importance score vector, and scale it with a temperature parameter to obtain a perturbed score of each element;

[0037] sort the perturbed score of each element in descending order to determine the pixel position of the selected key pixel.

[0038] Optionally, the specific operation of step S303 is:

[0039] obtain the RGB pixel value of the citrus RGB image sample data at the pixel position, the local feature vector, and the global context vector and band prior input of the entire image of the citrus RGB image sample data;

[0040] Based on the RGB pixel value, the local feature vector, the global context vector and the waveband prior input, an input vector of a spectral reconstruction network is constructed at the pixel position;

[0041] The input vector is input into the spectral reconstruction network to reconstruct a key predicted subspectrum at the pixel position; the spectral reconstruction network includes two layers of perceptron, which are two lightweight MLPs with residual feedforward structure.

[0042] Optionally, the specific operation of step S304 is:

[0043] The local feature vector, the key predicted subspectrum and the RGB pixel value are spliced in the channel dimension to obtain a representation of each key pixel;

[0044] The representation of each key pixel is processed by a channel-by-channel affine transformation to obtain a conditioned representation of all key pixels;

[0045] An aggregation weight of each key pixel is obtained;

[0046] The aggregation weight and the conditioned representation of all key pixels are non-uniformly aggregated to determine a global representation;

[0047] The global representation is input into a lightweight regression head to output a sugar content prediction result of the citrus fruit.

[0048] Optionally, the total loss objective function of the initial citrus sugar content prediction model includes selection dispersion constraint, spectral reconstruction consistency, boundary / background suppression and regression head supervision.

[0049] The present application has the following beneficial effects:

[0050] The method provided by the present application takes the task-driven key predicted subspectrum and key pixel as the core, and constructs an extremely simple, interpretable and deployable end initial citrus sugar content prediction model; in the case of only using RGB image input, the traditional RGB and HSI method are significantly surpassed, and a practical high-precision and low-cost solution for fruit nondestructive sugar content detection is provided. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A flowchart of a citrus sugar content prediction model training method based on lightweight RGB reconstruction HSI provided by an embodiment of the present application is shown;

[0052] Figure 2 A structure diagram of an initial citrus sugar content prediction model provided by an embodiment of the present application is shown;

[0053] Figure 3 A flowchart of training an initial citrus sugar content prediction model provided by an embodiment of the present application is shown;

[0054] Figure 4 The method provided by the embodiment of the present application provides mask prediction and key pixel selection results (1024 points) of the citrus sample; sugar degree prediction (left), mask visualization (middle), and Top-k selection point diagram.

[0055] Explanation of reference signs

[0056] The initial citrus sugar degree prediction model 200, the low-resolution pixel score map generation module 210, the key pixel selection module 220, the key pixel spectrum reconstruction module 230, and the context-aware regression module 240. DETAILED DESCRIPTION

[0057] For the convenience of those skilled in the art, the present application will be further described below in combination with embodiments and drawings, and the content mentioned in the embodiments is not a limitation on the present application.

[0058] To solve the above technical problems, as shown in the Figure 1 The present application proposes a citrus sugar degree prediction model training method based on light-weight RGB reconstruction HSI, which comprises:

[0059] Step S101: Obtain the citrus RGB image sample data and the citrus HSI image sample data subjected to systematic preprocessing and data enhancement operations, and obtain the sugar degree measured data of the citrus;

[0060] Before the citrus RGB image sample data and the citrus HSI image sample data are subjected to systematic preprocessing and data enhancement operations, the citrus RGB image sample data and the citrus HSI image sample data need to be collected first, and the sugar degree measured data of the citrus is obtained. The detailed process of collection and the detailed process of obtaining are as follows:

[0061] An RGB camera is used to realize the collection of the citrus RGB image sample data, and a hyperspectral camera (HSI, Hyperspectral Image) is used to realize the collection of the citrus HSI image sample data. To ensure the uniformity and comparability of the data, the hyperspectral camera (HSI) and the RGB industrial camera are rigidly fixed on the same platform, so that the relative poses of the two are kept constant. In the collection environment, a black light-absorbing cloth is preferably used as the background, and a halogen lamp array is used as the light source to provide continuous spectrum covering the visible light to near-infrared band, thereby effectively suppressing stray reflection and improving the stability of the imaging domain distribution. During the collection process, the triggering mode of the RGB and HSI is set to be synchronous or near-synchronous to ensure the consistency of the posture of the same fruit body when imaging, thereby providing a reliable basis for subsequent data calibration and spatial alignment.

[0062] After the RGB and HSI image sample data are collected, the spatial consistency of the two imaging results needs to be ensured. First, the internal and distortion correction of the two cameras is performed respectively to eliminate the geometric deviation caused by lens distortion. Then, the chessboard is used as a calibration target to establish the external parameter and solve the plane homography mapping relationship. Let the homography matrices of RGB and HSI on the chessboard plane be and respectively, then the pixel points of the two imaging satisfy the following mutual mapping relationship:

[0063] (1)

[0064] wherein, is the homogeneous coordinate, and the symbol "~" represents the similar relationship between the two with a scale factor. Based on the transformation, each HSI cube can be mapped to the pixel grid of the RGB image through grid resampling, thereby obtaining the corresponding pixel-level registration of the paired data.

[0065] To improve the stability of registration and prevent drift between batches, the chessboard needs to be repeatedly photographed multiple times during batch data collection, and the obtained homography matrix is reviewed and averaged to correct, thereby ensuring the robustness of cross-modal alignment. After the above steps, the RGB and HSI images are uniformly mapped in the same spatial domain, providing an accurate data alignment basis for subsequent sugar modeling.

[0066] After the spatial registration is completed, the collected data needs to be corrected in the spectral and radiation aspects to ensure the consistency and comparability of the cross-modal features. For the HSI channel, the dark current is first deducted to eliminate the detector background noise, and then the standard white board is used for normalization, thereby obtaining the spectral stable and physically meaningful reflectance cube . For the RGB channel, linearization processing needs to be performed on the original image, and it is uniformly normalized to the [0, 1] interval, so as to maintain the numerical scale consistency with the HSI output.

[0067] To further improve the robustness and generalization ability of the model, the normalized citrus RGB image sample data and citrus HSI image sample data need to be systematically preprocessed and data enhanced. The preprocessing of the citrus RGB image sample data only includes geometric cropping, and the preprocessing of the citrus HSI image sample data includes spectral smoothing, geometric cropping and threshold-based foreground mask generation operations in sequence. The data enhancement operations of the citrus RGB image sample data and the citrus HSI image sample data include horizontal or vertical flipping, 90° multiple rotation and random cropping.

[0068] For the spatially registered citrus HSI image sample data, Savitzky-Golay (SG) filtering is performed on the spectral dimension to reduce the sensor noise and preserve the local differential features of the spectral curve. The filtering window width is adaptively set to be no more than the maximum odd number of the band number, and the filtering polynomial order is no more than the window width minus one, as follows:

[0069] (2)

[0070] wherein, denotes the smoothed signal value at the time (or spectral) position , is the original signal value of the th sampling point in the window centered at , is the th order coefficient of the polynomial fitting, denotes the relative position index of the sampling point in the window, and the value range is , where is the width of the filtering window, and is the order of the fitting polynomial, which is usually less than the maximum odd number of the window width. The formula obtains the smoothed signal value by fitting a order polynomial in the local window with the least squares method, so as to suppress the noise while preserving the local features of the spectral curve. The smoothed spectrum not only suppresses the noise, but also retains the sugar-related spectral details. Subsequently, in the spatial dimension, the citrus RGB image sample data and the citrus HSI image sample data are center-cropped to pixel size to retain the fruit body, and then subjected to geometric enhancement operation (data enhancement operation), including horizontal or vertical flip, 90° multiple rotation and random cropping, to expand the data diversity. In the model training stage, low-amplitude brightness and contrast disturbances, as well as light Gaussian blur and Gaussian noise, can be superimposed to simulate the light and sensor fluctuations in the real application environment. To avoid information leakage in the verification and test process, only center cropping is performed on them without random enhancement.

[0071] The threshold-based foreground mask generation operation is performed on the data-enhanced citrus HSI image sample data, as follows:

[0072] To highlight the fruit body region and suppress irrelevant background interference, the foreground mask is automatically generated based on the maximum response value of HSI in the full band, which is defined as follows:

[0073] (3)

[0074]

[0075] ​​​In the formula, represents the pixel position The pixel intensity under the spectral channel is used to characterize the reflectivity of the waveband; represents the maximum value of the pixel in all spectral channels; threshold is the decision threshold generated for the mask, which can be determined according to the pixel value distribution or artificial experience, and is used to distinguish the foreground and the background; represents the mask label of the pixel position , which is 1 when the position satisfies the condition , and 0 otherwise. This method adaptively generates a binary foreground mask by comparing the maximum spectral response value of the pixel with the threshold value, and realizes automatic extraction of the fruit body region and background suppression.

[0076] The obtained binary mask is further subjected to an opening operation with an elliptical structural element to remove isolated noise points and smooth the boundary, so as to obtain a coherent mask of the fruit body region. This mask is used as a supervision signal to constrain the learning of key pixels in the training, which significantly reduces the interference of the background on feature extraction and regression prediction.

[0077] Through the combination of spectral smoothing, geometric enhancement and foreground mask, the entire data preprocessing link not only guarantees the fidelity of the spectrum and the geometric structure, but also improves the robustness of the model to complex environments through data enhancement and background suppression.

[0078] After the collection of the citrus RGB image sample data and the citrus HSI image sample data is completed, the sugar content of the citrus sample corresponding to the citrus RGB image sample data and the citrus HSI image sample data is determined, so as to obtain the real label required for modeling. The specific steps are as follows: first, peel the fruit, squeeze the pulp and remove the dregs through a filter to obtain clear juice. Then, use the ATAGO PAL-BX / ACID1 citrus sugar acid all-in-one machine to detect the juice, and measure the sugar content of each sample 3 times and take the average value. The measured sugar content is recorded in percentage form, and a one-to-one correspondence is established between the sample number NO and the corresponding RGB image sample data and HSI image sample data, so as to obtain the measured data of the sugar content of the citrus.

[0079] Step S102: determining a key prediction sub-spectrum based on the citrus HSI image sample data and the measured sugar content data;

[0080] In order to reduce the calculation and quickly predict the sugar content of citrus according to the spectrum, the application selects a certain number of key predictor spectra on the hyperspectral data of the citrus, which has more accurate prediction results and higher weight than other unselected sub-spectra. The main step is to calculate the citrus HSI image sample data and the measured sugar content data by using the cross-sample permutation property to determine the key predictor spectrum. The detailed process of calculation is as follows:

[0081] First, analyze the citrus HSI image sample data, that is, obtain the first number of candidate wavebands of the citrus hyperspectrum. Generally, the first number of candidate wavebands is 100-200. Then, a citrus sugar content estimation model based on a convolutional neural network is constructed on the first number of candidate hyperspectral numbers. The citrus sugar content estimation model is a regression model for estimating the sugar content value. The training target of the model is to minimize the mean square error (MSE) loss function, which is defined as follows:

[0082] (4)

[0083] Wherein, is the number of training samples, represents the predicted sugar content of the model for the th citrus HSI image sample data, is the corresponding measured sugar content (sugar content measured data). By minimizing the mean square error between the predicted value and the true value, the model can improve the regression accuracy of the sugar content as a whole.

[0084] After the above citrus sugar content estimation model is constructed, the citrus HSI image sample data is divided into a citrus HSI image sample verification set, a citrus HSI image sample training set, and a citrus HSI image sample test set according to a predetermined proportion. The citrus HSI image sample training set is used to train the citrus sugar content estimation model for multiple rounds. Each round of training updates the model parameters on the training set and evaluates the prediction performance on the verification set. According to the early stopping mechanism, the model weight with the best performance on the verification set is saved as the optimal weight of the trained citrus sugar content estimation model. Then, the optimal weight is fixed. To measure the relative contribution of each waveband to the sugar content regression, the cross-sample permutation importance is used, which is specifically:

[0085] On the citrus HSI image sample verification set, the baseline loss of each first number of candidate wavebands is calculated (without disturbing the input), and then each candidate waveband is randomly rearranged in the batch in the verification set (only the corresponding relationship between the wavebands in the samples is disturbed, and the remaining wavebands remain unchanged), and the permutation loss is obtained. The difference between the baseline loss and the permutation loss is defined as the importance ,as follows:

[0086] (5)

[0087] The larger the value, the greater the performance degradation caused by disrupting that band, meaning it carries more effective information. According to the definition of importance, that is... Descending order yields the sequence This is used as a priority for subsequent candidate sub-spectrums. This analysis reveals the independent contribution of each band to the model output, but it has not yet characterized the cumulative effect of multiple bands working together.

[0088] After obtaining the importance ranking, the first... Each band constitutes a subspectrum The coefficient of determination of its sugar content prediction performance was evaluated on the validation set. :

[0089] (6)

[0090] in, To predict sugar content, This is the actual measured sugar content (actual sugar content data). This is the measured mean. To characterize the marginal return of a newly added band, we define:

[0091] (7)

[0092] That is, the first Each band is relative to the previous one Improvements in prediction performance across all bands. The increase, The value usually decreases gradually, indicating that the marginal contribution of the newly added bands is approaching saturation.

[0093] That is, continuous The marginal gain is below the threshold. This yields the second number of key prediction sub-spectrums, which is generally obtained in this application. Then only Subsequent spectrum reconstruction and glycosylation regression were performed, followed by a final evaluation and report on the test set. (Before accumulation here) By analyzing the performance gains across different wavebands and summing them up, we can obtain the cumulative returns from multiple variables.

[0094] Step S103: Construct an initial citrus sugar content prediction model;

[0095] After the determination of the key predictor spectrum described above, an initial citrus sugar content prediction model is constructed for selecting key pixels and reconstructing their corresponding sub-spectrum under the premise of taking only RGB images as input, and then combining context information for sugar content regression. This design ensures that the inference stage is completely free of HSI dependence and directly outputs sugar content prediction values. As shown in Figure 2 The initial citrus sugar content prediction model 200 in this step includes, in sequence, a low-resolution pixel score map generation module 210, a key pixel selection module 220, a key pixel spectrum reconstruction module 230, and a context-aware regression module 240.

[0096] Step S104: Based on the key predictor spectrum and the citrus RGB image sample data, the initial citrus sugar content prediction model is trained, and after multiple rounds of training, a citrus sugar content prediction model that meets the prediction requirements is output.

[0097] The key predictor spectrum and the citrus RGB image sample data are used to train the initial citrus sugar content prediction model, and after multiple rounds of training, a citrus sugar content prediction model that meets the prediction requirements is output, so that the final confirmed citrus sugar content prediction model outputs the prediction result of the citrus sugar content according to the input citrus RGB image. In combination with the four modules contained in step 103, as shown in Figure 3 The training process is described in detail as follows:

[0098] Step S301: The citrus RGB image sample data is input into the low-resolution pixel score map generation module for generation operation to generate a low-resolution pixel score map.

[0099] The low-resolution pixel score map generation module and the key pixel selection module jointly undertake the core task of "locating representative pixels in the RGB image". Pixel importance estimation and selection are completed in a low-resolution feature space with smaller computational load and stronger semantic aggregation, and these low-resolution coordinates are mapped back to the original pixel positions to extract corresponding RGB, realize input preparation for spectrum reconstruction and sugar content regression.

[0100] The processing process of inputting the citrus RGB image sample data into the low-resolution pixel score map generation module to generate a low-resolution pixel score map is as follows:

[0101] The citrus RGB image sample data is first input into a lightweight backbone network (preferably ConvNeXt-Tiny). The output of different layers of the backbone contains multi-scale features from shallow texture to deep semantics for unified resolution and reduced computational burden. The features of each layer are aligned to the same spatial scale through pooling and concatenated in the channel dimension, and then Convolutional projection yields fused feature maps :

[0102] (8)

[0103] In the formula, This refers to the low-resolution size of the backbone network output.

[0104] Subsequently, the feature maps will be merged through independent... The convolution head performs convolution processing and outputs a low-resolution pixel score map. :

[0105] (9)

[0106] in, This represents the importance score of the corresponding region in the sugar content prediction task. The reason for choosing to calculate the score in low-resolution space instead of directly on the original image is that low-resolution features have a larger receptive field and more stable semantic aggregation effects, while significantly reducing... The complexity of sorting and feature sampling. Each low-resolution grid cell corresponds to a local region on the original image. In subsequent steps, these selected grid cells are precisely mapped to the center pixel of the original image for high-resolution feature extraction.

[0107] Step S302: Input the low-resolution pixel score map into the key pixel selection module for selection processing to determine the pixel position of the key pixel to be selected;

[0108] This step requires starting from... Selected from The most representative low-resolution pixel indexes are mapped onto the original image and used for subsequent reconstruction. Due to the direct... Since the operation is discrete and non-differentiable, backpropagation is not possible. Therefore, an approximation method combining Gumbel-Softmax and Straight-Through Estimator (STE) is used to make the point selection process differentiable during training.

[0109] First, the low-resolution pixel score map (size is) Flatten into pixel importance score vectors Each element Indicates the first The importance score of each pixel. To introduce randomness and exploration, independent and identically distributed Gumbel noise is added to each element. and using temperature parameters Scaling is performed to obtain the perturbation score U for each element:

[0110] (10)

[0111] where, each component of , and The introduction of noise can increase exploration at the beginning of training, avoiding the model falling into local optimum.

[0112] The perturbation scores of each element are sorted in descending order, and then the top key pixels are selected, and the pixel positions of the selected key pixels are determined. This step is only used in forward propagation. According to the index of the top maxima, a mask is formed to realize hard selection; in backpropagation, in order to enable the operation to pass the gradient, the STE mechanism is adopted to approximate the gradient back according to the smooth distribution of . The Softmax function is used to process to obtain a smooth selection probability distribution:

[0113]

[0114] (11)

[0115] where, and . represents the probability of the th pixel being selected. The temperature controls the "sharpness" of the distribution: a larger makes the distribution smoother, which is beneficial to early exploration; a smaller makes the distribution more concentrated, which is beneficial to late convergence. The annealing strategy is adopted to gradually reduce during training, balancing search and stability.

[0116] Synthesizing the two, an equivalent differentiable output is constructed:

[0117] (12)

[0118] where, represents the mask of the actual selected pixel positions in the forward stage; represents the smooth selection probability; represents the stop gradient in backpropagation; is the "approximately differentiable selection weight" combining hard selection and soft distribution, which is equivalent to ​​In backpropagation, gradient approximation is performed by distributed backpropagation. That is a bridge that can both "hard select points" and "soft transmit gradients", so that the point selection operation can be embedded in end-to-end training.

[0119] Finally, the selected low-resolution index is mapped to the original image pixel coordinates by a mapping function, and the pixel position of the selected key pixel is uniformly denoted as :

[0120] (13)

[0121] This design not only significantly improves the computational efficiency and training stability, but also enables the model to adaptively learn which spatial region best represents the true spectral and sugar information, while maintaining physical interpretability and considering end-to-end differentiability.

[0122] Step S303: input the pixel position of the selected key pixel into the key pixel spectrum reconstruction module for reconstruction processing to reconstruct the key predicted sub-spectrum at the pixel position of the key pixel;

[0123] For each selected pixel position , an input vector of the spectrum reconstruction network is constructed , and its corresponding key sub-spectrum is output (where is the number of wave bands determined in the preceding step to determine the key predicted sub-spectrum). The input is spliced from four types of information:

[0124] (14)

[0125] In the formula, is the RGB pixel of the original image at position . This is the only observation directly from the input citrus RGB image sample data, providing reflectance strength and color clues. is the local feature vector of the grid element corresponding to on the low-resolution feature map . It contains the texture, shape and local context information aggregated by the backbone network in that area. : the global context vector of the entire image of the citrus RGB image sample data. It is obtained by global average pooling of , representing global factors such as overall brightness distribution, maturity trend, imaging conditions, etc., for applying global priors in pixel-level reconstruction. is the wave band prior embedding. For the key predicted sub-spectrum The output is better corresponding to the selected physical waveband by obtaining the priori injection reconstruction of "which group of waveband is to be reconstructed" through a learnable linear mapping.

[0126] The constructed input vector is input into a spectral reconstruction network composed of two layers of perception machines to reconstruct the key prediction sub-spectrum at the pixel position. The perception machine is a lightweight MLP with a residual feedforward structure, which can complete the fusion and mapping of multi-modal features at low computational cost. The reconstruction spectrum network outputs the corresponding dimensional key prediction sub-spectrum of the pixel at one time:

[0127] (15)

[0128] Through this process, the model realizes the continuous mapping from the observable color , local feature , global scene constraint and waveband physical priori to the key prediction sub-spectrum . This fusion design semantically enables the network to have both local recognition ability and global consistency constraint, thereby realizing the feature alignment that conforms to the physical constraint and maintains the perceptual consistency in spectral reconstruction.

[0129] Step S304: inputting the key prediction sub-spectrum into the context perception regression module for prediction processing, and outputting the sugar content prediction result of the citrus.

[0130] After the reconstruction of the key prediction sub-spectrum of the key pixel is completed, the local feature, the global context and the reconstructed spectrum are further fused to obtain the final sugar content estimation. The context perception regression module in this step adopts a lightweight context perception regression structure, and the core idea is to use FiLM (Feature-wise Linear Modulation) to conditionally modulate the local representation of each key pixel by the global context, then use weighted aggregation to form a global representation, and finally output the variable prediction (i.e. the citrus sugar content prediction value) through the regression head. The specific process is as follows:

[0131] First, for each key pixel , the key prediction sub-spectrum , local feature and pixel RGB value are spliced in the channel dimension to obtain the representation of each key pixel:

[0132] (16)

[0133] with global context vector generate per-channel scaling and shifting parameters :

[0134] (17)

[0135] apply FiLM modulation (per-channel affine transformation) to the representation of each key pixel conditioned on the global context vector :

[0136] (18)

[0137] where, is per-channel multiplication. Through the above processing, the context of the whole image dynamically acts on the point-level representation in a conditioned manner, improving the generalization across samples and across shooting conditions.

[0138] According to formula (11), the probability of each key pixel being selected can be calculated , the probability components corresponding to the top selected pixels are renormalized to obtain the aggregated weights of the key pixels :

[0139] (19)

[0140] therefore, represents the relative confidence weight within the selected pixel set, satisfying . These weights are reused in the regression stage to guide the global feature aggregation, so that pixels with higher scores contribute more in the final prediction, thereby maintaining the weight consistency from pixel selection to regression fusion.

[0141] non-uniformly aggregate the aggregated weights and the conditioned representations of all key pixels to obtain the global representation :

[0142] (20)

[0143] This aggregation represents that high-weight pixels contribute more, both preserving the differences between key pixels and avoiding over-amplification of local features.

[0144] input the aggregated global representation into a lightweight regression head (two-layer fully connected, with normalization and nonlinearity) to output the sugar prediction result of the citrus:

[0145] (21)

[0146] The regression head can be optionally combined with to enhance the expression of nonlinear reflection relationship and maintain stable training. This module realizes both the contribution of global context constraints on each key pixel and the selection of weights

[0147] to explicitly pass the importance of selected points to the final prediction, ensuring a short and stable inference link.

[0148] In addition, to ensure the spatial rationality of pixel selection, the numerical and morphological consistency of spectral reconstruction, the inhibitory effect of boundaries / background on selection, and the accuracy of the final sugar prediction, the total loss objective function of the initial citrus sugar prediction model is defined by four parts: selection dispersion constraint , spectral reconstruction consistency , boundary / background inhibition and regression supervision .

[0149] Selection dispersion constraint is to backfill and upsample the full-image probability map on the low-resolution image . A power penalty is applied on the average pooling operator with a window size of :

[0150] (22)

[0151] This constraint inhibits the excessive concentration of probability in local areas and encourages key pixels to cover the fruit surface uniformly, improving the representativeness of spatial distribution.

[0152] Spectral reconstruction consistency is to constrain each selected pixel , let be the reconstructed key subspectrum, and be the corresponding HSI true value subspectrum. Jointly use intensity-consistent MSE and morphological-consistent SAM (spectral angle) loss:

[0153] (23)

[0154] MSE ensures overall intensity consistency, and SAM measures the directional consistency of spectral morphology, combining both to constrain numerical accuracy and maintain the consistency of spectral physical structure.

[0155] Boundary / background selection inhibition is: To reduce the excessive concentration of key pixels on the edge of the fruit or in the background area, introduce an inhibition term based on the foreground mask obtained in the data set preprocessing stage. The mask​​​ is static binary, whose sigmoid smoothing is . The gradient magnitude approximation is computed as

[0156] (24)

[0157] where is the Sobel gradient operator, and the result is normalized to . Combining the boundary band with the foreground confidence , the probability map is imposed with a suppression:

[0158] (25)

[0159] The first term penalizes "huddling edges"; the second term suppresses "falling into background". The two terms are controlled by weights , which represent the average over spatial locations. This static mask is involved in training as a data-level geometric prior, stabilizing the suppression of boundary drift and background interference.

[0160] The regression head is supervised by: The regression head outputs the final brix estimation , which corresponds to the measured brix . The overall regression accuracy is supervised by mean squared error:

[0161] (26)

[0162] This term ensures that the end-to-end output is numerically consistent with the true brix.

[0163] Combining all the above terms, the total loss objective function L of the initial citrus brix prediction model can be obtained:

[0164] (27)

[0165] The weight coefficients are determined by tuning on the validation set, balancing local spectral reconstruction, spatial selection constraint, boundary suppression, and final regression accuracy. This joint objective guarantees that the network learns the comprehensive ability of reasonable spatial distribution, accurate spectral reconstruction, robust geometric suppression, and precise brix estimation simultaneously in training.

[0166] ​​In the training of the initial citrus sugar content prediction model, the citrus RGB image sample data is divided into training / validation / test sets in the ratio of 8:1:1, and the global random seed (including the Generator used by PyTorch and Gumbel sampling) is fixed to ensure reproducibility. The input resolution is 448x448, and the RGB is standardized by ImageNet mean and variance; random horizontal flipping and random cropping (without changing the fruit visibility) are used for data augmentation.

[0167] Optimization uses AdamW (betas=(0.9, 0.999), weight decay=5e-4), batch size=32; initial learning rate 1e-4, using Cosine annealing and 5% linear warm-up. Enable AMP mixed precision (dynamic loss scaling) and gradient clipping (global norm 1.0). To stabilize convergence, the backbone is frozen for 5 epochs in the early training stage, and then unfreezes layer by layer for 10 epochs and fine-tunes with layer-wise LR decay=0.75.

[0168] The temperature parameter of Gumbel-Softmax is exponentially annealed:

[0169] (28)

[0170] where , is the epoch. The maximum number of candidates K=1024 (consistent with training and inference, can be adjusted within according to computing power).

[0171] Evaluation indicators: for regression ; for reconstruction , combined with selected point heat map and boundary superposition visualization. Early stopping (patience=10) and validation set optimal model selection; the final result is reported on the test set once.

[0172] Simulation analysis

[0173] To verify the effectiveness of the method of the application, it is compared with existing typical spectral reconstruction and prediction models. In the experimental setup, in addition to "ours" (the method of the application), "RGB" (directly using three-channel RGB prediction), "select_bands" (directly inputting 32 key bands for prediction), and "176bands" (inputting complete 176-band spectrum for prediction), the remaining comparison models all use a two-stage process of "first reconstructing the spectrum, and then predicting the sugar content based on the reconstructed spectrum".

[0174] From Table 1, it can be seen that the parameter scale of the traditional two-stage method is generally large, for example, the parameter amount of HINet and HRNet reaches 5.86M and 31.99M respectively, but the test set R2 is only in the interval of 0.705~0.711; even in the relatively optimal MST++ and Hdnet model, R2 is only about 0.721. In contrast, our method requires only 0.63M parameters, and achieves an R2 of 0.796 on the test set, which is superior to existing methods in both accuracy and efficiency.

[0175] At the same time, from the comparison results of different input modes, it can be seen that when directly using RGB three channels for prediction, the test set R2 is only 0.645, which shows that relying only on limited three-channel information will lead to significant underfitting; when directly inputting 32 key wavebands for prediction, R2 is improved to 0.726, verifying that the key wavebands indeed contain effective representation information; further inputting the complete 176-band spectrum, R2 reaches 0.738, although it performs well in accuracy, but its input dimension and computational overhead are significantly increased. In contrast, our method obtains an R2 of 0.796 with only 0.63M parameters, which is superior to the prediction effect of complete spectral input and significantly reduces the model complexity, thereby fully proving its technical advantages of "low dimension, high efficiency, accuracy and deployability".

[0176] As can be seen from the comparison results in Table 1, the proposed method has a significantly lower parameter amount than the traditional full-spectrum reconstruction-prediction paradigm, while still achieving better prediction accuracy than the complete spectral input (176 bands), fully verifying its technical advantages in "low dimension, high efficiency, accuracy and deployability". Compared with other complex models based on spectral reconstruction, the proposed method not only reduces the computational overhead and storage burden, but also maintains a leading level in R2 index, proving the effectiveness of the proposed end-to-end pixel spectrum reconstruction and context regression framework.

[0177] Table 1 Comparison of complexity and performance of existing methods and the proposed method

[0178]

[0179] In order to further intuitively show the effect of the method, Figure 4 The visualization results on real citrus samples are given. As can be seen from the figure, the proposed method can accurately predict the fruit area mask under the condition of only inputting the RGB image, and automatically select key pixel points uniformly distributed on the fruit surface. These key pixel points effectively avoid the situation of concentrating on the fruit edge or background area, thereby ensuring the stability and representativeness of spectral reconstruction and sugar content estimation. At the same time, the predicted sugar content value in the figure is highly consistent with the true measured value, which again verifies the reliability and generalizability of the proposed method in practical applications.

[0180] In summary, the method disclosed in the application takes the task-driven key predictor spectrum and key pixels as the core, constructs an extremely simple, interpretable and deployable end initial citrus sugar content prediction model, and realizes the significant outperformance of the traditional RGB and HSI method only by using the RGB image input, thereby providing a practical high-precision and low-cost solution for fruit nondestructive sugar content detection.

[0181] The above embodiments are the preferred implementation schemes of the application, and in addition thereto, the application can also be implemented in other manners, and any obvious replacement without departing from the technical scheme concept of the application is within the protection scope of the application.

[0182] In order for those skilled in the art to more conveniently understand the improvements of the application over the prior art, some drawings and descriptions of the application have been simplified, and some other elements have also been omitted from the application file for the sake of clarity, and those skilled in the art should realize that these omitted elements can also constitute the content of the application.

Claims

1. A method for training a citrus sugar content prediction model based on lightweight RGB reconstructed HSI, characterized in that, The method comprises the following steps: obtain citrus RGB image sample data and citrus HSI image sample data subjected to systematic preprocessing and data enhancement operations, and obtain measured data of sugar content of the citrus; determine key predictor spectra based on the citrus HSI image sample data and the measured data of sugar content; construct an initial citrus sugar content prediction model; the initial citrus sugar content prediction model comprises a low-resolution pixel score map generation module, a key pixel selection module, a key pixel spectrum reconstruction module, and a context-aware regression module connected in sequence; train the initial citrus sugar content prediction model based on the key predictor spectra and the citrus RGB image sample data, and output a citrus sugar content prediction model meeting prediction requirements after multiple rounds of training; the method comprises the following steps: calculate the citrus HSI image sample data and the measured data of sugar content using cross-sample permutation importance to determine key predictor spectra; the method comprises the following steps: determine a first number of candidate wavebands of the citrus hyperspectrum based on the citrus HSI image sample data; construct a citrus sugar content estimation model based on a convolutional neural network; divide the citrus HSI image sample data into a citrus HSI image sample validation set, a citrus HSI image sample training set, and a citrus HSI image sample test set according to a preset ratio; determine the optimal weight of the citrus sugar content estimation model after training by using the citrus HSI image sample training set to train the citrus sugar content estimation model multiple times; fix the optimal weight, calculate the baseline loss of each first number of candidate wavebands on the citrus HSI image sample validation set, and obtain permutation loss after random rearrangement of each candidate waveband within a batch; arrange the candidate wavebands in descending order based on the difference between the baseline loss and the permutation loss to obtain the priority of the candidate spectra; determine the determination coefficient of the candidate waveband sugar content prediction performance based on the priority of the candidate spectra and the measured data of sugar content; determine a second number of key predictor spectra based on the determination coefficient of the candidate waveband sugar content prediction performance, and evaluate and report on the citrus HSI image sample test set; the method comprises the following steps: step S301: input the citrus RGB image sample data into the low-resolution pixel score map generation module to generate a low-resolution pixel score map; step S302: input the low-resolution pixel score map into the key pixel selection module to determine the pixel position of the selected key pixel; step S303: input the pixel position of the selected key pixel into the key pixel spectrum reconstruction module to reconstruct the key predictor spectra at the pixel position; Step S304: inputting the key predictor spectrum into the context-aware regression module for prediction processing, and outputting a sugar content prediction result of the citrus.

2. The method of claim 1, wherein, The citrus sugar content prediction model outputs a prediction result of the sugar content of the citrus according to an inputted RGB image of the citrus. The pre-processing of the citrus RGB image sample data comprises the following steps: The citrus RGB image sample data is subjected to a geometric cropping operation. The pre-processing of the citrus HSI image sample data comprises the following steps: The citrus RGB image is subjected to a spectral smoothing operation, a geometric cropping operation and a threshold-based foreground mask generation operation in sequence.

3. The method of claim 1, wherein, The specific operation of step S301 comprises the following steps: The citrus RGB image sample data is inputted into a lightweight backbone network, and multi-scale features are outputted. The multi-scale features are subjected to a pooling alignment and a channel dimension splicing, and a fusion feature map is obtained after convolution projection. The fusion feature map is subjected to convolution processing, and a low-resolution pixel score map is outputted.

4. The method of claim 3, wherein, The specific operation of step S302 comprises the following steps: The low-resolution pixel score map is flattened into a pixel importance score vector. adding to each element of the pixel importance score vector an independent identically distributed Gumbel noise, scaled by a temperature parameter, resulting in a perturbed score for each element; The perturbation scores of each element are sorted in descending order, and the pixel positions of the selected key pixels are determined.

5. The method of claim 4, wherein, The specific operation of step S303 comprises the following steps: The RGB pixel value, the local feature vector, the global context vector and the waveband prior input of the citrus RGB image sample data at the pixel positions are obtained. Based on the RGB pixel value, the local feature vector, the global context vector and the waveband prior input, an input vector of the spectral reconstruction network is constructed at the pixel positions. inputting the input vector to the spectral reconstruction network to reconstruct a key prediction sub-spectrum at the pixel location, the spectral reconstruction network comprising two layers of perceptron, the two layers of perceptron being two lightweight residual feed-forward structures MLP .

6. The method of claim 5, wherein, The specific operation of step S304 comprises the following steps: The local feature vector, the key predictor spectrum and the RGB pixel value are spliced in the channel dimension, and the representation of each key pixel is obtained. The representation of each key pixel is subjected to a channel-by-channel affine transformation processing, and the conditioned representation of all key pixels is obtained. The aggregated weight of each key pixel is obtained. The aggregated weight and the conditioned representation of all key pixels are subjected to non-uniform aggregation, and a global representation is determined. The global representation is inputted into a lightweight regression head, and a sugar content prediction result of the citrus is outputted.

7. The method of claim 6, wherein, The total loss objective function of the initial citrus sugar content prediction model comprises a selection dispersion constraint, a spectral reconstruction consistency, a boundary / background suppression and a regression head supervision.

Images