Milk fat content prediction method and system based on deep learning combined with hyperspectral imaging technology

Abstract

The present invention belongs to the field of hyperspectral imaging technology and discloses a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology. The method comprises: using a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content; eliminating interference from light and environmental factors on the milk spectral data through a preprocessing method; using JLSP to select characteristic wavelengths and predict fat content of the milk spectral data, and comparing with traditional prediction models PLSR and SVR and traditional wavelength selection methods SPA and CARS. The present invention explores the feasibility of using deep learning methods to predict fat content on milk hyperspectral datasets. A hyperspectral image dataset of milk covering the entire shelf life and the corresponding fat content index are constructed, which solves the problem of incomplete datasets caused by different nutritional components of milk samples during different storage periods.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of hyperspectral imaging technology, and in particular relates to a method and system for predicting milk fat content based on deep learning combined with hyperspectral imaging technology. Background Art

[0002] Milk is a nutritious natural food, containing essential nutrients for young mammals and humans. In recent years, milk has become a vital component of the human diet. People of different age groups have different requirements for milk fat content, leading to the emergence of a growing number of milk varieties on the market. Adolescents often choose whole milk for their growing needs; some adults opt for low-fat milk for fitness and weight loss; and those with liver and gallbladder diseases choose skim milk to maintain their health. Furthermore, the fat content of milk affects its color, flavor, and mouthfeel, giving fat unique nutritional and sensory properties in dairy products such as butter, cream, and cheese. Therefore, rapid and accurate testing of milk fat content is crucial.

[0003] Hyperspectral imaging is an advanced optical imaging technique that uses hundreds or even thousands of narrowband spectral channels, covering the visible and infrared spectral ranges. The spectral absorption of organic matter in this region is primarily due to the harmonic and combination absorption of hydrogen-containing groups. The position and intensity of spectral absorption peaks generated by different groups vary. As the sample's component content changes, its spectral characteristics also change. Because organic matter in milk contains these hydrogen-containing groups, hyperspectral imaging can be used to rapidly determine milk composition. Zhao et al. used hyperspectral imaging to determine the fat content of milk. They selected a region of interest (ROI) and established a PLSR and multidimensional partial least squares regression (N-PLSR) model. The results showed that N-PLSR significantly outperformed the PLSR method. Milk is a complex colloidal mixture, so its spectrum includes both reflectance and transmission spectra. Luo et al. used hyperspectral imaging to perform linear regression analysis between the absorption and reflectance coefficients of milk samples at continuous wavelengths and the corresponding milk fat content. The results showed that spectral reflectance was more accurate in predicting milk fat content. French researchers Coppa et al. used near-infrared reflectance spectroscopy to predict the fatty acid composition of milk. Using an improved partial least squares regression model, they predicted the contents of saturated fatty acids, unsaturated fatty acids, and lauric acid in milk. This study demonstrated a high correlation between near-infrared reflectance spectroscopy and most fatty acids in milk.

[0004] Hyperspectral image data has a large number of continuous wavelengths and carries a wealth of milk-related information. This also results in high dimensionality and redundancy in hyperspectral image data. To further improve model performance, it is often necessary to select wavelengths that strongly correlate with the predicted indicators. However, milk is a complex liquid, which results in significant band overlap in hyperspectral image data. That is, within the same band, information about fat, water, protein, and other components can be present. This complicates feature selection. Hyperspectral image data is also susceptible to interference from lighting and environmental factors. Inadequate preprocessing of the spectral curves can make it difficult to address spectral data redundancy when selecting characteristic wavelengths. Some researchers have attempted to improve the accuracy of prediction models by combining various preprocessing methods with feature selection. Xu et al. used hyperspectral imaging technology (400-1000 nm) to predict the protein and fat content of milk. Convolution smoothing and first-order derivative preprocessing of the raw spectral curves significantly improved model prediction accuracy. Based on this, an improved spatial leapfrog feature selection method was proposed. Partial least squares regression (PLSR) and voting regression (VR) prediction models for protein and fat content were established, respectively. The coefficients of determination for the PLSR model were 0.8623 and 0.9608, respectively, and for the VR model were 0.9607 and 0.9834, respectively. This method addresses the redundancy issue in hyperspectral data to a certain extent and enhances the stability of wavelength selection. Huang et al. explored characteristic wavelengths associated with milk fat content based on both linear and nonlinear relationships. They proposed a wavelength selection method based on an improved ant colony-genetic algorithm. Results showed that both models have their own advantages. The support vector regression model achieved the highest accuracy of 0.9869, but its detection efficiency was relatively low. The multivariate linear regression model demonstrated relatively good detection efficiency, with a runtime as low as 0.02 seconds, but its prediction accuracy was slightly lower than the SVR model. This further reveals the potential for research on the nonlinear relationship between milk spectral reflectance data and fat properties.

[0005] Most researchers, when estimating the nutritional composition of milk, often only collect data from a single period, failing to consider that milk samples vary in freshness over time. Furthermore, milk stored at different times exhibits some differences in its nutritional composition. For example, the fat in milk decomposes and oxidizes during storage. Oxidation destroys the structure of fat molecules, leading to a decrease in fat content. Vitamins A and D in milk are also susceptible to light and oxidation, leading to a decrease in vitamin content. Therefore, it is necessary to factor storage time into the experiment. This experiment collected hyperspectral image data covering the entire shelf life of milk. Data collection lasted from January to June 2024. This full-period data is more conducive to analyzing nutritional indicators in milk, such as fat, protein, and carbohydrates, which are easily affected by freshness. Furthermore, given the large number of samples and the relatively large data volume, we introduced a deep learning model to analyze the milk hyperspectral data.

[0006] Deep learning models, with their multi-layered neural network architecture and nonlinear activation functions, can better capture complex nonlinear relationships in data. Deep learning, also known as representation learning, automatically learns feature representations from data layer by layer, offering high flexibility and expressive power. Furthermore, deep learning allows for the construction of an end-to-end model learning framework, automatically learning the entire process from raw input to final output, simplifying the modeling process. Therefore, compared to traditional machine learning methods, deep learning is more suitable for processing large-scale hyperspectral data.

[0007] There are many types of milk on the market today. Different types of milk have different nutritional components and different reflectivities in different wavelength ranges. When selecting features for a certain nutritional indicator, the influence of the content of other indicators should be considered. Not only global features should be considered, but also how to select effective characteristic wavelengths for different types of milk samples should be considered. It is crucial to understand which features are most relevant to the model output. Therefore, in order to explore the potential of combining deep learning with hyperspectral imaging technology to estimate milk fat content, we proposed a Selector and Predictor neural network (JLSP) based on joint learning, which can simultaneously complete the prediction of milk fat content and the selection of characteristic wavelengths. The present invention compares JLSP with traditional SVR, PLSR prediction models and SPA, CARS feature selection methods to explore the nonlinear relationship between milk hyperspectral data and fat content based on deep learning. It provides a new method for subsequent non-destructive testing of milk nutritional quality. Summary of the Invention

[0008] In response to the problems existing in the existing technology, the present invention provides a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology.

[0009] The present invention is achieved by a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology, the method comprising:

[0010] S1: Hyperspectral imaging system was used to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content;

[0011] S2: The interference of light and environmental factors on the milk spectral data was eliminated through preprocessing methods;

[0012] S3: Use JLSP to select characteristic wavelengths and predict fat content of milk spectral data, and compare with traditional prediction models PLSR, SVR and traditional wavelength selection methods SPA, CARS.

[0013] Furthermore, the hyperspectral imaging system is mainly composed of a hyperspectral imager, a whiteboard, a halogen lamp, a computer and corresponding supporting control software;

[0014] The spectral wavelength range measured by the hyperspectral image data is 400-1000nm, the spectral resolution is 4.8nm, and the number of spectral channels is 750; the experiment was conducted in a dark room, using a 50W halogen lamp to simulate natural light, setting the distance between the stage and the lens to 30cm, and the system exposure time to 10ms. In order to eliminate baseline drift, the spectrometer needs to be preheated for 30 minutes before measurement; during the experiment, pure milk is poured into a glass culture dish, the culture dish is placed in the center of the stage, and black flannel is arranged around it to avoid interference from irrelevant reflection sources; when collecting spectral data, due to the uneven intensity distribution of the light source in different bands, the presence of dark current and some ambient light in the hyperspectral imager, the collected image is accompanied by certain noise, and black and white correction processing is required to try to eliminate the influence of objective conditions; the black and white correction formula is as follows:

[0015]

[0016] Among them, R is the image data after black and white correction, W is the whiteboard data, B is the blackboard data, and I is the original image data.

[0017] Furthermore, the fat content test was conducted by storing milk samples in an environment with a temperature of 28°C and a humidity of 17%. Considering that some protein and fat in liquid milk will form flocculent substances and precipitate, the milk needs to be shaken before measurement. The fat content of the milk samples was measured using the Foss milk composition analyzer MilkoScan FT120.

[0018] The milk samples were collected from 14 liquid milk brands including Yili, Mengniu, Bright, Sanyuan, etc., totaling 83 types of samples, including pure milk, low-fat milk, skimmed milk, high-calcium milk, organic milk, etc.; all types of milk were within the shelf life of 6 months. In order to cover the freshness range of all milk, a total of 5 tests were conducted on each type of milk, once every month, 3 samples were tested each time, and a total of 1,245 samples were tested.

[0019] Furthermore, the S2 specifically includes: using the second-order derivative preprocessing method to preprocess the spectral data and perform subsequent analysis in the SVR and JLSP models, and using the original spectral data to perform subsequent analysis in the PLSR model.

[0020] Furthermore, the JLSP model is an end-to-end deep learning model framework that can simultaneously perform feature selection and regression prediction tasks. Inspired by INVASE, JLSP consists of three neural networks: Selector, Predictor, and Baseline. Selector is a feature selection network used to learn feature representations related to milk fat content. Predictor and Baseline are regression prediction networks used to predict milk fat content. d is a eigenvector of dimension d, x d Will participate in the input of all neural networks; among them, Selector receives the original features x of the sample d And output a probability vector p d ; Perform Bernoulli sampling based on the probability vector to obtain a selection vector s with only 0 and 1 d On the one hand, the original feature x d With the selection vector s d Multiply to get the features after dimensionality reduction This is an instance-based feature selection method that can flexibly select different numbers of characteristic wavelengths for each milk sample among many types of milk; Predictor receives the feature vector after dimensionality reduction by Selector And output the predicted value y p On the other hand, the original feature x d With the probability vector p d Multiply to get the weighted eigenvector By weighting, the model can focus on the important parts of the data while maintaining the original number of features, thereby improving the ability to identify key information. The baseline receives the weighted feature vector And output the predicted value b; the model is updated using a joint learning strategy, in which all networks are trained through back propagation; the Predictor network and the Baseline network can directly calculate the mean square error (MSE) loss through the predicted value; the loss of the Selector network is jointly calculated by the losses of the Predictor and Baseline networks.

[0021] Furthermore, the Selector network is a one-dimensional convolutional neural network (1D-CNN), consisting of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. 1D-CNN extracts local features from the input data by sliding the convolution kernel, enabling the model to capture characteristic information between adjacent wavelengths. Furthermore, the convolution kernels share parameters across the entire input data, reducing the number of parameters required to learn in the network, making the network more lightweight and mitigating the risk of overfitting. Compared with traditional feature selection methods, 1D-CNN can extract features from spectral data layer by layer through multi-layer nonlinear modules, achieving higher learning efficiency and stronger generalization. The Selector network has three convolutional layers with a convolution kernel size of 3 and output channels of 32, 64, and 128, respectively. Given the feature dimension of a single sample is 1×125, the scan step size in the first convolution layer is set to 2, and the scan step size in the other two layers is set to 1. Each convolution operation is followed by a pooling layer, which performs max pooling on the data within the pooling window to further compress the feature dimension. After three layers of convolution and pooling, a 128×5 feature representation is ultimately obtained. Before entering the fully connected layer, the data from all channels is expanded into a one-dimensional tensor and fed into two fully connected layers in sequence. The number of neurons in the two fully connected layers is 256 and 125, respectively. The output layer is connected to a sigmoid function to ensure that the output is a probability vector bounded between 0 and 1, with the same dimension as the input data.

[0022] Furthermore, the PLSR is a linear regression algorithm, which is mainly used to deal with multicollinearity and high-dimensional data. It maps the relationship between the original spectral reflectance of milk and milk components into a new space by extracting latent variables, reducing the data dimension while retaining important information in the data. The SVR is a nonlinear regression algorithm, whose main goal is to establish a hyperplane that can find the best fit in high-dimensional space to maximize the boundary between the predicted value and the actual value. The kernel function of SVR has unique advantages in high-dimensional space and can handle nonlinear relationships of sample data, and is particularly suitable for processing hyperspectral data sets with multiple continuous wavelengths.

[0023] Furthermore, the SPA is a feature variable selection method that can map high-dimensional data into a low-dimensional space while preserving the main information of the original data. By projecting a wavelength onto other wavelengths and comparing the magnitudes of the projection vectors, the wavelength with the largest projection vector is selected as the candidate wavelength. Multiple regression analysis is then used to determine the characteristic wavelength combination. CARS combines Monte Carlo sampling with PLS analysis, using the regression coefficient as the wavelength importance and an exponential decay method to determine the number of wavelengths. Reweighted sampling is used to obtain candidate feature subsets. Finally, the optimal feature set is determined based on the minimum RMSECV of the PLS model.

[0024] Another object of the present invention is to provide a milk fat content prediction system based on deep learning combined with hyperspectral imaging technology based on the milk fat content prediction method based on deep learning combined with hyperspectral imaging technology, the system specifically comprising:

[0025] The image acquisition and content determination module uses a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content;

[0026] The preprocessing module is connected to the image acquisition and content determination module, and eliminates the interference of light and environmental factors on the milk spectral data through preprocessing methods;

[0027] The comparison module is connected to the preprocessing module and uses JLSP to select characteristic wavelengths and predict fat content of milk spectral data, and compares them with traditional prediction models PLSR and SVR and traditional wavelength selection methods SPA and CARS.

[0028] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the milk fat content prediction method based on deep learning combined with hyperspectral imaging technology.

[0029] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0030] First, this invention collects hyperspectral image data of milk throughout its shelf life and measures its fat content. This full-lifecycle data not only facilitates analysis of milk's nutritional indicators, such as fat, protein, and carbohydrates, which are easily affected by freshness, but also improves model performance, providing consumers with more accurate reference values ​​for these indicators.

[0031] We compared various preprocessing methods based on milk spectral data to eliminate interference from light and environmental factors and increase spectral resolution. Experimental validation revealed that the second-order derivative preprocessing method was ideal, separating the smooth and slowly varying portions of the raw milk spectral data, thereby highlighting the rapidly varying portions. Inflection points and slope changes in the raw milk spectral data were particularly emphasized. These correspond to changes in the content of macromolecular compounds such as fat and protein in milk, paving the way for subsequent feature selection.

[0032] The JLSP model proposed in the present invention is an instance-based feature selection method. It solves the problem that the number of features of all samples in the global feature selection method is always the same. For milk of different types and different nutrient contents, JLSP will select effective feature wavelengths for them, and the number of features of each sample is calculated separately and is not limited by the number of global features. For example, the water content in skim milk is higher than that in whole milk, so more water-related wavelengths will be selected during the feature selection process of skim milk. High-protein whole milk contains a large number of macromolecular compounds, and compared with skim milk, more wavelengths related to NH bond and CH bond stretching vibration will be selected during the feature selection process.

[0033] This technical solution uses two traditional machine learning algorithms to predict milk fat content, namely the linear regression model PLSR and the nonlinear regression model SVR. This approach can better analyze the relationship pattern between milk spectral data and fat content. The experimental results show that in terms of prediction accuracy, SVR performs better than PLSR, demonstrating the prediction potential of nonlinear regression models on milk hyperspectral data. JLSP, as a deep learning framework, has a multi-layer neural network structure and nonlinear activation function, which further expands the advantages of nonlinear models and can better capture the complex nonlinear relationships in milk spectral data. Experimental results show that JLSP can not only complete the selection of characteristic wavelengths, but also show excellent performance in predicting milk fat content. The prediction effect of JLSP is better than the above-mentioned traditional methods. Test set R 2 It reaches 0.9734, and the mean square error is 0.0573. In the feature selection part, JLSP will generate n (number of iterations) probability heat maps of size of number of samples × number of features during the training process. Each element represents the probability of the feature being selected in a certain sample, which allows the neural network originally running under black box to visualize the feature representation, increasing the interpretability of the JLSP feature selection results. Subsequent analysis combined with the corresponding chemical bonds proved that JLSP can effectively select wavelengths related to milk fat content. The application of JLSP can also be extended to other nutrients such as milk protein and carbohydrates, providing new methods for quality control and nutritional monitoring of dairy products.

[0034] Second, the technical solution of the present invention fills the technical gap in the industry at home and abroad: the present invention applies the instance feature selection method to the research on milk fat content prediction for the first time, providing new ideas for subsequent milk nutritional index detection.

[0035] The technical solution of the present invention solves a technical problem that people have long desired to solve but have never been able to solve: the current traditional feature selection method has some problems. After calculating the correlation between all wavelengths and fat content and sorting them from high to low, the model performance is compared using an incremental feature selection method. The results show that the milk fat content prediction results using spectral data of all wavelengths have the highest accuracy. This may be because the selected characteristic wavelengths are global characteristic wavelengths, that is, the selection is based on all samples, rather than feature selection based on individual samples. All samples have the same characteristics, but the milk varieties are different. The global characteristics have affected the prediction of some milk samples to a certain extent.

[0036] The technical solution of the present invention overcomes technical bias: Current research on predicting the nutrient content of milk suffers from incomplete data coverage of milk shelf life. When selecting samples, researchers or testing departments often only purchase milk samples within a single shelf life and bring them back to the laboratory for testing. Consumers, however, may purchase milk within its shelf life but stored at different times. The nutritional content of milk changes during different storage periods. Relying solely on the nutrient content labels on milk products or milk content predictions for a single time period cannot provide accurate reference for consumers. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology provided by an embodiment of the present invention;

[0038] Figure 2 is a structural diagram of a hyperspectral imaging system provided by an embodiment of the present invention;

[0039] Figure 3 This is the JLSP model structure provided by the embodiment of the present invention;

[0040] Figure 4 This is the Selector network structure provided by the embodiment of the present invention;

[0041] Figure 5 This is the Predictor network structure provided by the embodiment of the present invention;

[0042] Figure 6 This is the ROI selection process provided by the embodiment of the present invention;

[0043] Figure 7The original spectral reflectance and the pre-processed spectral reflectance provided by the embodiment of the present invention;

[0044] Figure 8 This is the wavelength Pearson correlation analysis provided by the embodiment of the present invention;

[0045] Figure 9 It is the predicted value and actual value of the Predictor combined with different feature selection methods provided by the embodiment of the present invention;

[0046] Figure 10 are the predicted values ​​and actual values ​​of different prediction models combined with the optimal feature selection method provided by the embodiment of the present invention;

[0047] Figure 11 This is a comparison of the SPA, CARS, and Selector feature selection results provided by the embodiment of the present invention;

[0048] Figure 12 spectral reflectance of milk at different fat contents provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] like Figure 1 As shown, an embodiment of the present invention provides a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology, the method comprising:

[0051] S1: Hyperspectral imaging system was used to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content;

[0052] S2: The interference of light and environmental factors on the milk spectral data was eliminated through preprocessing methods;

[0053] S3: Use JLSP to select characteristic wavelengths and predict fat content of milk spectral data, and compare with traditional prediction models PLSR, SVR and traditional wavelength selection methods SPA, CARS.

[0054] 1 Materials and Methods

[0055] 1.1 Experimental Sample

[0056] The experimental samples included 83 types of milk, including pure milk, low-fat milk, skim milk, high-calcium milk, and organic milk, from 14 liquid milk brands, including Yili, Mengniu, Bright Dairy, and Sanyuan. All milk types were within their six-month shelf life. To cover the full range of milk freshness, each type of milk was tested five times, once every month, with three samples tested each time. This totaled 1,245 samples.

[0057] 1.2 Determination of milk fat content

[0058] Milk samples were stored at 28°C and 17% humidity. Because some protein and fat in liquid milk can form floccules and precipitate, the milk was shaken thoroughly before measurement. The milk fat content was determined using a Foss MilkoScan FT120 milk analyzer. Table 1 shows the statistical values ​​of fat content for all milk batches. 1.3 Hyperspectral image data acquisition The hyperspectral imaging system mainly consists of a hyperspectral imager, a whiteboard, a halogen lamp, a computer and corresponding supporting control software. Figure 2 The experiment used the model produced by Headwall Company of the United States in 2010. The PTU-D48E hyperspectral imager measures spectral wavelengths from 400 to 1000 nm, with a spectral resolution of 4.8 nm and 750 spectral channels. Experiments were conducted in a darkroom, using a 50W halogen lamp to simulate natural light. The distance between the stage and the lens was set at 30 cm, and the system exposure time was 10 ms. To eliminate baseline drift, the spectrometer was preheated for 30 minutes before measurement. Pure milk was poured into a glass Petri dish, placed in the center of the stage, and surrounded by black flannel to prevent interference from irrelevant reflections. During spectral data acquisition, due to the uneven intensity distribution of the light source across different wavelengths, dark current in the hyperspectral imager, and some ambient light influence, the captured images were accompanied by a certain amount of noise. Black-white correction was required to minimize the effects of these conditions. The black-white correction formula is as follows: Among them, R is the image data after black and white correction, W is the whiteboard data, B is the blackboard data, and I is the original image data. 1.4 Selector and predictor neural network based on joint learning (JLSP) 1.4.1 Model Framework In order to better extract the local features of milk hyperspectral data, we proposed the JLSP model, such as Figure 3 As shown in Figure 2. This is an end-to-end deep learning model framework that can perform both feature selection and regression prediction tasks. Inspired by INVASE, JLSP consists of three neural networks: Selector, Predictor, and Baseline. Selector is a feature selection network used to learn feature representations related to milk fat content; Predictor and Baseline are regression prediction networks used to predict milk fat content. Input sample x d is a eigenvector of dimension d, x d Will participate in the input of all neural networks. Among them, Selector receives the original features x of the sample d And output a probability vector p d . According to the probability vector, Bernoulli sampling is performed to obtain a selection vector s with only 0 and 1 d On the one hand, the original feature x d With the selection vector s d Multiply to get the features after dimensionality reduction This is an instance-based feature selection method that can flexibly select different numbers of characteristic wavelengths for each milk sample among many types of milk. Predictor receives the feature vector after dimensionality reduction by Selector. And output the predicted value y p On the other hand, the original feature x d With the probability vector p d Multiply to get the weighted eigenvector By weighting, the model can focus on the important parts of the data while maintaining the original number of features, thereby improving the ability to identify key information. The baseline receives the weighted feature vector And output the predicted value y b The model is updated using a joint learning strategy, where all networks are trained via backpropagation. The mean squared error (MSE) loss of the Predictor and Baseline networks is directly calculated using the predicted values; the loss of the Selector network is calculated jointly using the losses of the Predictor and Baseline networks.

[0059] The Selector, Predictor, and Baseline are described in detail below. The Selector network is a one-dimensional convolutional neural network (1D-CNN) consisting of an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer. 1D-CNN extracts local features from the input data by sliding the convolution kernel. This enables the model to capture feature information between adjacent wavelengths. In addition, the convolution kernel shares parameters across the entire input data, which reduces the number of parameters that need to be learned in the network, making the network more lightweight and reducing the risk of overfitting. Compared with traditional feature selection methods, 1D-CNN can extract features from spectral data layer by layer through multi-layer nonlinear modules, with higher learning efficiency and stronger generalization ability

[18] .

[0060] like Figure 4 As shown, the Selector has three convolutional layers with a convolution kernel size of 3 and output channels of 32, 64, and 128, respectively. Considering that the feature dimension of a single sample is 1×125, the scan step in the first convolutional layer is set to 2, and the scan step in the other two layers is set to 1. Each convolution operation is followed by a pooling layer to perform maximum pooling on the data within the pooling window to further compress the feature dimension. After three layers of convolution and pooling operations, a 128×5 feature representation is finally obtained. Before entering the fully connected layer, the data of all channels are expanded into a one-dimensional tensor and input into two fully connected layers in sequence. The number of neurons in the two fully connected layers is 256 and 125, respectively. The output layer is connected to a Sigmoid function to ensure that the output is a probability vector with a bounded range between 0 and 1 and the same dimension as the input data.

[0061] The Predictor network and the Baseline network are both multi-layer perceptrons, but their input data are different. Figure 5 As shown, Predictor receives the feature vector selected by Selector, which is composed of {x i ,0}; Baseline receives the product of the original feature vector and the probability vector directly output by Selector, which is composed of The network structure of the two is the same, both have three hidden layers, and the number of neurons are 64, 32, and 16 respectively. The SELU activation function is used between each layer. The SELU function is a nonlinear function that can help the network learn complex nonlinear patterns. At the same time, the function introduces a self-normalization mechanism to keep the mean and variance of the output of each layer close to 1 during the training process, which to a certain extent solves the problems of gradient explosion and gradient disappearance in deep neural networks. Compared with the ReLU activation function, the independent variable of the SELU function takes values ​​close to 0 but not equal to 0 in the range of 0 to negative infinity, which helps to avoid the problem of neuron "death". Both networks have been fully designed from the input layer to the output layer. Therefore, Predictor and Baseline can be independently trained as regression models for predicting fat content.

[0062] 1.4.2 Joint Learning Strategy

[0063] Inspired by the actor-critic algorithm in reinforcement learning

[20] , we propose a joint learning strategy that can train three neural networks simultaneously. Selector receives a full spectrum output vector x of dimension d and generates a probability vector p of the same dimension. Each value of p is a probability value between 0 and 1, indicating the possibility of selecting the corresponding wavelength in the input vector. Bernoulli sampling is performed using the probability vector p to obtain a selection vector s of dimension d, which consists only of 0 and 1. 0 indicates that the wavelength at the corresponding position is not selected, and 1 indicates that the wavelength at the corresponding position is selected. Multiply the selection vector s with the original input vector x to obtain the selected spectral data x*, which is defined as x*.

[0064]

[0065] Multiply the probability vector by the original input vector x to obtain the weighted spectral data Predictor receives the selected spectral data x* and performs an iterative training. In order to evaluate the performance of x* on the Predictor network, the MSE loss function is used to calculate the Predictor loss. This loss can be used to update the Predictor network parameters through back propagation. In addition, Baseline receives the weighted spectral data And perform one iteration of training. Use MSE loss function to calculate Baseline loss to evaluate feature representation Performance on the Baseline network. This loss is used by the Baseline network to update network parameters in back propagation, and is also used to calculate the reward in combination with the Predictor loss. The reward calculation formula is as follows:

[0066] Reward=-(L predictor -Lbaseline )#(3)

[0067] Reward value represents x* and The performance of the network is quantified during iterative training, and the reward will be used to calculate the Selector loss. If the reward is positive, it means that x* is better than This further indicates that the Selector has performed effective feature selection and encourages the Selector to maintain its current feature selection strategy. If the Reward is negative, it means that some spectral information of x* has been lost in the feature representation, further indicating that the Selector has not performed effective feature selection. Based on the Reward value, the Selector is given a corresponding penalty item and the current feature selection strategy is adjusted. It is worth noting that the purpose of the Reward is to minimize the loss of the Predictor, not to minimize the difference in loss between the Baseline and the Predictor.

[0068] Table 2.JLSP training process

[0069]

[0070]

[0071] The training process of JLSP is shown in Table 2. The validation set is used to evaluate the performance of the Selector, Predictor, and Baseline networks during each iterative training process, and training continues until the Selector network converges. Since the input features of the Predictor and Baseline change with each update of the Selector, the Predictor and Baseline must be updated after each Selector update. Therefore, the Predictor and Baseline will not converge before the Selector network reaches convergence. The trained Selector and Predictor can be used separately as feature selection models and regression prediction models. Whether the Baseline converges is not of particular concern in this experiment. In the actor-critic algorithm, the Baseline is usually used to reduce the variance of the model and "standardize" the Predictor network. Keeping the training progress of the Baseline and Predictor close to the same level helps train the Predictor network.

[0072] 1.4.3 Loss Function

[0073] The present invention proposes a joint learning strategy based on three neural networks: Selector, Predictor and Baseline. Therefore, it is necessary to design a loss function for the Selector network. Considering that the Selector loss is related to the difference between the Predictor and Baseline losses, the feature selection probability of the Selector output and the Bernoulli sampling result, the Selector loss function designed by the present invention consists of three parts: Reward, L cross And L1 regularization, the specific calculation formula is as follows.

[0074]

[0075] Among them, L cross This parameter represents the cross-entropy loss between the feature selection probability p output by the Selector and the feature selection vector s. Cross-entropy compares the difference between two probability distributions. By minimizing the cross-entropy loss, the feature selection probability is continuously approached to 0 or 1, making the Bernoulli sampling results more stable and enabling the model to better learn label-related features. L1 regularization is a technique that adds the L1 norm of the weights as a penalty term to the model's loss function. It is used to constrain model complexity during training to prevent overfitting. The L1 norm is the sum of the absolute values ​​of each element in the vector. λ is the regularization parameter, which controls the weight of the regularization term in the overall loss. Compared to L2 regularization, L1 regularization penalizes all weights equally, causing smaller weights to become zero after being penalized. Since a large number of model parameters are set to zero, they do not appear in the final model, achieving sparseness. This also demonstrates that L1 regularization is beneficial for feature selection and improves model interpretability. Reward is the specific performance of the Selector output on the Predictor and Baseline. The optimization trend of Selector is affected by Reward. When Reward is positive, it means that the current wavelength selection result is good. selector To encourage the maintenance of the current wavelength selection strategy and further optimize the strategy. When the reward value is negative, it means that the current wavelength selection result is poor. In the back propagation process, by maximizing L selector This penalizes the current wavelength selection strategy and prompts a new one to be found. The overall selector loss function is calculated by adding the difference between the feature selection probability and the feature selection vector, adding the L1 regularization term, and multiplying the result by the reward. This design benefits both by continuously improving the matching between the feature selection probability and the feature selection vector. Furthermore, it allows the selector to find the optimal feature wavelength for each instance while also taking into account the sparsity of wavelength selection.

[0076] 1.5 Traditional methods

[0077] 1.5.1 Traditional machine learning algorithms

[0078] This study used two regression models commonly used in traditional spectral analysis, namely support vector regression (SVR) and partial least squares regression (PLSR). PLSR is a linear regression algorithm mainly used to handle the cases of multicollinearity and high-dimensional data. It maps the relationship between the original spectral reflectance of milk and milk components to a new space by extracting latent variables, reducing the data dimension while retaining the important information in the data. SVR is a non-linear regression algorithm whose main goal is to establish a hyperplane that can find the best fit in a high-dimensional space to maximize the margin between the predicted value and the actual value. The kernel function of SVR has unique advantages in high-dimensional spaces and can handle the non-linear relationships of sample data, especially suitable for processing hyperspectral datasets with multiple continuous wavelengths.

[0079] 1.5.2 Wavelength selection

[0080] Due to the problems of high redundancy and multicollinearity in hyperspectral image data, it brings difficulties to data analysis. Hyperspectral data with all wavelengths often require a large amount of storage space and increase the model calculation time. Therefore, effective wavelength selection is a key step in traditional hyperspectral data analysis. SPA is a method for selecting characteristic variables that can map high-dimensional data to a low-dimensional space while retaining the main information of the original data. By projecting wavelengths onto other wavelengths and comparing the magnitudes of the projection vectors, the wavelength with the largest projection vector is taken as the candidate wavelength. Combining multiple regression analysis to determine the characteristic wavelength combination. CARS analyzes by combining Monte Carlo sampling method with PLS, takes the regression coefficient as the importance of the wavelength, and uses the exponential decay method to determine the number of wavelengths. Obtain candidate characteristic subsets through reweighted sampling. Finally, determine the best characteristic set according to the minimum RMSECV of the PLS model.

[0081] 1.6 Evaluation metrics

[0082] Based on three model evaluation metrics, namely the correlation coefficient R, the coefficient of determination R 2 and the mean squared error MSE, the performances of three algorithms were compared. R calculates the similarity using the quotient of the covariance and standard deviation between two variables, and its value range is from -1 to 1. When 0 < R ≤ 1, it indicates that the two sets of variables are positively correlated; when -1 ≤ R < 0, it indicates that the two sets of variables are negatively correlated; when R = 0, it indicates that there is no linear correlation between the two sets of variables. The calculation formula of R is as follows:

[0083]

[0084] where, σX and σ Y represent the standard deviation of variables X and Y respectively, and Represents the mean of X and Y respectively. This formula calculates the ratio of the covariance of variables X and Y to the product of their respective standard deviations. The standardized covariance is dimensionless.

[0085] R 2 Measures the proportion of variance in the dependent variable that can be explained by the independent variable. It ranges from 0 to 1. A well-performing model usually has a high R 2 . R 2 The calculation formula is as follows.

[0086]

[0087] where y i represents the true value of the dependent variable, represents the predicted value of the dependent variable, represents the mean of the true value of the dependent variable, and n represents the number of samples.

[0088] MSE represents the mean deviation between the predicted value and the true value. Models with good MSE performance have low MSE values. The formula for calculating MSE is as follows.

[0089]

[0090] where y i is the true value of the dependent variable, is the predicted value of the dependent variable, and n is the sample size.

[0091] In modern milk production and processing, milk fat content prediction plays a crucial role, influencing manufacturers' production processes, product development, and marketing strategies. First, manufacturers can use fat content prediction models to monitor and adjust milk fat content in real time to ensure product quality standards are met. Manufacturers can promptly identify potential problems and take appropriate measures to ensure consistent product quality. Second, fat content prediction models can promote product innovation and development. Manufacturers can customize milk products with different fat contents, such as low-fat milk and skim milk, based on consumer demand and market trends to meet the health needs and taste preferences of different groups. Furthermore, the combination of hyperspectral imaging technology and deep learning has expanded its application possibilities in many fields. In agriculture, this combination can help farmers monitor crop growth, detect pests and diseases, and refine fertilization management, thereby improving crop quality and yield. In the food industry, analyzing spectral information on food surfaces can assess food freshness and quality. In food safety testing, it can help detect harmful substances and microbial contamination in food, providing consumers with food safety assurance.

[0092] Relevant evidence of the technical effects achieved by the embodiments of the present invention.

[0093] 1.1 Acquisition of spectral reflectance of milk samples

[0094] like Figure 6 (a) shows the hyperspectral image data of a milk sample. This data has three dimensions, including image space and spectral space. The image space consists of 816×1004 pixels, and the spectral space consists of 125 wavelengths in the range of 400-1000nm. This study first selects 450nm, 550nm, and 650nm channels from the numerous channels of the hyperspectral image data to form an RGB visualization image, as shown in Figure 1. Figure 6 (b) RGB is one of the most commonly used color modes and one of the most fundamental color modes in digital image processing. Especially for images with complex backgrounds, researchers often visualize hyperspectral images as RGB images before selecting regions of interest. Figure 6 The red circle in (c) represents the region of interest selected on the hyperspectral data. The spectral reflectance of all pixels within the ROI at each wavelength is calculated to obtain the average spectral reflectance. Figure 6 (d) is the spectral reflectance curve for all milk samples. The dark blue curve represents the average spectral reflectance of all samples, and the light blue area represents the standard deviation of the spectral reflectance of all samples. This spectral reflectance will be used for subsequent data analysis.

[0095] 1.2 Dataset Division and Parameter Setting

[0096] The present invention divides the data set into training set, validation set and test set in a ratio of 6:2:2. The number of samples is 747, 249 and 249 respectively. This study uses SVR and PLSR as conventional prediction models and compares the performance of different preprocessing methods and feature selection methods on the two conventional prediction models. The grid search method is used to optimize the C value, gamma value and kernel function of SVR, and the number of principal components of the PLSR model is optimized. To avoid overfitting, 5-fold cross validation is used to evaluate model performance and generalization ability.

[0097] 1.3 Data Preprocessing

[0098] Data preprocessing is an important step in preparing spectral data to obtain more stable model performance. Figure 7(a) It can be seen that the original spectral curve does not have much noise, but there is a baseline drift phenomenon. The present invention compares four preprocessing methods: multivariate scatter correction (MSC), standard normal transformation (SNV), derivative transformation and convolution smoothing (SG). MSC can eliminate spectral differences caused by uneven particle size and different scattering levels. By establishing a mathematical model to solve the baseline translation and offset, the spectrum of each sample is corrected. SNV standardizes each column of spectral values ​​of the sample by mean and variance, which can reduce the impact of unstable spectral intensity in spectral data. The preprocessing effects of MSC and SNV are similar and can eliminate the baseline drift phenomenon. The main idea of ​​SG smoothing filtering is to perform polynomial least squares fitting on each point in the sliding window to obtain the best estimate of the smooth point. This method can remove the noise of the spectral data and keep the shape and width of the spectral information unchanged. The basic idea of ​​derivative transformation is to separate the smooth part and the slowly changing part in the original spectral data by performing a derivative operation on the spectral data, thereby highlighting the rapidly changing part in the spectral data. Commonly used derivative transformation methods include first-order derivative (FD) and second-order derivative (SD). The first-order derivative can highlight peaks and valleys in the raw spectral data, while the second-order derivative can highlight inflection points and slope changes. Since the spectral curves after first-order and second-order derivative preprocessing exhibit significant noise, we added an experimental combination of these two methods with convolution smoothing to eliminate baseline drift and reduce data noise.

[0099] As shown in Table 3, after model testing, it can be seen that the accuracy of the second-order derivative preprocessing method based on the SVR model is relatively high. On the one hand, it demonstrates the prediction potential of the nonlinear regression model on milk hyperspectral data, and on the other hand, it proves the effectiveness of the second-order derivative preprocessing. Figure 7 (b) It can be seen that the second-order derivative transform removes the drift that is unrelated to the wavelength and enhances the peak information of the spectral curve, making the peak clearer and more prominent. This increases the resolution of the spectral data to a certain extent, making the method more accurate. Since there is less noise in the original spectral data, excessive convolution smoothing will lead to the loss of spectral information, which ultimately leads to unsatisfactory results of the derivative transform combined with the convolution smoothing method. In addition, since the surface of fresh milk is smooth and homogeneous, the MSC and SNV treatment effects on the scattering phenomenon are not obvious. Therefore, the present invention uses the second-order derivative preprocessing method in the SVR and JLSP models to preprocess the spectral data and perform subsequent analysis, and uses the original spectral data in the PLSR model for subsequent analysis.

[0100] Table 3. Performance Comparison of Different Preprocessing Methods on SVR and PLSR Models

[0101]

[0102] 3.4 Results Analysis

[0103] This study first performed Peason correlation analysis on 125 wavelengths between 400-1000nm. Figure 8 As can be seen, HIS has highly correlated adjacent wavelengths, with correlation coefficients between adjacent wavelengths exceeding 0.95. The correlation decreases gradually with increasing wavelength span. The correlation coefficient between wavelengths between 861.3994nm and 933.4224nm and those before 861.3994nm is approximately 0.85; the correlation coefficient between wavelengths after 933.4224nm and those between 400.4518nm and 746.1625nm is approximately 0.75. This leads to multicollinearity between wavelengths. To address this issue, a feature wavelength selection method is used to reduce the dimensionality of the HIS data and reduce the amount of computation. This dimensionality reduction strategy can significantly improve the accuracy and generalization ability of the model.

[0104] The experiment used full wavelength (125), SPA-selected wavelength (29), CARS-selected wavelength (42), and Selector-selected wavelength (63) to predict milk fat content. The prediction performance of the SVR, PLSR, and Predictor models was compared. The parameters of SVR and PLSR were the same as the parameter optimization steps in the preprocessing process mentioned above. The specific parameters of the JLSP model were set as follows: the learning rate of the three networks was 0.001, λ was set to 0.1, the batch size was 83, and the number of iterations was 10,000.

[0105] The experimental results are shown in Table 4. Among the traditional methods, the prediction results of the SVR model are better than those of the PLSR model in various feature selection methods. This may be because SVR can better fit the nonlinear relationship in hyperspectral data. Among them, the wavelength selected by Selector achieves the highest performance in the SVR model. 2 It is 0.9704. This shows that the feature selection method proposed in this study can effectively match the nonlinear model and improve the model prediction accuracy. Figure 10(a) shows the prediction results of the milk fat content of the test set using the SVR model using the Selector method. Overall, most prediction points are close to the fitted line, indicating a good model fit. However, some prediction points near the fat content of 1.0 (g / 100ml) exhibit significant deviations. Most sample points between 0.0 and 1.0 (g / 100ml) are predicted as 0.0 (g / 100ml).

[0106] Among the PLSR models, the model using full wavelength achieved the highest performance, and the test set R 2 The value is 0.9435. This may be due to the loss of some spectral information in the data after dimensionality reduction using feature selection methods, while the PLSR model relies on the linear relationship between wavelength and fat content for prediction. The number of wavelengths selected by different feature selection methods shows that the performance of the PLSR model on the test set is positively correlated with the number of wavelengths. Therefore, it can be concluded that the lost spectral information affects the predictive performance of the PLSR model. Figure 10 (b) shows the prediction results of the PLSR model at all wavelengths, which is consistent with Figure 10 Compared to (a), the problem of large errors in individual prediction points persists. Furthermore, most sample points near 0.0 (g / 100ml) are predicted as negative. This is because the PLSR training process fails to capture the data characteristics well, resulting in underfitting of the model.

[0107] When using only Predictor as a regression predictor, the model performance on full wavelength data and feature selection data was compared. The results show that, in terms of traditional feature selection methods, Predictor outperforms PLSR but is slightly lower than SVR. This is partly due to SVR's excellent ability to handle nonlinear relationships and high-dimensional data, and partly due to the fact that the present invention designed Predictor to be relatively simple when considering the overall model size and number of parameters of JLSP, resulting in slightly lower performance than SVR. However, as can be seen from Table 4, the wavelength selected by Selector achieves the highest performance, with the test set R 2 The value is 0.9724. This is due to the Selector's excellent cross-channel information interaction and integration capabilities, which effectively improve the Predictor's network expression capabilities. Especially for hyperspectral data with high correlation between adjacent wavelengths, the Selector can capture local features of the data, reducing feature dimensionality while improving the model's prediction efficiency. Figure 9 The prediction point fitting results of Predictor on full wavelength data and different feature selection data are shown respectively. Figure 9 (b) It can be seen that the prediction point deviation of the combination based on the SPA method is slightly larger than that of the other three combinations when the fat content is around 4.0-6.0 (g / 100ml). Figure 9From the sample points near 0.0 (g / 100ml) in (a), 9 (c) and 9 (d), Figure 9 The prediction point deviation of (d) is the smallest, indicating that the Predictor-Selector combination achieves the highest performance. This combination corresponds to the Selector feature selection method under the Predictor model in Table 4.

[0108] Table 4. Model performance of different feature selection methods

[0109]

[0110] JLSP achieves the highest performance in predicting milk fat content, with the test set R 2 The prediction results are as follows: Figure 10 As shown in (d), compared with Figure 9 In (d), Predictor is trained alone, and JLSP using the joint training strategy achieves better prediction results. Figure 10 (d) The predicted points near the fat content of 4.0-6.0 (g / 100ml) are closer to the fitted curve. This is because the reward in the joint training strategy provides timely feedback to the Selector, allowing the Predictor to participate in the feature selection process. The Selector's output at different training stages increases the diversity of the Predictor's input data, thereby improving the Predictor's generalization ability.

[0111] 1.5 Discussion

[0112] In the discussion section, we try to use the wavelength corresponding to the chemical bond to explain the selection results of the three feature selection methods: SPA, CARS and Selector. The feature selection results of the three methods are as follows Figure 11As shown in the figure, the three methods have six commonly selected wavelengths, namely 410.0549nm, 496.4826nm, 818.1885nm, 928.6208nm, 943.0255nm and 957.4301nm. Among them, the wavelength of 928.6208nm is a spectral band near the third harmonic of the fundamental frequency vibration of the CH chemical bond

[29] ; the wavelengths of 943.0255nm and 957.4301nm may be related to the stretching of the second overtone of the OH chemical bond. The wavelengths selected by the Selector are mainly distributed in the peaks or troughs of the spectral curve, which is more conducive to analyzing the changes in the content of different substances in milk. The wavelengths 722.1548 nm and 746.1625 nm correspond to the third overtones of the CH stretching vibrations of the methyl and methylene groups, respectively. The wavelength around 765.3686 nm is likely related to water and carbohydrate content. The wavelengths between 971.8347 and 986.2393 nm are associated with the second overtone of the fundamental NH bond vibration, corresponding to milk protein. The wavelength 995.8423 nm is likely related to the third overtone of the OH bond vibration, corresponding to water in milk.

[0113] To analyze the fat characteristic wavelengths selected by Selector, we calculated the average spectral reflectance of fat content from 0 (g / 100ml) to 7.0 (g / 100ml) from the overall data, with a fat content growth gradient of 1.0 (g / 100ml). Figure 12 The average spectral reflectance curves of milk at different fat contents are shown. As the fat content increases, the absorption peaks of the spectral curves near 400.4518nm, 438.8641nm, 482.078nm, 520.4902nm-664.5363nm, and 702.9486-789.3763nm become more obvious, making it easy to distinguish the spectral curves with different fat content gradients. Because fat is a macromolecular compound, the higher the fat content near the above wavelength range, the stronger the absorption intensity of the sample. In the near-infrared spectral region, the spectral curves near 875.804-995.8423nm also show different absorption intensities according to different fat content gradients. However, unlike the visible light region, the absorption intensity of the sample in this region gradually decreases as the fat content increases. It can be clearly observed near 928.6208nm and 971.8347nm that the absorption intensity of the spectral curve of the low-fat sample is larger. This may be due to the relatively high water content of the low-fat and skim milk samples. The absorption peak generated by the vibration stretching of the OH bond in water covers the absorption peak generated by the stretching vibration of the CH and NH bonds at nearby wavelengths.

[0114] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0115] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology, characterized in that: The method comprises the following steps: S1: Use a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content; S2: Eliminate interference from light and environmental factors in milk spectral data through preprocessing methods; S3: Joint Learning Feature Selection and Prediction (JLSP) model is used to select characteristic wavelengths and predict fat content in milk spectral data, and compared with partial least squares regression (PLSR), support vector regression (SVR) and other traditional wavelength selection methods (SPA, CARS); The joint learning feature selection and prediction model consists of three parts: Selector, Predictor and Baseline: Selector selects spectral features through a one-dimensional convolutional neural network (1D-CNN) to generate a feature vector after dimensionality reduction; Predictor predicts fat content based on the feature vector after dimensionality reduction; Baseline improves the ability to identify key information through weighted feature vectors; The model adopts a joint learning strategy and uses the back-propagation algorithm to update all network parameters; The joint learning strategy trains three neural networks simultaneously. The Selector receives a full spectrum output vector x of dimension d and generates a probability vector p of the same dimension. Each value of p is a probability value between 0 and 1, indicating the possibility of selecting the corresponding wavelength in the input vector. Bernoulli sampling is performed using the probability vector p to obtain a selection vector s of dimension d. S consists only of 0 and 1. 0 indicates that the wavelength at the corresponding position is not selected, and 1 indicates that the wavelength at the corresponding position is selected. The selection vector s is multiplied by the original input vector x to obtain the selected spectral data. It is defined as; Multiply the probability vector by the original input vector x to obtain the weighted spectral data Predictor receives the selected spectral data And perform one iteration of training; in order to evaluate The performance of the Predictor network is calculated using the MSE loss function; this loss can be used to update the Predictor network parameters through back propagation; in addition, the Baseline receives weighted spectral data And perform one iterative training; use the MSE loss function to calculate the Baseline loss to evaluate the feature representation Performance on the Baseline network; this loss is used on the one hand to update the network parameters of the Baseline network in back propagation, and on the other hand to calculate the Reward in combination with the Predictor loss; the calculation formula of the Reward is as follows, Reward=-(L predictor -L baseline ) (3) Reward value represents x* and The performance of the network is quantified during iterative training. At the same time, Reward will participate in the calculation of Selector loss. If Reward is positive, it means Better than in feature representation This further indicates that the Selector has performed effective feature selection and encourages the Selector to maintain the current feature selection strategy; if the Reward is negative, it means The loss of some spectral information in the feature representation further indicates that the Selector does not perform effective feature selection. According to the Reward value, the Selector is given a corresponding penalty item and the current feature selection strategy is adjusted. The Selector loss function consists of three parts: Reward, L cross And L1 regularization, the specific calculation formula is as follows; Among them, L cross It represents the cross entropy loss between the feature selection probability p output by Selector and the feature selection vector s; L1 norm is the sum of the absolute values ​​of each element in the vector; λ is the regularization parameter, which is used to control the weight of the regularization term in the entire loss; the optimization trend of Selector is affected by Reward. When the Reward value is positive, it means that the current wavelength selection result is good; in the back propagation process, by minimizing L selector To encourage the maintenance of the current wavelength selection strategy and further optimize the strategy; when the Reward value is negative, it means that the current wavelength selection result is poor; in the back propagation process, by maximizing L selector To punish the current wavelength selection strategy and search for a new strategy.

2. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, wherein The hyperspectral imaging system consists of a hyperspectral imager, a whiteboard, a halogen lamp, a computer and corresponding supporting control software; The hyperspectral image data has a spectral wavelength range of 400-1000 nm, a spectral resolution of 4.8 nm, and 750 spectral channels; Data were collected in a darkroom with a 50W halogen lamp as the light source, a 30cm distance between the lens and the stage, and a system exposure time of 10ms; The spectrometer needs to be preheated for 30 minutes before data collection. The data is processed using the black and white correction formula. The formula is as follows: the original image data is subtracted from the blackboard data, and the result is divided by the value obtained by subtracting the whiteboard data from the blackboard data to obtain the corrected image data.

3. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, wherein The experimental conditions for determining fat content are: Milk samples should be stored in an environment with a temperature of 28°C and a humidity of 17%. They should be shaken thoroughly before testing. The fat content of milk samples was determined using the Foss milk composition analyzer MilkoScan FT120; The milk samples included 83 samples from 14 brands, all within the shelf life. Each sample was tested 5 times, with 3 samples each time, for a total of 1,245 samples.

4. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, wherein In step S2, the spectral data is preprocessed using the following method: In the SVR and JLSP models, the second-order derivatives are used to preprocess the spectral data; In the PLSR model, the original spectral data are directly used for analysis.

5. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 4, wherein The Selector network is a one-dimensional convolutional neural network 1D-CNN, including: Three convolutional layers with a kernel size of 3, output channels of 32, 64, and 128, and strides of 2, 1, and 1, respectively; Each convolution is followed by a maximum pooling layer, and finally a 128×5 feature representation is obtained; After flattening the features, they pass through two fully connected layers with 256 and 125 neurons respectively; The output layer uses the Sigmoid activation function to generate a probability vector.

6. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, wherein The PLSR model reduces the dimension of milk spectral data while retaining important information by extracting latent variables; the SVR model processes nonlinear data relationships through kernel functions and searches for the best fitting hyperplane in high-dimensional space.

7. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, wherein SPA selects characteristic wavelengths by the size of the projection vector; CARS combines Monte Carlo sampling with the PLS model for wavelength analysis, and screens the optimal wavelength combination through the exponential decay method and regression coefficient.

8. A milk fat content prediction system based on the milk fat content prediction method based on deep learning combined with hyperspectral imaging technology according to any one of claims 1 to 7, characterized in that: The system specifically includes: The image acquisition and content determination module uses a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content; The preprocessing module is connected to the image acquisition and content determination module, and eliminates the interference of light and environmental factors on the milk spectral data through preprocessing methods; The comparison module is connected to the preprocessing module and uses JLSP to select characteristic wavelengths and predict fat content of milk spectral data, and compares them with traditional prediction models PLSR and SVR and traditional wavelength selection methods SPA and CARS.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the milk fat content prediction method based on deep learning combined with hyperspectral imaging technology as described in any one of claims 1 to 7.

Images