Milk fat content prediction method and system based on deep learning combined with hyperspectral imaging technology

Abstract

The invention belongs to the technical field of hyperspectral imaging, and discloses a milk fat content prediction method based on deep learning in combination with a hyperspectral imaging technology, and the method comprises the following steps: collecting milk hyperspectral image data covering the whole shelf life by using a hyperspectral imaging system, and measuring the fat content; interference of illumination and environmental factors on milk spectral data is eliminated through a preprocessing method; characteristic wavelength selection and fat content prediction are carried out on milk spectral data by using JLSP, and comparison with traditional prediction models PLSR and SVR and traditional wavelength selection methods SPA and CARS is carried out. The feasibility of predicting the fat content on a milk hyperspectral data set by using a deep learning method is discussed. A milk hyperspectral image data set covering the whole shelf life and a corresponding fat content index are constructed, and the problem that the data set is not comprehensive due to the fact that nutritional ingredients of milk samples are different in different storage periods is solved.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of hyperspectral imaging technology, and in particular relates to a method and system for predicting milk fat content based on deep learning combined with hyperspectral imaging technology. Background Art

[0002] Milk is a nutritious natural food that contains essential nutrients for mammal cubs and humans. In recent years, milk has become an important part of the human diet. People of different ages have different demands for milk fat content, so more and more types of milk have appeared on the market. Teenagers often choose whole milk due to their physical growth needs; some adults choose low-fat milk for fitness and fat loss; special people with liver and gallbladder diseases choose skim milk to ensure their health. At the same time, the fat content in milk also affects the color, taste and mouthfeel of milk, which makes fat have unique characteristics in the nutrition and sensory aspects of dairy products such as butter, cream and cheese. Therefore, it is particularly important to quickly and accurately detect the fat content of milk.

[0003] Hyperspectral imaging is an advanced optical imaging technology that uses hundreds or even thousands of narrow-band spectral channels to cover the visible and infrared spectral ranges. The spectral absorption of organic matter in this region is mainly the frequency doubling and combination frequency absorption of hydrogen-containing groups. The positions and intensities of the spectral absorption peaks produced by different groups are different. As the content of sample components changes, its spectral characteristics will also change. Since organic matter in milk contains these hydrogen-containing groups, hyperspectral imaging can be used to quickly determine the composition of milk. Zhao et al. used hyperspectral imaging technology to determine the fat content in milk, and established PLSR and multidimensional partial least squares regression models (N-PLSR) by selecting regions of interest (ROI). The results showed that the performance of N-PLSR was significantly better than that of the PLSR method. Milk is a complex colloidal mixture, so its spectrum includes reflectance and transmission spectra. Luo et al. used hyperspectral imaging technology to perform linear regression analysis on the absorption coefficient and reflectance coefficient of milk samples at continuous wavelengths and the corresponding fat content in milk. The results showed that spectral reflectance was more accurate in predicting milk fat content. French scholars Coppa and others used near-infrared reflectance spectroscopy to predict the composition of fatty acids in milk, and predicted the contents of saturated fatty acids, unsaturated fatty acids, lauric acid, etc. in milk by improving the partial least squares regression model. It was proved that near-infrared reflectance spectroscopy has a high correlation with most fatty acids in milk.

[0004] Hyperspectral image data has continuous wavelengths and a large number of wavelengths, carrying a large amount of milk-related information. This also leads to high dimensionality and high redundancy of hyperspectral image data. In order to further improve the performance of the model, it is often necessary to select wavelengths that are highly correlated with the prediction indicators. However, milk is a liquid with complex ingredients, which causes serious band overlap in hyperspectral image data. That is, in the same band, there is both fat information and water, protein and other information. This brings difficulties to feature selection. In addition, hyperspectral image data is easily disturbed by light and environmental factors. If the spectral curve is not preprocessed sufficiently, it is difficult to solve the problem of spectral data redundancy when selecting characteristic wavelengths. Some researchers have tried to use various preprocessing methods combined with feature selection methods to improve the accuracy of the prediction model. Xu et al. used hyperspectral imaging technology (400-1000nm) to predict the protein and fat content in milk, and performed convolution smoothing and first-order derivative preprocessing on the original spectral curve, which significantly improved the model prediction accuracy. On this basis, an improved spatial frog leaping feature selection method was proposed, and partial least squares regression (PLSR) and voting regression (VR) protein and fat content prediction models were established respectively. The determination coefficients of the PLSR model were 0.8623 and 0.9608, and the determination coefficients of the VR model were 0.9607 and 0.9834, respectively. To a certain extent, the redundancy problem of hyperspectral data was solved and the stability of wavelength selection was enhanced. Huang et al. explored the characteristic wavelengths related to milk fat content based on linear and nonlinear relationships. A wavelength selection method based on an improved ant colony-genetic algorithm was proposed. The results show that the two models have their own advantages. The support vector regression model achieved the highest accuracy of 0.9869, but the detection efficiency was low. The detection efficiency of the multivariate linear regression model is relatively ideal, with a running time as low as 0.02s, but the prediction accuracy is slightly lower than that of the SVR model. The research potential of the nonlinear relationship between milk spectral reflectance data and fat properties is further revealed.

[0005] When most researchers estimate the nutritional content of milk, they often only collect milk data from the same period, without considering that the freshness of milk samples changes over time. There are also some differences in the nutritional content of milk stored at different periods. For example, the fat in milk will decompose and oxidize during storage. Oxidation will destroy the structure of fat molecules, resulting in a decrease in the fat content of milk; vitamin A and vitamin D in milk are easily affected by light and oxidation, resulting in a decrease in vitamin content. Therefore, it is necessary to take the storage time factor into account in the experiment. This experiment collected hyperspectral image data covering the entire shelf life of milk. The data collection time lasted from January 2024 to June 2024. This full-cycle data is more conducive to researchers analyzing nutritional indicators such as fat, protein, and carbohydrates in milk that are easily affected by freshness. At the same time, considering the large number of samples and the relatively large amount of data, we introduced a deep learning model to analyze milk hyperspectral data.

[0006] The deep learning model has a multi-layer neural network structure and nonlinear activation functions, which can better capture the complex nonlinear relationships in the data. Deep learning, also known as representation learning, can automatically learn feature representations from data layer by layer, with high flexibility and expressiveness. In addition, deep learning allows the construction of an end-to-end model learning framework, and the entire process from raw input to final output is automatically learned by the model, simplifying the modeling process. Therefore, compared with traditional machine learning methods, deep learning is more suitable for processing large-scale hyperspectral data.

[0007] There are many kinds of milk on the market nowadays. Different kinds of milk have different nutritional components and different reflectivities in different wavelength ranges. When selecting features for a certain nutritional indicator, the influence of the content of other indicators should be considered. Not only global features should be considered, but also how to select effective characteristic wavelengths for different types of milk samples should be considered. It is crucial to understand which features are most relevant to the model output. Therefore, in order to explore the potential of combining deep learning with hyperspectral imaging technology to estimate milk fat content, we proposed a Selector and Predictor neural network (JLSP) based on joint learning, which can simultaneously complete the prediction of milk fat content and the selection of characteristic wavelengths. The present invention compares JLSP with traditional SVR, PLSR prediction models and SPA, CARS feature selection methods to explore the nonlinear relationship between milk hyperspectral data and fat content based on deep learning. Provide a new method for subsequent non-destructive testing of the nutritional quality of milk. Summary of the invention

[0008] In response to the problems existing in the prior art, the present invention provides a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology.

[0009] The present invention is implemented as follows: a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology, the method comprising:

[0010] S1: Hyperspectral imaging system was used to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content;

[0011] S2: The interference of light and environmental factors on the milk spectral data was eliminated through preprocessing methods;

[0012] S3: JLSP is used to select characteristic wavelengths and predict fat content of milk spectral data, and compared with traditional prediction models PLSR, SVR and traditional wavelength selection methods SPA, CARS.

[0013] Furthermore, the hyperspectral imaging system is mainly composed of a hyperspectral imager, a whiteboard, a halogen lamp, a computer and corresponding supporting control software;

[0014] The spectral wavelength range measured by the hyperspectral image data is 400-1000nm, the spectral resolution is 4.8nm, and the number of spectral channels is 750; the experiment was conducted in a dark room, using a 50W halogen lamp to simulate natural light, setting the distance between the stage and the lens to 30cm, and the system exposure time to 10ms. In order to eliminate baseline drift, the spectrometer needs to be preheated for 30 minutes before measurement; during the experiment, pure milk is poured into a glass culture dish, the culture dish is placed in the center of the stage, and black flannel is arranged around it to avoid interference from irrelevant reflection sources; when collecting spectral data, due to the uneven intensity distribution of the light source in different bands, the presence of dark current and some ambient light in the hyperspectral imager, the collected image is accompanied by certain noise, and black and white correction processing is required to try to eliminate the influence of objective conditions; the black and white correction formula is as follows:

[0015]

[0016] Among them, R is the image data after black and white correction, W is the whiteboard data, B is the blackboard data, and I is the original image data.

[0017] Furthermore, the experiment for determining the fat content is to store the milk sample in an environment with a temperature of 28°C and a humidity of 17%. Considering that some proteins and fats in the liquid milk will form flocculent substances and precipitate, the milk needs to be shaken before measurement. The experiment uses the Foss milk composition analyzer MilkoScan FT120 to determine the fat content of the milk sample.

[0018] The milk samples were obtained from 14 liquid milk brands including Yili, Mengniu, Bright, Sanyuan, etc., including pure milk, low-fat milk, skimmed milk, high-calcium milk, organic milk, etc., totaling 83 types of samples; all types of milk were within the shelf life of 6 months. In order to cover the freshness range of all milk, a total of 5 tests were conducted on each type of milk, once every month, 3 samples were tested each time, and a total of 1,245 samples were tested.

[0019] Furthermore, the S2 specifically includes: using a second-order derivative preprocessing method in the SVR and JLSP models to preprocess the spectral data and perform subsequent analysis, and using the original spectral data in the PLSR model to perform subsequent analysis.

[0020] Furthermore, the JLSP model is an end-to-end deep learning model framework that can perform feature selection and regression prediction tasks simultaneously. Inspired by INVASE, JLSP consists of three neural networks, namely Selector, Predictor and Baseline. Selector is a feature selection network used to learn feature representations related to milk fat content. Predictor and Baseline are regression prediction networks used to predict milk fat content. d is a eigenvector of dimension d, x d Will participate in the input of all neural networks; among them, Selector receives the original features x of the sample d And output a probability vector p d ; Perform Bernoulli sampling based on the probability vector to obtain a selection vector s with only 0 and 1 d On the one hand, the original feature x d With the selection vector s d Multiply to get the features after dimensionality reduction This is an instance-based feature selection method that can flexibly select different numbers of feature wavelengths for each milk sample among many types of milk; Predictor receives the feature vector after dimensionality reduction by Selector And output the predicted value y p On the other hand, the original feature x d With the probability vector p d Multiply to get the weighted eigenvector By weighting, the model can focus on the important parts of the data while maintaining the original number of features, thereby improving the ability to identify key information. Baseline receives the weighted feature vector And output the predicted value b; the model is updated using a joint learning strategy, in which all networks are trained through back propagation; the Predictor network and the Baseline network can directly calculate the mean square error MSE loss through the predicted value; the loss of the Selector network is calculated jointly by the losses of the Predictor and Baseline networks.

[0021] Furthermore, the Selector network is a one-dimensional convolutional neural network 1D-CNN, which consists of an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; 1D-CNN extracts local features in the input data by sliding the convolution kernel; this enables the model to capture feature information between adjacent wavelengths. And the convolution kernel shares parameters on the entire input data, which reduces the amount of parameters that need to be learned in the network, makes the network more lightweight, and reduces the risk of overfitting. Compared with traditional feature selection methods, 1D-CNN can extract features from spectral data layer by layer through multi-layer nonlinear modules, with higher learning efficiency and stronger generalization ability; Selector has three convolution layers, the convolution kernel size is 3, and the number of output channels is 32, 64 and 128 respectively; considering that the feature dimension of a single sample is 1×125, the scanning step size in the first convolution layer is set to 2, and the scanning step sizes of the other two layers are set to 1; a pooling layer is immediately followed by each convolution operation, and the data in the pooling window is pooled to further compress the feature dimension. After three layers of convolution and pooling operations, a 128×5 feature representation is finally obtained; before entering the fully connected layer, the data of all channels are expanded into a one-dimensional tensor and input into two fully connected layers in sequence. The number of neurons in the two fully connected layers is 256 and 125 respectively; the output layer is connected to a Sigmoid function to ensure that the output is a probability vector with a bounded range between 0 and 1 and the same dimension as the input data.

[0022] Furthermore, the PLSR is a linear regression algorithm, which is mainly used to deal with multicollinearity and high-dimensional data. It maps the relationship between the original spectral reflectance of milk and milk components into a new space by extracting latent variables. It reduces the data dimension while retaining important information in the data. The SVR is a nonlinear regression algorithm, whose main goal is to establish a hyperplane that can find the best fit in high-dimensional space to maximize the boundary between the predicted value and the actual value. The kernel function of SVR has unique advantages in high-dimensional space and can handle nonlinear relationships of sample data, and is particularly suitable for processing hyperspectral data sets with multiple continuous wavelengths.

[0023] Furthermore, the SPA is a feature variable selection method that can map high-dimensional data to low-dimensional space while retaining the main information of the original data; by projecting the wavelength onto other wavelengths, comparing the size of the projection vector, and taking the wavelength with the largest projection vector as the candidate wavelength; combining multiple regression analysis to determine the feature wavelength combination; CARS uses the Monte Carlo sampling method combined with PLS for analysis, taking the regression coefficient as the importance of the wavelength, and using the exponential decay method to determine the number of wavelengths; and obtaining the candidate feature subset through reweighted sampling. Finally, the optimal feature set is determined based on the minimum RMSECV of the PLS model.

[0024] Another object of the present invention is to provide a milk fat content prediction system based on deep learning combined with hyperspectral imaging technology based on the milk fat content prediction method based on deep learning combined with hyperspectral imaging technology, the system specifically comprising:

[0025] The image acquisition and content determination module uses a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content;

[0026] The preprocessing module is connected with the image acquisition and content determination module, and eliminates the interference of light and environmental factors on the milk spectrum data through preprocessing methods;

[0027] The comparison module is connected to the preprocessing module, and uses JLSP to select characteristic wavelengths and predict fat content of milk spectral data, and compares it with traditional prediction models PLSR and SVR and traditional wavelength selection methods SPA and CARS.

[0028] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology.

[0029] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0030] First, the present invention collects hyperspectral image data of milk covering the entire shelf life and measures the fat content. This full-cycle data, on the one hand, is conducive to researchers analyzing nutritional indicators such as fat, protein, and carbohydrates in milk that are easily affected by freshness; on the other hand, the improvement of model performance is conducive to providing consumers with more accurate reference values ​​for indicator content.

[0031] Based on the milk spectral data, a variety of preprocessing methods are compared to eliminate the interference of light and environmental factors and increase the resolution of spectral data. After experimental verification, it is known that the second-order derivative preprocessing method is ideal, which separates the smooth part and the slowly changing part in the original milk spectral data, thereby highlighting the rapidly changing part of the spectral data. In particular, the inflection point and slope change part in the original milk spectral data are emphasized. Both correspond to the changes in the content of macromolecular compounds such as fat and protein in milk. It paves the way for subsequent feature selection.

[0032] The JLSP model proposed in the present invention is an instance-based feature selection method. It solves the problem that the number of features of all samples in the global feature selection method is always the same. For milk of different types and different nutrient contents, JLSP will select effective feature wavelengths for them, and the number of features for each sample is calculated separately and is not limited by the number of global features. For example, the water content in skim milk is higher than that in whole milk, so more water-related wavelengths will be selected in the feature selection process of skim milk. High-protein whole milk contains a large number of macromolecular compounds, and more wavelengths related to NH bond and CH bond stretching vibration will be selected in the feature selection process compared to skim milk.

[0033] This technical solution uses two traditional machine learning algorithms to predict milk fat content, namely the linear regression model PLSR and the nonlinear regression model SVR. This approach can better analyze the relationship pattern between milk spectral data and fat content. The experimental results show that in terms of prediction accuracy, SVR performs better than PLSR, demonstrating the prediction potential of nonlinear regression models on milk hyperspectral data. JLSP, as a deep learning framework, has a multi-layer neural network structure and nonlinear activation function, which further expands the advantages of nonlinear models and can better capture the complex nonlinear relationships in milk spectral data. Experimental results show that JLSP can not only complete the selection of characteristic wavelengths, but also show excellent performance in predicting milk fat content. The prediction effect of JLSP is better than the above traditional methods. Test set R 2 It reaches 0.9734, and the mean square error is 0.0573. In the feature selection part, JLSP will generate n (number of iterations) probability heat maps of size of number of samples × number of features during the training process. Each element represents the probability of the feature being selected in a certain sample, which allows the neural network that originally runs under the black box to visualize the feature representation, increasing the interpretability of the JLSP feature selection results. Subsequent analysis combined with the corresponding chemical bonds proved that JLSP can effectively select the relevant wavelengths of milk fat content. The application of JLSP can also be extended to other nutrients such as milk protein and carbohydrates, providing new methods for quality control and nutritional monitoring of dairy products.

[0034] Second, the technical solution of the present invention fills the technical gap in the industry at home and abroad: the present invention applies the instance feature selection method to the research on milk fat content prediction for the first time, providing a new idea for the subsequent detection of milk nutritional indicators.

[0035] The technical solution of the present invention solves a technical problem that people have been eager to solve but have never succeeded in solving: the current traditional feature selection method has some problems. After calculating the correlation between all wavelengths and fat content and sorting them from high to low, the incremental feature selection method is used to compare the model performance. The results show that the prediction result of milk fat content using full-wavelength spectral data has the highest accuracy. This may be because the selected characteristic wavelength is a global characteristic wavelength, that is, it is selected based on all samples, and no feature selection is performed based on a single sample. The characteristics of all samples are the same, but the milk varieties are different. The global characteristics have affected the prediction of some milk samples to a certain extent.

[0036] The technical solution of the present invention overcomes technical bias: the current research on the prediction of the nutrient content of milk has the problem of incomplete coverage of milk shelf life data. When researchers or testing departments select samples, they often only purchase milk samples within a single shelf life and take them back to the laboratory for testing. When consumers buy milk products, they may buy milk within the shelf life but at different storage periods. The nutrient content of milk at different storage periods will change. Relying solely on the nutrient content label on the milk label or the prediction of the milk component content in a single time period cannot provide consumers with an accurate reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology provided by an embodiment of the present invention;

[0038] Figure 2 is a structural diagram of a hyperspectral imaging system provided by an embodiment of the present invention;

[0039] Figure 3 is the JLSP model structure provided by the embodiment of the present invention;

[0040] Figure 4 It is a Selector network structure provided by an embodiment of the present invention;

[0041] Figure 5 It is the Predictor network structure provided by the embodiment of the present invention;

[0042] Figure 6 This is the ROI selection process provided by the embodiment of the present invention;

[0043] Figure 7is the original spectral reflectance and the pre-processed spectral reflectance provided by the embodiment of the present invention;

[0044] Figure 8 It is the wavelength Pearson correlation analysis provided by the embodiment of the present invention;

[0045] Fig. 9 It is the predicted value and actual value of the Predictor combined with different feature selection methods provided in the embodiment of the present invention;

[0046] Fig.10 are the predicted values ​​and actual values ​​of different prediction models combined with the best feature selection method provided by the embodiment of the present invention;

[0047] Fig.11 It is a comparison of the SPA, CARS and Selector feature selection results provided by the embodiment of the present invention;

[0048] Fig.12 It is the spectral reflectance of milk at different fat contents provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] like Figure 1 As shown, an embodiment of the present invention provides a method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology, the method comprising:

[0051] S1: Hyperspectral imaging system was used to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content;

[0052] S2: The interference of light and environmental factors on the milk spectral data was eliminated through preprocessing methods;

[0053] S3: JLSP is used to select characteristic wavelengths and predict fat content of milk spectral data, and compared with traditional prediction models PLSR, SVR and traditional wavelength selection methods SPA, CARS.

[0054] 1 Materials and methods

[0055] 1.1 Experimental Sample

[0056] The experimental samples were obtained from 14 liquid milk brands including Yili, Mengniu, Guangming, Sanyuan, etc., including pure milk, low-fat milk, skim milk, high-calcium milk, organic milk, etc., a total of 83 samples. All types of milk were within the shelf life of 6 months. In order to cover the freshness range of all milk, a total of 5 tests were conducted on each type of milk, once every month, and 3 samples were tested each time. A total of 1,245 samples.

[0057] 1.2 Determination of milk fat content

[0058] The milk samples were stored in an environment with a temperature of 28°C and a humidity of 17%. Considering that some proteins and fats in liquid milk will form flocculent substances and precipitate, the milk needs to be shaken before measurement. The fat content of milk samples was measured using Foss milk composition analyzer MilkoScan FT120. The statistical values ​​of fat content for all batches of milk samples are shown in Table 1. Table 1 Statistical values ​​of fat content for all batches of milk samples. 1.3 Hyperspectral image data acquisition The hyperspectral imaging system is mainly composed of a hyperspectral imager, a whiteboard, a halogen lamp, a computer and corresponding supporting control software. The structure is as follows Figure 2 The experiment used the model produced by Headwall Company of the United States in 2010. The PTU-D48E hyperspectral imager. The measured spectral wavelength range is 400-1000nm, the spectral resolution is 4.8nm, and the number of spectral channels is 750. The experiment was conducted in a dark room, using a 50W halogen lamp to simulate natural light, setting the distance between the stage and the lens to 30cm, and the system exposure time to 10ms. In order to eliminate baseline drift, the spectrometer needs to be preheated for 30 minutes before measurement. During the experiment, pure milk was poured into a glass culture dish, the culture dish was placed in the center of the stage, and black flannel was arranged around it to avoid interference from irrelevant reflection sources. When collecting spectral data, due to the uneven intensity distribution of the light source in different bands, the presence of dark current and some ambient light in the hyperspectral imager, the collected image is accompanied by a certain amount of noise, and black and white correction processing is required to try to eliminate the influence of objective conditions. The black and white correction formula is as follows: Among them, R is the image data after black and white correction, W is the whiteboard data, B is the blackboard data, and I is the original image data. 1.4 Selector and predictor neural network based on joint learning (JLSP). 1.4.1 Model framework In order to better extract the local features of milk hyperspectral data, we proposed a JLSP model, such as Figure 3 As shown in the figure. This is an end-to-end deep learning model framework that can perform both feature selection and regression prediction tasks. Inspired by INVASE, JLSP consists of three neural networks, namely Selector, Predictor, and Baseline. Selector is a feature selection network used to learn feature representations related to milk fat content; Predictor and Baseline are regression prediction networks used to predict milk fat content. Input sample x d is a eigenvector of dimension d, x d Will participate in the input of all neural networks. Among them, Selector receives the original features x of the sample d And output a probability vector p d . Bernoulli sampling is performed based on the probability vector to obtain a selection vector s with only 0 and 1 d On the one hand, the original feature x d With the selection vector s d Multiply to get the features after dimensionality reduction This is an instance-based feature selection method that can flexibly select different numbers of characteristic wavelengths for each milk sample among many types of milk. Predictor receives the feature vector after dimensionality reduction by Selector. And output the predicted value y p On the other hand, the original feature x d With the probability vector p d Multiply to get the weighted eigenvector By weighting, the model can focus on the important parts of the data while maintaining the original number of features, thereby improving the ability to identify key information. Baseline receives the weighted feature vector And output the predicted value y b The model is updated using a joint learning strategy, where all networks are trained via back-propagation. The Predictor network and the Baseline network can directly calculate the mean square error (MSE) loss through the predicted values; the Selector network's loss is calculated jointly by the Predictor and Baseline network's losses.

[0059] The Selector, Predictor, and Baseline are described in detail below. The Selector network is a one-dimensional convolutional neural network (1D-CNN) consisting of an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer. 1D-CNN extracts local features from the input data by sliding the convolution kernel. This enables the model to capture feature information between adjacent wavelengths. In addition, the convolution kernel shares parameters across the entire input data, which reduces the amount of parameters that need to be learned in the network, making the network more lightweight and reducing the risk of overfitting. Compared with traditional feature selection methods, 1D-CNN can extract features from spectral data layer by layer through multi-layer nonlinear modules, and has higher learning efficiency and stronger generalization ability

[18] .

[0060] like Figure 4 As shown in the figure, the Selector has three convolutional layers with a convolution kernel size of 3 and output channels of 32, 64, and 128, respectively. Considering that the feature dimension of a single sample is 1×125, the scan step in the first convolutional layer is set to 2, and the scan step in the other two layers is set to 1. After each convolution operation, a pooling layer is immediately followed to perform maximum pooling on the data in the pooling window to further compress the feature dimension. After three layers of convolution and pooling operations, a 128×5 feature representation is finally obtained. Before entering the fully connected layer, the data of all channels are expanded into a one-dimensional tensor and input into two fully connected layers in sequence. The number of neurons in the two fully connected layers is 256 and 125, respectively. The output layer is connected to a Sigmoid function to ensure that a probability vector with a bounded range between 0 and 1 and the same dimension as the input data is output.

[0061] Both the Predictor network and the Baseline network are multi-layer perceptrons, but their input data are different. Figure 5 As shown, Predictor receives the feature vector selected by Selector, which is composed of {x i , 0}; Baseline receives the product of the original feature vector and the probability vector directly output by Selector, which is composed of The network structures of the two are the same, both have three hidden layers, with 64, 32, and 16 neurons respectively. The SELU activation function is used between each layer. The SELU function is a nonlinear function that can help the network learn complex nonlinear patterns. At the same time, the function introduces a self-standardization mechanism to keep the mean and variance of the output of each layer close to 1 during the training process, which to a certain extent solves the problems of gradient explosion and gradient vanishing in deep neural networks. Compared with the ReLU activation function, the independent variable of the SELU function takes values ​​close to 0 but not equal to 0 in the range of 0 to negative infinity, which helps to avoid the problem of neuron "death". Both networks have been fully designed from the input layer to the output layer. Therefore, Predictor and Baseline can be independently trained as regression models for predicting fat content.

[0062] 1.4.2 Joint Learning Strategy

[0063] Inspired by the actor-critic algorithm in reinforcement learning

[20] , we propose a joint learning strategy that can train three neural networks simultaneously. Selector receives a full-spectrum output vector x of dimension d and generates a probability vector p of the same dimension. Each value of p is a probability value between 0 and 1, indicating the possibility of selecting the corresponding wavelength in the input vector. Bernoulli sampling is performed using the probability vector p to obtain a selection vector s of dimension d, which consists only of 0 and 1. 0 indicates that the wavelength at the corresponding position is not selected, and 1 indicates that the wavelength at the corresponding position is selected. Multiply the selection vector s with the original input vector x to obtain the selected spectral data x*, where x* is defined as.

[0064]

[0065] Multiply the probability vector by the original input vector x to obtain the weighted spectral data Predictor receives the selected spectral data x* and performs an iterative training. In order to evaluate the performance of x* on the Predictor network, the MSE loss function is used to calculate the Predictor loss. This loss can be used to update the Predictor network parameters through back propagation. In addition, Baseline receives the weighted spectral data And perform one iteration of training. Use the MSE loss function to calculate the Baseline loss to evaluate the feature representation Performance on the Baseline network. This loss is used to update the network parameters of the Baseline network in back propagation, and is also used to calculate the Reward in combination with the Predictor loss. The calculation formula for Reward is as follows:

[0066] Reward=-(L predictor -Lbaseline )#(3)

[0067] Reward value represents x* and The performance of the network is quantified during iterative training. At the same time, Reward will participate in the calculation of Selector loss. If Reward is positive, it means that x* is better than This further indicates that the Selector has performed effective feature selection, and encourages the Selector to maintain the current feature selection strategy. If the Reward is negative, it means that x* has lost some spectral information in the feature representation, which further indicates that the Selector has not performed effective feature selection. According to the Reward value, the Selector is given a corresponding penalty item and the current feature selection strategy is adjusted. It is worth noting that the purpose of the Reward is to try to minimize the loss of the Predictor, rather than to minimize the difference in loss between the Baseline and the Predictor.

[0068] Table 2.JLSP training process

[0069]

[0070]

[0071] The training process of JLSP is shown in Table 2. The validation set is used to evaluate the performance of the Selector, Predictor, and Baseline networks during each iterative training process, and the training continues until the Selector network converges. Since the input features of the Predictor and Baseline change with each update of the Selector, the Predictor and Baseline must be updated after each Selector update. Therefore, the Predictor and Baseline will not converge before the Selector network reaches convergence. The trained Selector and Predictor can be used separately as feature selection models and regression prediction models. Whether the Baseline converges is not of particular concern in this experiment. In the actor-critic algorithm, the Baseline is usually used to reduce the variance of the model and "standardize" the Predictor network. Keeping the training progress of the Baseline and Predictor at the same level helps the training of the Predictor network.

[0072] 1.4.3 Loss Function

[0073] The present invention proposes a joint learning strategy based on three neural networks: Selector, Predictor and Baseline. Therefore, it is necessary to design a loss function for the Selector network. Considering that the Selector loss is related to the difference between the Predictor and Baseline losses, the feature selection probability of the Selector output and the Bernoulli sampling result, the Selector loss function designed by the present invention contains three parts: Reward, L cross And L1 regularization, the specific calculation formula is as follows.

[0074]

[0075] Among them, L cross Represents the cross entropy loss between the feature selection probability p output by the Selector and the feature selection vector s. Cross entropy can compare the difference between two probability distributions. By minimizing the cross entropy loss, the feature selection probability can be continuously approached to 0 or 1, which makes the Bernoulli sampling result more stable, and thus enables the model to better learn label-related features. L1 regularization is a technique that adds the L1 norm of the weight as a penalty term in the loss function of the model. It is used to constrain the complexity of the model during model training to prevent overfitting. The L1 norm is the sum of the absolute values ​​of each element in the vector. λ is the regularization parameter, which is used to control the weight of the regularization term in the entire loss. Compared with L2 regularization, L1 regularization punishes all weights with the same intensity, which makes smaller weights become 0 after being punished. Since a large number of model parameters become 0, these parameters will not appear in the final model, thus achieving the effect of sparsification, which also shows that L1 regularization is conducive to feature selection and increasing model interpretability. Reward is the specific performance of the Selector output on the Predictor and Baseline. The optimization trend of Selector is affected by Reward. When Reward is positive, it means that the current wavelength selection result is good. selector To encourage the current wavelength selection strategy to be maintained and further optimized. When the Reward value is negative, it means that the current wavelength selection result is poor. In the back propagation process, by maximizing L selector The overall loss function of the Selector is obtained by calculating the difference between the feature selection probability and the feature selection vector plus the L1 regularization term and multiplying it by the reward. The advantage of this design is that the matching degree between the feature selection probability and the feature selection vector is continuously improved. On the other hand, the Selector takes into account the sparsity of wavelength selection while searching for the optimal feature wavelength for each instance.

[0076] 1.5 Traditional methods

[0077] 1.5.1 Traditional machine learning algorithms

[0078] This study used two regression models commonly used in traditional spectral analysis, namely support vector regression (SVR) and partial least squares regression (PLSR). PLSR is a linear regression algorithm mainly used to handle the situation of multicollinearity and high-dimensional data. It maps the relationship between the original spectral reflectance of milk and milk components to a new space by extracting latent variables, reducing the data dimension while retaining important information in the data. SVR is a non-linear regression algorithm, and its main goal is to establish a hyperplane that can find the best fit in a high-dimensional space to maximize the margin between the predicted value and the actual value. The kernel function of SVR has unique advantages in high-dimensional spaces, can handle the non-linear relationship of sample data, and is especially suitable for processing hyperspectral datasets with multiple continuous wavelengths.

[0079] 1.5.2 Wavelength selection

[0080] Due to the problems of high redundancy and multicollinearity in hyperspectral image data, it brings difficulties to data analysis. Hyperspectral data with all wavelengths often require a large amount of storage space and increase the model calculation time. Therefore, effective wavelength selection is a key step in traditional hyperspectral data analysis. SPA is a feature variable selection method that can map high-dimensional data to a low-dimensional space while retaining the main information of the original data. By projecting wavelengths onto other wavelengths and comparing the magnitudes of the projection vectors, the wavelength with the largest projection vector is used as the wavelength to be selected. Combining multiple regression analysis to determine the characteristic wavelength combination. CARS analyzes by combining Monte Carlo sampling method with PLS, takes the regression coefficient as the importance of the wavelength, and uses the exponential decay method to determine the number of wavelengths. Candidate feature subsets are obtained through reweighted sampling. Finally, the best feature set is determined according to the minimum RMSECV of the PLS model.

[0081] 1.6 Evaluation metrics

[0082] Based on three model evaluation metrics, namely the correlation coefficient R, the coefficient of determination R 2 and the mean squared error MSE, the performances of the three algorithms were compared. R calculates the similarity using the quotient of the covariance and standard deviation between two variables, and its value range is from -1 to 1. When 0 < R ≤ 1, it indicates that the two groups of variables are positively correlated; when -1 ≤ R < 0, it indicates that the two groups of variables are negatively correlated; when R = 0, it indicates that there is no linear correlation between the two groups of variables. The calculation formula of R is as follows:

[0083]

[0084] where, σX and σ Y represent the standard deviation of variables X and Y respectively, and Respectively represent the mean values ​​of X and Y. This formula calculates the ratio of the covariance of variables X and Y to the product of their respective standard deviations. The standardized covariance is dimensionless.

[0085] R 2 Measures the proportion of variance in the dependent variable that can be explained by the independent variable. The value ranges from 0 to 1. A well-performing model usually has a high R 2 . R 2 The calculation formula is as follows.

[0086]

[0087] where y i represents the true value of the dependent variable, represents the predicted value of the dependent variable, represents the mean of the true value of the dependent variable, and n represents the number of samples.

[0088] MSE represents the average deviation between the predicted value and the true value. Models with good MSE performance have lower MSE values. The calculation formula of MSE is as follows.

[0089]

[0090] where y i is the true value of the dependent variable, is the predicted value of the dependent variable, and n is the sample size.

[0091] In the field of modern milk production and processing, the prediction of milk fat content plays an important role, affecting the production process, product development and marketing strategy of manufacturers. First, manufacturers can use the fat content prediction model to monitor and adjust the fat content in milk in real time to ensure that the product meets the quality standards. Manufacturers can promptly discover potential problems and take corresponding measures to ensure stable product quality. Secondly, the prediction model based on fat content can also promote product innovation and development. Manufacturers can customize milk products with different fat contents, such as low-fat milk, skim milk, etc., according to consumer needs and market trends to meet the health needs and taste preferences of different groups of people. In addition, the combination of hyperspectral imaging technology and deep learning has expanded the application possibilities in many fields. In the agricultural field, this combination can help farmers monitor crop growth, detect pests and diseases, and refine fertilization management, thereby improving crop quality and yield. In the food field, the freshness and quality of food can be evaluated by analyzing the spectral information on the surface of food. In terms of food safety testing, it can help detect harmful substances and microbial contamination in food and provide food safety protection for consumers.

[0092] Relevant evidence of the technical effects achieved by the embodiments of the present invention.

[0093] 1.1 Acquisition of spectral reflectance of milk samples

[0094] like Figure 6 (a) shows the hyperspectral image data of milk samples. The data has three dimensions, including image space and spectral space. The image space consists of 816×1004 pixels, and the spectral space consists of 125 wavelengths in the range of 400-1000nm. This study first selects 450nm, 550nm and 650nm channels from the many channels of hyperspectral image data to form an RGB visualization image, as shown in Figure 6 (b) is shown. RGB is one of the most commonly used color modes and one of the most basic color modes in digital image processing. Especially for images with complex backgrounds, researchers usually need to visualize the hyperspectral image as an RGB image before selecting the area of ​​interest. Figure 6 The red circle in (c) represents the region of interest selected on the hyperspectral data. The spectral reflectance of all pixels within the ROI at each wavelength is calculated to obtain the average spectral reflectance. Figure 6 (d) is the spectral reflectance curve of all milk samples. The dark blue curve represents the average spectral reflectance of all samples, and the light blue area represents the standard deviation of the spectral reflectance of all samples. The present invention will use this spectral reflectance for subsequent data analysis.

[0095] 1.2 Dataset division and parameter setting

[0096] The present invention divides the data set into a training set, a validation set, and a test set in a ratio of 6:2:2. The number of samples is 747, 249, and 249, respectively. This study uses SVR and PLSR as conventional prediction models to compare the performance of different preprocessing methods and feature selection methods on two conventional prediction models. The grid search method is used to optimize the C value, gamma value, and kernel function of SVR, and the number of principal components of the PLSR model is optimized. In order to avoid overfitting, 5-fold cross validation is used to evaluate model performance and generalization ability.

[0097] 1.3 Data Preprocessing

[0098] Data preprocessing is an important step in preparing spectral data to obtain more stable model performance. Figure 7(a) It can be seen that the original spectral curve does not have much noise, but there is a baseline drift phenomenon. The present invention compares four pre-processing methods: multivariate scatter correction (MSC), standard normal transformation (SNV), derivative transformation and convolution smoothing (SG). MSC can eliminate the spectral differences caused by uneven particle size and different scattering levels, and solve the baseline translation and offset by establishing a mathematical model, and then correct the spectrum of each sample. SNV standardizes each column of spectral values ​​of the sample by mean and variance, which can reduce the impact of spectral intensity instability in spectral data. MSC and SNV have similar pre-processing effects and can eliminate baseline drift. The main idea of ​​SG smoothing filtering is to perform polynomial least squares fitting on each point in the sliding window to obtain the best estimate of the smooth point. This method can remove the noise of the spectral data and keep the shape and width of the spectral information unchanged. The basic idea of ​​derivative transformation is to separate the smooth part and the slowly changing part in the original spectral data by performing a derivative operation on the spectral data, thereby highlighting the rapidly changing part in the spectral data. Commonly used derivative transformation methods include first-order derivative (FD) and second-order derivative (SD). The first-order derivative can highlight the peaks and valleys in the original spectral data, while the second-order derivative can highlight the inflection points and slope changes in the original spectral data. Since the spectral curves after first-order derivative and second-order derivative preprocessing will have obvious noise, an experimental combination of these two methods combined with the convolution smoothing method is added to eliminate baseline drift and reduce data noise.

[0099] As shown in Table 3, after model testing, it can be seen that the second-order derivative preprocessing method based on the SVR model has a high accuracy rate. On the one hand, it shows the prediction potential of the nonlinear regression model on milk hyperspectral data, and on the other hand, it proves the effectiveness of the second-order derivative preprocessing. Figure 7 (b) It can be seen that the second-order derivative transform removes the drift that is independent of the wavelength and enhances the peak information of the spectral curve, making the peak clearer and more prominent. This increases the resolution of the spectral data to a certain extent, making the method more accurate. Since there is less noise in the original spectral data, excessive convolution smoothing will lead to the loss of spectral information, which ultimately leads to the unsatisfactory effect of the derivative transform combined with the convolution smoothing method. In addition, since the surface of fresh milk is smooth and homogeneous, the MSC and SNV treatment effects of the scattering phenomenon are not obvious. Therefore, the present invention uses the second-order derivative preprocessing method in the SVR and JLSP models to preprocess the spectral data and perform subsequent analysis, and uses the original spectral data in the PLSR model for subsequent analysis.

[0100] Table 3. Performance Comparison of Different Preprocessing Methods on SVR and PLSR Models

[0101]

[0102] 3.4 Results Analysis

[0103] This study first performed Peason correlation analysis on 125 wavelengths between 400-1000nm. Figure 8 It can be seen that HIS has highly correlated adjacent wavelengths, and the correlation coefficients between adjacent wavelengths are all above 0.95. As the wavelength span increases, the correlation decreases step by step. The correlation coefficient between the wavelengths of 861.3994nm~933.4224nm and the wavelengths before 861.3994nm is about 0.85; the correlation coefficient between the wavelengths after 933.4224nm and the wavelengths of 400.4518nm~746.1625nm is about 0.75. This leads to multicollinearity problems between wavelengths. In order to solve this problem, the characteristic wavelength selection method is used to reduce the dimension of HIS data and reduce the amount of calculation. This dimensionality reduction strategy can significantly improve the accuracy and generalization ability of the model.

[0104] The experiment used full wavelength (125), wavelength selected by SPA (29), wavelength selected by CARS (42) and wavelength selected by Selector (63) to predict milk fat content. The prediction performance of SVR, PLSR and Predictor models was compared. The parameters of SVR and PLSR are the same as the parameter optimization steps in the above preprocessing process. The specific parameters of the JLSP model are set as follows: the learning rate of the three networks is 0.001, λ is set to 0.1, the batch size is 83, and the number of iterations is 10000.

[0105] The experimental results are shown in Table 4. In the traditional method, the prediction results of the SVR model are better than those of the PLSR model in various feature selection methods. This may be because SVR can better fit the nonlinear relationship in hyperspectral data. Among them, the wavelength selected by Selector achieves the highest performance in the SVR model. 2 It is 0.9704. This shows that the feature selection method proposed in this study can effectively match the nonlinear model and improve the prediction accuracy of the model. Fig.10(a) shows the prediction results of the SVR model for the milk fat content of the test set under the Selector method. Overall, most of the prediction points are around the fitting line, which shows that the model has a good fitting effect. However, there are large deviations in individual prediction points near the fat content of 1.0 (g / 100ml). Most of the sample points between 0.0 and 1.0 (g / 100ml) are predicted to be 0.0 (g / 100ml).

[0106] Among the PLSR models, the model using full wavelength achieved the highest performance, with the test set R 2 is 0.9435. This may be because the data lost some spectral information after the feature selection method reduced the dimension, and the PLSR model needs to rely on the linear relationship between wavelength and fat content to achieve prediction. From the number of wavelengths selected by different feature selection methods, it can be seen that the performance index of the PLSR model on the test set is positively correlated with the number of wavelengths. Therefore, it can be judged that the lost spectral information affects the prediction performance of the PLSR model. Fig.10 (b) shows the prediction results of the PLSR model at all wavelengths, which is consistent with Fig.10 Compared with (a), there is also the problem of large errors in individual prediction points. Moreover, most of the sample points near 0.0 (g / 100ml) are predicted as negative values. This is because PLSR does not capture the data features well during the training process, resulting in underfitting of the model.

[0107] When only Predictor is used as a regression predictor, the model performance on full wavelength data and feature selection data is compared. The results show that Predictor is better than PLSR but slightly lower than SVR in traditional feature selection methods. On the one hand, this is because SVR itself has excellent ability to handle nonlinear relationships and high-dimensional data. On the other hand, this is because the design of Predictor is relatively simple when considering the overall model size and parameter amount of JLSP, resulting in slightly lower performance than SVR. However, as shown in Table 4, the wavelength selected by Selector achieves the highest performance, and the test set R 2 The value is 0.9724. This is because the Selector has excellent cross-channel information interaction and integration capabilities, which effectively improves the network expression ability of the Predictor. Especially in hyperspectral data with high correlation between adjacent wavelengths, the Selector can capture the local features of the data, reducing the feature dimension while improving the prediction efficiency of the model. Fig. 9 The prediction point fitting results of using Predictor on full wavelength data and different feature selection data are shown respectively. Fig. 9 (b) It can be seen that the prediction point deviation of the combination based on the SPA method is slightly larger than that of the other three combinations when the fat content is around 4.0-6.0 (g / 100ml). Fig. 9From the sample points near 0.0 (g / 100ml) in (a), 9 (c) and 9 (d), Fig. 9 The prediction point deviation of (d) is the smallest, indicating that the Predictor-Selector combination achieves the highest performance. This combination corresponds to the Selector feature selection method under the Predictor model in Table 4.

[0108] Table 4. Model performance of different feature selection methods

[0109]

[0110] JLSP achieves the highest performance in predicting milk fat content, test set R 2 is 0.9734. The prediction results are as follows Fig.10 As shown in (d), compared with Fig. 9 In (d), Predictor is used alone for training, and JLSP using the joint training strategy achieves better prediction results. Fig.10 (d) The predicted points near the fat content of 4.0-6.0 (g / 100ml) are closer to the fitting curve. This is because the Reward in the joint training strategy gives the Selector timely feedback, so that the Predictor also participates in the feature selection process. The output of the Selector at different training stages increases the diversity of the Predictor input data, thereby improving the generalization ability of the Predictor.

[0111] 1.5 Discussion

[0112] In the discussion section, we try to explain the selection results of the three feature selection methods, SPA, CARS and Selector, by using the wavelength corresponding to the chemical bond. Fig.11As shown in the figure, there are 6 wavelengths selected by the three methods, namely 410.0549nm, 496.4826nm, 818.1885nm, 928.6208nm, 943.0255nm and 957.4301nm. Among them, the wavelength of 928.6208nm is a spectrum band near the third harmonic of the fundamental frequency vibration of the CH chemical bond

[29] ; the wavelengths of 943.0255nm and 957.4301nm may be related to the stretching of the second overtone of the OH chemical bond. The wavelengths selected by Selector are mainly distributed in the peaks or troughs of the spectral curve, which is more conducive to analyzing the changes in the content of different substances in milk. 722.1548nm and 746.1625nm correspond to the spectrum bands near the third harmonics of the stretching vibration of the CH chemical bond of the methyl and methylene groups respectively; the wavelength near 765.3686nm may be related to the water and carbohydrate content; the wavelength between 971.8347 and 986.2393nm is the spectrum band near the second harmonics of the fundamental vibration of the NH chemical bond, corresponding to the protein in milk. 995.8423nm may be related to the third overtone of the OH chemical bond vibration, corresponding to the water in milk.

[0113] In order to analyze the fat characteristic wavelengths selected by Selector, we calculated the average spectral reflectance of fat content from 0 (g / 100ml) to 7.0 (g / 100ml) from the overall data, with a fat content growth gradient of 1.0 (g / 100ml). Fig.12 The average spectral reflectance curve of milk at different fat contents is shown. As the fat content increases, the absorption peaks of the spectral curves near 400.4518nm, 438.8641nm, 482.078nm, 520.4902nm~664.5363nm and 702.9486~789.3763nm become more obvious, and the spectral curves with different fat content gradients can be easily distinguished. Since fat is a macromolecular compound, the higher the fat content near the above wavelength range, the stronger the absorption intensity of the sample. In the near-infrared spectral region, the spectral curves near 875.804~995.8423nm also show different absorption intensities according to different fat content gradients. However, unlike the visible light region, the absorption intensity of the sample in this region gradually weakens as the fat content increases. It can be clearly observed near 928.6208nm and 971.8347nm that the absorption intensity of the spectral curve of the low-fat content sample is larger. This may be due to the relatively high water content of the low-fat and skim milk samples. The absorption peaks generated by the vibration stretching of the OH bond in water cover the absorption peaks generated by the stretching vibrations of the CH and NH bonds at nearby wavelengths.

[0114] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0115] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology, characterized in that: The method comprises the following steps: S1: Use a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content; S2: Eliminate the interference of milk spectral data caused by light and environmental factors through preprocessing methods; S3: The joint learning feature selection and prediction model (JLSP) is used to select feature wavelengths and predict fat content in milk spectral data, and compared with partial least squares regression (PLSR), support vector regression (SVR) and other traditional wavelength selection methods (SPA, CARS).

2. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, characterized in that: The hyperspectral imaging system is composed of a hyperspectral imager, a whiteboard, a halogen lamp, a computer and corresponding supporting control software; The spectral wavelength range of the hyperspectral image data is 400-1000nm, the spectral resolution is 4.8nm, and the number of spectral channels is 750; The data were collected in a dark room, the light source was a 50W halogen lamp, the distance between the lens and the stage was 30cm, and the system exposure time was 10ms; The spectrometer needs to be preheated for 30 minutes before data collection. The data is processed using the black and white correction formula. The specific formula is: subtract the blackboard data from the original image data, and then divide the result by the value obtained by subtracting the whiteboard data from the blackboard data to obtain the corrected image data.

3. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, characterized in that: The experimental conditions for determining fat content are: The milk samples were stored in an environment with a temperature of 28°C and a humidity of 17%, and they needed to be shaken thoroughly before testing; The fat content of milk samples was determined using the Foss milk composition analyzer MilkoScan FT120; The milk samples included 83 samples from 14 brands, all within the shelf life. Each sample was tested 5 times, 3 samples each time, for a total of 1,245 samples.

4. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, characterized in that: In step S2, the spectral data is preprocessed using the following method: In the SVR and JLSP models, the spectral data are preprocessed using the second-order derivative; In the PLSR model, the original spectral data is directly used for analysis.

5. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, characterized in that: The JLSP model consists of three parts: Selector, Predictor and Baseline: Selector selects spectral features through a one-dimensional convolutional neural network (1D-CNN) to generate a feature vector after dimensionality reduction; Predictor predicts fat content based on the feature vector after dimensionality reduction; Baseline improves the key information recognition ability by weighting feature vectors; The model adopts a joint learning strategy and uses the back-propagation algorithm to update all network parameters.

6. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 5, characterized in that: The Selector network is a one-dimensional convolutional neural network 1D-CNN, including: Three convolutional layers, the convolution kernel size is 3, the number of output channels is 32, 64, and 128, and the step sizes are 2, 1, and 1 respectively; Each convolution is followed by a maximum pooling layer, and finally a 128×5 feature representation is obtained; After flattening the features, they pass through two fully connected layers with 256 and 125 neurons respectively; The output layer uses the Sigmoid activation function to generate a probability vector.

7. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, characterized in that: The PLSR model reduces the dimension of milk spectrum data while retaining important information by extracting latent variables; the SVR model processes nonlinear data relationships through kernel functions and searches for the best fitting hyperplane in high-dimensional space.

8. The method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology according to claim 1, characterized in that: SPA selects characteristic wavelengths by the size of the projection vector; CARS combines Monte Carlo sampling with the PLS model for wavelength analysis, and screens the optimal wavelength combination through the exponential decay method and regression coefficient.

9. A milk fat content prediction system based on deep learning combined with hyperspectral imaging technology based on the milk fat content prediction method based on deep learning combined with hyperspectral imaging technology as described in claims 1-8, characterized in that: The system specifically includes: The image acquisition and content determination module uses a hyperspectral imaging system to collect hyperspectral image data of milk covering the entire shelf life and determine the fat content; The preprocessing module is connected with the image acquisition and content determination module, and eliminates the interference of light and environmental factors on the milk spectrum data through preprocessing methods; The comparison module is connected to the preprocessing module, and uses JLSP to select characteristic wavelengths and predict fat content of milk spectral data, and compares it with traditional prediction models PLSR and SVR and traditional wavelength selection methods SPA and CARS.

10. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for predicting milk fat content based on deep learning combined with hyperspectral imaging technology as described in any one of claims 1 to 8.

Images