Rapid detection method for quality grade of fresh milk
Through near-infrared spectroscopy technology and spectral feature selection algorithm, a detection model for raw milk quality level was constructed, which solved the problems of complex and high cost of raw milk quality detection in the existing technology, and achieved rapid, accurate and comprehensive evaluation of raw milk quality level.
Patent Information
- Application Number
- CN202411830431.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-05-13
AI Technical Summary
The existing raw milk quality detection methods are complex and costly, and it is difficult to meet the needs of large-scale sample testing or online testing. Most of them are single indicator testing, so comprehensive evaluation cannot be carried out.
Near infrared spectroscopy technology combined with spectral feature selection algorithm is used to collect near infrared spectroscopy data of fresh milk samples, pre-process and spectral variable selection, and a detection model for raw milk quality level is constructed to achieve a comprehensive evaluation of protein content, fat content and somatic cell number.
It realizes rapid detection of the quality grade of raw milk, reduces the testing cost, and can quickly and accurately evaluate the raw milk to meet the needs of large-scale sample testing in actual production.
Smart Images
Figure CN119985389A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of livestock product quality and safety detection, and in particular relates to a method for rapid detection of raw milk quality grade. Background Art
[0002] Fresh milk is the source of dairy product safety, and its quality is the key to ensuring the development of the dairy industry. Fresh milk quality evaluation is crucial to both animal husbandry and dairy manufacturing. Existing fresh milk evaluation mainly uses physical and chemical methods to analyze and detect indicators such as the main components of fresh milk, somatic cell count, and colony count. These methods can accurately detect the content of various indicators in fresh milk, but the detection process is complicated and requires professionals to undergo a long period of detection, which is difficult to meet the needs of large-scale sample detection or online detection in actual production. At present, large enterprises or professional institutions use proprietary instruments and equipment to detect indicators for fresh milk evaluation, such as somatic cell count detectors and dairy component analysis instruments. The prices of these instruments are usually around hundreds of thousands, and the cost of the instruments is relatively high. Near-infrared spectroscopy analysis technology has been widely used in various industries due to its low cost and rapidity. There have been many studies on near-infrared rapid detection methods for raw milk components, somatic cells, illegal additives, etc., and portable near-infrared spectroscopy detection instruments for components such as protein and fat have also appeared. However, the characteristics of existing studies are to detect a single indicator and establish a content analysis model. For example, testing the protein content, fat content, and somatic cell count in raw milk. Such testing methods are limited in their application scenarios in actual production. From the perspective of the dairy manufacturing industry, it is meaningful and necessary to conduct a comprehensive evaluation of the quality of raw milk from multiple perspectives. For actual production applications, a rapid detection method for the quality grade of raw milk is needed. Summary of the invention
[0003] In order to solve the above technical problems, the present invention proposes a method for rapid detection of raw milk quality grade, which speeds up the detection speed and reduces the detection cost.
[0004] To achieve the above object, the present invention provides a method for rapid detection of raw milk quality grade, comprising:
[0005] Collecting raw milk samples, measuring the protein content, fat content and somatic cell count of the raw milk samples, and classifying the raw milk samples into three grades: high-quality milk, qualified milk and unqualified milk;
[0006] Collecting near infrared spectral data of the raw milk sample;
[0007] Preprocessing the near infrared spectrum data of the raw milk sample to obtain sample preprocessing data;
[0008] Perform spectral variable selection on sample preprocessing data to obtain spectral variables involved in building a raw milk quality grade detection model;
[0009] Constructing a raw milk quality grade detection model, wherein the raw milk quality grade detection model is trained using the spectral variables and the corresponding grades;
[0010] A raw milk sample to be tested is obtained, and the raw milk quality grade detection model is input to obtain a quality grade result of the raw milk sample to be tested.
[0011] Optionally, collecting near infrared spectroscopy data of the raw milk sample includes:
[0012] The measuring instrument was calibrated and preheated for 1 hour. The raw milk sample was kept at 40°C. The near-infrared spectral data were collected using the integrating sphere diffuse reflectance method to obtain a resolution of 4 cm -1 And it covers 2075 spectral variables in the range of 833 to 2500nm.
[0013] Optionally, the sample spectral data is preprocessed using a standard normal transformation method.
[0014] Optionally, spectral variable selection is performed on sample preprocessing data including:
[0015] The XGBoost model was used to establish the relationship between sample spectral analysis data and raw milk grade. The importance of sample spectral variables to raw milk grade was calculated during model construction. The exponential decay recursive feature elimination method was used to remove spectral variables with low importance values. The genetic algorithm was then used to further optimize and screen the remaining spectral variables to obtain the spectral variables involved in building a raw milk quality grade detection model.
[0016] Optionally, building a raw milk quality grade detection model includes:
[0017] The spectral variables involved in constructing the raw milk quality grade detection model are used to input the set support vector machine model to construct the raw milk quality grade detection model.
[0018] Optionally, the method for calculating the variable importance of the sample preprocessing data in the model building process is:
[0019] The feature importance is evaluated by three methods: average information gain AG, the total number of times the feature is used as a segmentation sample in all trees during the training process FS, and the average coverage range AC of the sample. XGBoost is an ensemble model composed of n trees. The objective function of the ensemble model is expressed as:
[0020]
[0021] Where L is the loss function, To penalize the model complexity regularization term, the loss function is expressed as:
[0022]
[0023] Where n is the number of samples, y s , are the label value and predicted value of the kth sample respectively, which are expressed as follows during the tth iteration:
[0024]
[0025] in, is the estimated value of the first t-1 trees known The estimated value f of the function to be learned in this iteration t (x s ); after adding the error term, the objective function is expressed as:
[0026]
[0027] The second-order Taylor expansion of the objective function is approximately expressed as:
[0028]
[0029] Among them, g s ,h s Represents the sample s loss function respectively The first and second derivatives of , deleting the known constants, the objective function is simplified to:
[0030]
[0031] Ω(f t ) consists of the number of leaf nodes T and the corresponding weight ω i Decision, expressed as:
[0032]
[0033] Among them, γ and λ are hyperparameters that control the number and weight of leaf nodes respectively;
[0034] Define the sample set of the i-th leaf node as S i ={s|q(x s )=i}, the objective function is expressed as:
[0035]
[0036] To simplify the formula, let For a fixed structure q (x) , optimal weight of leaf node i It is expressed as: The corresponding optimal value is expressed as:
[0037]
[0038] Assuming that after the dth node is split based on a split point of a certain feature, the sample sets of the parent node, left node, and right node are S, SL, and SR respectively, and S = SL ∪ SR, then the gain after the split is expressed as:
[0039]
[0040] d Coverage of node features Conver d ,Conver d =|S|, |S| represents the number of samples covered by the node;
[0041] According to the splitting process, XGBoost uses the total number of split samples FS, the average information gain AG, and the average coverage of samples AC as indicators to measure the importance of features. The variable importance of the sample preprocessing data is expressed as:
[0042] FS=|D|,
[0043]
[0044] Where D is the set of all nodes in all XGBoost trees that use D features as the basis for splitting data, |D| is the number of nodes in the set, and Gain d is the gain after node d is split, Cover d is the coverage of the d-node feature.
[0045] Optionally, exponential decay recursive feature elimination is performed based on the variable importance of the sample preprocessing data in the model building process, and the ratio of eliminated variables is calculated as:
[0046] ED=a·e -bx
[0047] Among them, a is the amplitude parameter, b is the decay rate parameter, and x is the independent variable determined by the number of remaining features.
[0048] Optionally, the method of performing genetic algorithm processing based on the variables after eliminating the variables includes:
[0049] Binary coding, uniform crossover, tournament selection and bit flipping mutation strategies are adopted. The initial population size is 25, the number of individuals participating in the competition is 2, the crossover rate is 0.5, the mutation rate is 0.05, and the maximum number of iterations is 50.
[0050] Optionally, the parameters of the set support vector machine model are: C is the penalty coefficient of the objective function, and g is the kernel function coefficient.
[0051] Technical effect of the invention: The invention discloses a method for rapid detection of raw milk quality grade, which effectively solves the variable selection problem in multi-substance detection by using a spectral feature selection algorithm; it not only realizes a comprehensive evaluation of protein content, fat content and somatic cell count, but also realizes rapid and accurate detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0053] Figure 1 This is a schematic diagram of a process for rapid detection of raw milk quality grade according to an embodiment of the present invention;
[0054] Figure 2 Schematic diagram of near infrared spectrum data of samples of the embodiment of the present invention, wherein (a) is the original spectrum, and (b) is the average spectrum of raw milk of different grades;
[0055] Figure 3 A schematic diagram of sample distribution of a training set and a test set in an embodiment of the present invention;
[0056] Figure 4 Schematic diagram of variable importance of XGBoost under different evaluation indicators in the embodiment of the present invention, where (a) is AG, (b) is AC, and (c) is FS;
[0057] Figure 5 This is a schematic diagram of the iterative process of the XGBoost-EDRFE method according to an embodiment of the present invention;
[0058] Figure 6 This is a distribution diagram of the variable screening results of an embodiment of the present invention;
[0059] Figure 7 It is an iterative change diagram of individual fitness in an embodiment of the present invention;
[0060] Figure 8 Schematic diagram of hybrid variable selection strategy according to an embodiment of the present invention. DETAILED DESCRIPTION
[0061] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0062] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0063] like Figure 1 As shown, this embodiment provides a method for rapid detection of raw milk quality grade, comprising:
[0064] Collect raw milk samples, measure the protein content, fat content and somatic cell count of the raw milk samples, and classify the raw milk samples into three levels: high-quality milk, qualified milk and unqualified milk;
[0065] Collect near infrared spectroscopy data of raw milk samples;
[0066] Preprocessing the near infrared spectrum data of the raw milk sample to obtain sample preprocessing data;
[0067] Perform spectral variable selection on sample preprocessing data to obtain spectral variables involved in building a raw milk quality grade detection model;
[0068] A raw milk quality grade detection model is constructed, wherein the raw milk quality grade detection model is trained using spectral variables and corresponding grades;
[0069] Obtain a raw milk sample to be tested, input the raw milk quality grade detection model, and obtain the quality grade result of the raw milk sample to be tested.
[0070] Furthermore, before collecting the near-infrared spectral data of the raw milk sample, the measuring instrument needs to be calibrated and preheated for 1 hour. The raw milk sample is kept at a constant temperature of 40°C, and the near-infrared spectral data are collected using the integrating sphere diffuse reflectance method to obtain a resolution of 4 cm -1 And it covers 2075 spectral variables in the range of 833 to 2500nm.
[0071] Furthermore, the sample near-infrared spectral analysis data is preprocessed using a standard normal transformation method to obtain the sample spectral analysis data after noise removal.
[0072] Furthermore, spectral variables are selected based on the sample preprocessing data, including: using the XGBoost model to establish the relationship between the sample spectral analysis data and the raw milk grade, calculating the importance of the sample spectral variables to the raw milk grade during the model construction process, combining the exponential decay recursive feature elimination method to remove spectral variables with low importance values, and then using the genetic algorithm to further optimize and screen the remaining spectral variables to obtain the spectral variables involved in building the raw milk quality grade detection model.
[0073] Specifically, Figure 8The variable selection strategies shown include:
[0074] The present invention measures the importance of features by the average information gain (AG) brought by a feature as a split node in all trees of XGBoost, the total number of times (FS) that the feature is used as a split sample in all trees during the training process, and the average coverage of samples (AC). The information gain here is not the information gain used only for classification decision trees in a narrow sense. It is measured by the loss reduction before and after the node split when the feature is used as a partitioning attribute, that is, the split gain. XGBoost is regarded as an integrated model composed of n trees, and its objective function is expressed as:
[0075]
[0076] Where L is the loss function, To penalize the model complexity regularization term, the loss function is expressed as:
[0077]
[0078] Where n is the number of samples, y s , are the label value and predicted value of the kth sample respectively. When performing the tth iteration,
[0079]
[0080] in, is the estimated value of the first t-1 trees known The estimated value f of the function to be learned in this iteration t (x s ). After adding the error term, the objective function is expressed as:
[0081]
[0082] The second-order Taylor expansion of the objective function is approximately expressed as:
[0083]
[0084] Among them, g s ,h s Represents the sample s loss function respectively The first and second derivatives of , deleting the known constants, the objective function is simplified to:
[0085]
[0086] Regularization term Ω(f t) can control the complexity of the model, which helps to smooth the learned weights and reduce the risk of overfitting. Ω(f t ) consists of the number of leaf nodes T and the corresponding weight ω i Decision, expressed as:
[0087]
[0088] Where γ and λ are hyperparameters that control the number and weight of leaf nodes respectively. The more leaf nodes there are, the more complex the model is. The sample set of the i-th leaf node is defined as S i ={s|q(x s )=i}, the objective function is expressed as:
[0089]
[0090] To simplify the formula, let For a fixed structure q (x) , optimal weight of leaf node i It can be expressed as: The corresponding optimal value is expressed as:
[0091]
[0092] Assuming that after the dth node is split based on a split point of a certain feature, the sample sets of the parent node, left node, and right node are S, SL, and SR respectively, and S = SL ∪ SR, then the gain after the split is expressed as:
[0093]
[0094] d Coverage of node features d , Cover d =|S|, |S| represents the number of samples covered by the node.
[0095] According to this splitting process, XGBoost uses the total number of split samples FS, the average information gain AG, and the average coverage of samples AC as indicators to measure the importance of features. The variable importance of the sample preprocessing data is expressed as:
[0096] FS=|D|,
[0097]
[0098] Where D is the set of all nodes in all XGBoost trees that use this feature as the basis for splitting data, |D| is the number of nodes in the set, and Gain d is the gain after node d is split, Cover d is the coverage of the d-node feature.
[0099] Furthermore, exponential decay recursive feature elimination is performed based on the variable importance of sample preprocessing data in the process of XGBoost model construction to eliminate variables with low importance rankings. The ratio of eliminated variables is calculated as:
[0100] ED=a·e -bx
[0101] Among them, a is the amplitude parameter, b is the decay rate parameter, and x is the independent variable determined by the number of remaining features, where a=0.5 and b=-5.
[0102] Furthermore, based on the change in the accuracy of the XGBoost model during the exponential decay recursive feature elimination process, the optimal preliminary variable screening results were selected.
[0103] Furthermore, the construction of the raw milk quality grade detection model includes:
[0104] The raw milk quality grade detection model is constructed by using the spectral variables involved in constructing the raw milk quality grade detection model to input the set support vector machine model.
[0105] Further, the method of performing genetic algorithm processing based on the variables after eliminating the variables includes:
[0106] Using binary coding, uniform crossover, tournament selection and bit flip mutation strategies, the individual fitness is calculated based on the accuracy of the SVM model and the number of remaining variables, the initial population size is 25, the number of individuals participating in the competition is 2, the crossover rate is 0.5, the mutation rate is 0.05, and the maximum number of iterations is 50. Furthermore, the parameters of the support vector machine model are set as C is the penalty coefficient of the objective function, g is the kernel function coefficient, where C = 222.06, g = 2.47.
[0107] A specific application example of the present invention is as follows:
[0108] 617 raw milk samples of different qualities were collected, and the protein content and fat content were measured using a milk component analyzer, and the somatic cell count was measured using a somatic cell counter. According to the Chinese National Food Safety Standard GB 19301-2010 Raw Milk Standard, the 617 raw milk samples were divided into three grades: high-quality milk, qualified milk, and unqualified milk. The classification standards and sample distribution of each grade are shown in Table 1.
[0109] Table 1
[0110]
[0111] The sample spectral analysis data was collected using a near-infrared spectrometer. The original spectra and the average spectra of various types of raw milk were as follows: Figure 2 As shown, Figure 2(a) is the original spectrum, Figure 2 (b) is the average spectrum of raw milk of different grades.
[0112] The Kennard-Stone method was used to divide the raw milk samples into training set and test set in a ratio of 3:1. The sample distribution after set division is as follows: Figure 3 As shown, the training data set effectively covers the main information of the test data set, and the sample data is divided reasonably.
[0113] The XGBoost-EDRFE-GA hybrid variable selection method proposed in this invention is used to select variables for spectral data. First, AG, AC and FS are used as evaluation indicators, and the original spectrum is used to build an XGBoost classification model to obtain the following Figure 4 The variable importance shown is Figure 4 (a) is AG, Figure 4 (b) is AC, Figure 4 (c) is FS.
[0114] In order to accelerate the convergence speed, the variables with low importance are removed. The EDRFE method is combined to iteratively remove the variables with low importance. The results are as follows: Figure 5 As shown in the figure, it can be seen that the performance model under the AG evaluation index has the best performance.
[0115] The results were applied to the GA algorithm to further optimize the preliminary screening results. In order to verify the performance of the hybrid variable selection method, the effect of using GA alone under the full spectrum was compared. XGBoost was used as the classification model to calculate the individual fitness. The results are shown in Figure 6 , Figure 7 As shown in the figure, it can be seen that the XGBoost-EDRFE-GA hybrid variable selection method has achieved better individual fitness and stability than GA under the three indicators. Among them, AG is the optimal indicator when the XGBoost classification model is used to calculate the fitness. The model reaches the optimal level after 46 iterations, and the individual fitness is 0.9507.
[0116] In order to verify the variable selection advantage of this method in dealing with multi-substance detection, the method was compared with the commonly used variable selection methods competitive adaptive reweighting algorithm (CARS), ReliefF, GA and uninformative variable elimination (UVE) and the results are shown in Table 2.
[0117] Table 2
[0118]
[0119]
[0120] As can be seen from Table 2, all variable selections significantly reduced the variable space, among which the XGBoost-EDRFE-GA hybrid method showed the highest screening efficiency.
[0121] XGBoost and SVM were used as classification models, and the selected variables were input into the models. The results are shown in Tables 3 and 4.
[0122] Table 3
[0123]
[0124] Table 4
[0125]
[0126]
[0127] As can be seen from the table, under the XGBoost-EDRFE-GA hybrid variable selection method, both XGBoost and SVM showed the best prediction performance, which was significantly better than the accuracy of modeling under the CARS, UVE and ReliefF variable selection methods. XGBoost (FS) -EDRFE-GA-SVM was determined to be the optimal model for raw milk grade identification in this study. A total of 54 features were screened, and the accuracy of the training set and test set were 0.9783 and 0.9742, respectively.
[0128] The present invention discloses a method for rapid detection of raw milk quality grade, which effectively solves the variable selection problem in multi-substance detection by using a spectral feature selection algorithm; it not only realizes a comprehensive evaluation of protein content, fat content and somatic cell count, but also realizes rapid and accurate detection.
[0129] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for rapid detection of raw milk quality grade, characterized in that: include: Collecting raw milk samples, measuring the protein content, fat content and somatic cell count of the raw milk samples, and classifying the raw milk samples into three grades: high-quality milk, qualified milk and unqualified milk; Collecting near infrared spectral data of the raw milk sample; Preprocessing the near infrared spectrum data of the raw milk sample to obtain sample preprocessing data; Spectral variables are selected for sample preprocessing data to obtain spectral variables involved in building a raw milk quality grade detection model; Constructing a raw milk quality grade detection model, wherein the raw milk quality grade detection model is trained using the spectral variables and the corresponding grades; A raw milk sample to be tested is obtained, and the raw milk quality grade detection model is input to obtain a quality grade result of the raw milk sample to be tested.
2. The method for rapid detection of raw milk quality grade according to claim 1, characterized in that: The near infrared spectroscopy data collected from the raw milk sample includes: The measuring instrument was calibrated and preheated for 1 hour. The raw milk sample was kept at 40°C. The near-infrared spectral data were collected using the integrating sphere diffuse reflectance method to obtain a resolution of 4 cm -1 And it covers 2075 spectral variables in the range of 833 to 2500nm.
3. The method for rapid detection of raw milk quality grade according to claim 1, characterized in that: The sample spectral data were preprocessed using the standard normal transformation method.
4. The method for rapid detection of raw milk quality grade according to claim 1, characterized in that: Spectral variable selection for sample preprocessing data includes: The XGBoost model was used to establish the relationship between sample spectral analysis data and raw milk grade. The importance of sample spectral variables to raw milk grade was calculated during model construction. The exponential decay recursive feature elimination method was used to remove spectral variables with low importance values. The genetic algorithm was then used to further optimize and screen the remaining spectral variables to obtain the spectral variables involved in building a raw milk quality grade detection model.
5. The method for rapid detection of raw milk quality grade according to claim 1, characterized in that: The construction of raw milk quality grade detection model includes: The spectral variables involved in constructing the raw milk quality grade detection model are used to input the set support vector machine model to construct the raw milk quality grade detection model.
6. The method for rapid detection of raw milk quality grade according to claim 4, characterized in that: The method for calculating the variable importance of the sample preprocessing data in the model construction process is: The feature importance is evaluated by three methods: average information gain AG, the total number of times the feature is used as a segmentation sample in all trees during the training process FS, and the average coverage range AC of the sample. XGBoost is an ensemble model composed of n trees. The objective function of the ensemble model is expressed as: Where L is the loss function, To penalize the model complexity regularization term, the loss function is expressed as: Where n is the number of samples, y s , are the label value and predicted value of the kth sample respectively, which are expressed as follows during the tth iteration: in, is the estimated value of the first t-1 trees known The estimated value f of the function to be learned in this iteration t (x s ); after adding the error term, the objective function is expressed as: The second-order Taylor expansion of the objective function is approximately expressed as: Among them, g s ,h s Represents the sample s loss function respectively The first and second derivatives of , deleting the known constants, the objective function is simplified to: Ω(f t ) consists of the number of leaf nodes T and the corresponding weight ω i Decision, expressed as: Among them, γ and λ are hyperparameters that control the number and weight of leaf nodes respectively; Define the sample set of the i-th leaf node as S i ={s|q(x s )=i}, the objective function is expressed as: To simplify the formula, let For a fixed structure q (x) , optimal weight of leaf node i It is expressed as: The corresponding optimal value is expressed as: Assuming that after the dth node is split based on a split point of a certain feature, the sample sets of the parent node, left node, and right node are S, SL, and SR respectively, and S = SL ∪ SR, then the gain after the split is expressed as: d Coverage of node features d , Cover d =|S|, |S| represents the number of samples covered by the node; According to the splitting process, XGBoost uses the total number of split samples FS, the average information gain AG, and the average coverage of samples AC as indicators to measure the importance of features. The variable importance of the sample preprocessing data is expressed as: FS=|D|, Where D is the set of all nodes in all XGBoost trees that use D features as the basis for splitting data, |D| is the number of nodes in the set, and Gain d is the gain after node d is split, Cover d is the coverage of the d-node feature.
7. The method for rapid detection of raw milk quality grade according to claim 4, characterized in that: Based on the variable importance of the sample preprocessing data in the model construction process, exponential decay recursive feature elimination is performed, and the ratio of eliminated variables is calculated as: ED=a·e -bx Among them, a is the amplitude parameter, b is the decay rate parameter, and x is the independent variable determined by the number of remaining features.
8. The method for rapid detection of raw milk quality grade according to claim 7, characterized in that: Methods for genetic algorithm processing based on variables after eliminating variables include: Binary coding, uniform crossover, tournament selection and bit flipping mutation strategies are adopted. The initial population size is 25, the number of individuals participating in the competition is 2, the crossover rate is 0.5, the mutation rate is 0.05, and the maximum number of iterations is 50.
9. The method for rapid detection of raw milk quality grade according to claim 5, characterized in that: The parameters of the support vector machine model are as follows: C is the penalty coefficient of the objective function, and g is the kernel function coefficient.