Method and device for determining pollutants in edible oil and storage medium

Abstract

The invention relates to a method and a device for determining pollutants in edible oil and a storage medium. The method comprises the following steps: constructing a mixture near infrared spectrum set according to the edible oil containing the pollutants; determining the near infrared spectrum of to-be-determined edible oil to obtain the absorbance of the to-be-determined edible oil; the absorbance of the to-be-detected edible oil is input into a pollutant type recognition model, the pollutant type in the to-be-detected edible oil is recognized, and the pollutant type recognition model is obtained by training the mixture near infrared spectrum set; the identified pollutant types in the edible oil to be detected are input into a pollutant content regression model, the content of each pollutant type is predicted, and the pollutant content regression model is obtained by training the mixture near infrared spectrum set. Therefore, the qualitative and quantitative detection of the pollutants can be realized only by carrying out spectral analysis on the edible oil containing the pollutants once, the operation process is simple and convenient, and the analysis speed is high.

Description

Technical Field

[0001] The present application mainly relates to the application field of near-infrared spectroscopy, and in particular to a method, device and storage medium for determining pollutants in edible oil. Background Art

[0002] Edible oil is an indispensable part of life. There are many types of edible oil in my country, including soybean oil, rapeseed oil, corn oil, rice oil, peanut oil, sunflower oil, olive oil, sesame oil, walnut oil, palm oil and avocado oil. Once there is a problem with the oil, it will often lead to serious food safety accidents.

[0003] Throughout the production, storage, transportation, and use of edible oil, there is a risk of contamination by other chemical oils. This can occur, for example, when gasoline and kerosene tankers mix edible oil and chemical oils. However, traditional edible oil contamination testing standards suffer from complex sampling methods, multiple steps, and time-consuming procedures when testing some edible oils. These methods cannot quickly and effectively analyze the types and levels of chemical oil contaminants. Summary of the Invention

[0004] One purpose of the present application is to provide a method, device and storage medium for determining pollutants in edible oil, so as to solve the problems in the prior art of complex sampling methods, multiple operating steps and inability to quickly and effectively analyze the types and contents of chemical oil pollutants.

[0005] According to one aspect of the present application, a method for determining pollutants in edible oil is provided, the method comprising:

[0006] Constructing a collection of near-infrared spectra of mixtures based on edible oils containing contaminants;

[0007] Measuring the near infrared spectrum of the edible oil to be tested to obtain the absorbance of the edible oil to be tested;

[0008] Inputting the absorbance of the edible oil to be tested into a pollutant type identification model to identify the types of pollutants in the edible oil to be tested, wherein the pollutant type identification model is trained by the near-infrared spectrum set of the mixture;

[0009] The identified pollutant types in the edible oil to be tested are input into a pollutant content regression model to predict the content of each pollutant type, wherein the pollutant content regression model is trained by the near-infrared spectrum set of the mixture.

[0010] Optionally, the mixture near-infrared spectrum set includes the absorbance of the mixture of edible oil containing pollutants, the concentration of each pollutant, and a label of each pollutant type.

[0011] Optionally, a mixture near-infrared spectrum set is constructed based on the edible oil containing the contaminant, including:

[0012] Obtaining a near infrared spectrum of pure edible oil and a near infrared spectrum of edible oil containing contaminants, wherein the edible oil containing contaminants is obtained by adding at least one chemical oil to pure edible oil;

[0013] The near infrared spectra of the mixture with different gradients are constructed according to the near infrared spectra of the pure edible oil and the near infrared spectra of the edible oil containing pollutants.

[0014] Optionally, the edible oil containing contaminants includes a simulated mixture sample, and constructing a mixture near-infrared spectrum set based on the edible oil containing contaminants includes:

[0015] Set the content of pollutants in the simulated mixture sample;

[0016] measuring a near-infrared spectrum of the purified edible oil to obtain a corresponding first absorbance;

[0017] measuring a near-infrared spectrum of the simulated mixture sample to obtain a corresponding second absorbance;

[0018] Performing vector fitting on the first absorbance and the second absorbance to generate absorbance vectors of near-infrared spectra of the mixture with different gradients, thereby obtaining a mixed near-infrared spectrum set.

[0019] Optionally, vector fitting is performed on the first absorbance and the second absorbance to generate absorbance vectors of near-infrared spectra of mixtures with different gradients, satisfying the following formula:

[0020] Absor mix =x%×Absor EO +y%×Absor CO +c;

[0021] Among them, Absor mix Absor represents the absorbance vector of the near-infrared spectrum of the mixture. EO The vector representing the first absorbance, Absor CO represents the vector of the second absorbance, x% represents the percentage content of each edible oil, y% represents the percentage content of each pollutant, and c is the compensation term.

[0022] Optionally, the process of constructing the pollutant type identification model includes:

[0023] The pollutant type label is used as the target feature space in the KNN model, and the absorbance vector of each mixture is input into the KNN model to calculate the sample similarity;

[0024] A decision is made based on the K value and the sample similarity, and a KNN classification model is established, and the KNN classification model is used as a pollutant type identification model.

[0025] Optionally, the process of constructing the pollutant type identification model includes:

[0026] Apply local linear embedding to reduce the dimension of the absorbance vector of each mixture to obtain a low-dimensional vector;

[0027] The low-dimensional vector and the pollutant type label are used to train a naive Bayes model to obtain a pollutant type recognition model.

[0028] Optionally, the pollutant type identification model includes a clustering algorithm, and the absorbance of the edible oil to be tested is input into the pollutant type identification model:

[0029] Performing dimensionality reduction on the absorbance vectors of each mixture and the absorbance of the edible oil to be tested using a specified analysis method to obtain corresponding low-dimensional vectors, wherein the specified analysis method includes principal component analysis or sparse principal component analysis;

[0030] Use the K-means algorithm to cluster the corresponding low-dimensional vectors to determine the target cluster corresponding to the mixture and the cluster where the edible oil to be tested is located;

[0031] Determining the pollutant type of the target cluster according to the pollutant type label in the target cluster;

[0032] The type of pollutant is determined based on the cluster where the edible oil to be tested belongs.

[0033] Optionally, the process of constructing a pollutant content regression model includes:

[0034] Performing interval partial least squares feature selection on the absorbance of the mixture to obtain an absorbance feature vector;

[0035] A partial least squares model is trained according to the absorbance characteristic vector and the concentration of each pollutant to obtain a pollutant content regression model.

[0036] Optionally, the process of constructing a pollutant content regression model includes:

[0037] Performing interval partial least squares feature selection on the absorbance of the mixture to obtain an absorbance feature vector;

[0038] Establishing a partial least squares model based on the absorbance characteristic vector and the concentration of each pollutant;

[0039] Calculating the training set prediction deviation in the partial least squares model and calculating the absolute value of the regression coefficient to extract a new feature set;

[0040] An XGboost model is trained according to the prediction deviation and the new feature set, and a pollutant content regression model is obtained according to the partial least squares model and the trained XGboost model.

[0041] According to another aspect of the present application, a device for determining pollutants in edible oil is provided, the device comprising:

[0042] one or more processors; and

[0043] A memory storing computer-readable instructions, which, when executed, cause the processor to perform the operations of the method described above.

[0044] According to another aspect of the present application, a computer-readable medium is provided, on which computer instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described above.

[0045] Compared with the prior art, the present application constructs a mixture near-infrared spectrum set based on edible oil containing pollutants; measures the near-infrared spectrum of the edible oil to be tested to obtain the absorbance of the edible oil to be tested; inputs the absorbance of the edible oil to be tested into a pollutant type identification model to identify the types of pollutants in the edible oil to be tested, wherein the pollutant type identification model is trained by the mixture near-infrared spectrum set; inputs the identified types of pollutants in the edible oil to be tested into a pollutant content regression model to predict the content of each type of pollutant, wherein the pollutant content regression model is trained by the mixture near-infrared spectrum set. Therefore, only one spectral analysis of the contaminated edible oil is required to achieve qualitative and quantitative detection of pollutants, and the operation process is simple and the analysis speed is fast. It can effectively solve the problems of high similarity between pollutants and their spectral characteristics, excessively high data dimensions, and a lot of redundant information, thereby efficiently extracting spectral features and promoting the establishment of qualitative and quantitative models. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the above-mentioned objects, features and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings, wherein:

[0047] Figure 1 A schematic flow chart of a method for determining pollutants in edible oil according to one aspect of the present application is shown;

[0048] Figure 2 The near-infrared spectrum of soybean oil in one embodiment of the present application is shown;

[0049] Figure 3 The near infrared spectrum of rapeseed oil in one embodiment of the present application is shown;

[0050] Figure 4The near-infrared spectrum of gasoline in one embodiment of the present application is shown;

[0051] Figure 5 shows a near-infrared spectrum of kerosene in one embodiment of the present application;

[0052] Figure 6 A schematic diagram showing the effect of different concentrations of gasoline on the near-infrared spectral absorbance of soybean oil in one embodiment of the present application;

[0053] Figure 7 A schematic diagram showing the effect of different concentrations of kerosene on the near-infrared spectral absorbance of rapeseed oil in one embodiment of the present application;

[0054] Figure 8 A schematic diagram showing the degree of deviation between the predicted value and the true value of the soybean oil regression model in one embodiment of the present application;

[0055] Figure 9 A schematic diagram showing the degree of deviation between the predicted value and the true value of the rapeseed oil regression model in one embodiment of the present application is shown;

[0056] Figure 10 A schematic diagram showing a comparative analysis of reference values ​​and predicted values ​​of pollutant types and their contents in one embodiment of the present application;

[0057] Figure 11 A system block diagram of a device for measuring pollutants in edible oil according to an embodiment of the present application is shown.

[0058] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objectives, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0061] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0062] Figure 1A schematic flow chart of a method for determining pollutants in edible oil according to one aspect of the present application is shown. The method includes steps S11 to S14.

[0063] Step S11: constructing a mixture near-infrared spectrum set based on the edible oil containing pollutants.

[0064] The edible oil containing pollutants is edible oil contaminated by chemical oil, and the edible oil is at least one of soybean oil, rapeseed oil, corn oil, rice oil, peanut oil, sunflower oil, olive oil, sesame oil, walnut oil, palm oil, avocado oil, etc. Figure 2 and 3 As shown in FIG, they are the near infrared spectra of soybean oil and rapeseed oil respectively; chemical oil pollutants can be one or more of petroleum ether, gasoline, kerosene, marine fuel oil, deasphalted oil, catalytic raw material residual oil, crude oil, diesel, lubricating oil, insulating oil, etc.; Figure 4 and 5 The figures shown are the near-infrared spectra of gasoline and kerosene, respectively. The mixture near-infrared spectrum set can be a real mixture near-infrared spectrum set, a simulated mixture near-infrared spectrum set, or a real-simulated combined near-infrared spectrum set. The mixture near-infrared spectrum set serves as the training set for the subsequent pollutant type identification model and content regression model.

[0065] Step S12: measuring the near infrared spectrum of the edible oil to be tested, and obtaining the absorbance of the edible oil to be tested.

[0066] The edible oil to be tested is the edible oil that needs to be tested for pollutants, the type of pollutants contained, and the content. The near-infrared spectrum of the edible oil to be tested is measured to obtain the corresponding absorbance. A single near-infrared spectrum measurement is combined with a subsequent model to perform qualitative and quantitative analysis to determine the type and content of pollutants in the edible oil.

[0067] Step S13: inputting the absorbance of the edible oil to be tested into a pollutant type identification model to identify the types of pollutants in the edible oil to be tested, wherein the pollutant type identification model is trained by the near-infrared spectrum set of the mixture.

[0068] A pollutant type identification model is established based on the obtained mixture near-infrared spectrum set, so as to accurately identify the edible oil to be tested and identify the types of pollutants in the edible oil to be tested.

[0069] Next, in step S14, the identified pollutant types in the edible oil to be tested are input into a pollutant content regression model to predict the content of each pollutant type. The pollutant content regression model is trained using the mixture near-infrared spectrum set. After the pollutant types are identified, the content of each pollutant type is tested to determine the specific content of each pollutant type. The obtained mixture near-infrared spectrum set is also used as training samples to train the model to obtain the corresponding pollutant content regression model.

[0070] In some embodiments of the present application, the mixture near-infrared spectrum set includes the absorbance of the mixture of contaminant-containing edible oils, the concentrations of each contaminant, and the type labels of each contaminant. When constructing training samples, a batch of edible oils known to be contaminated with contaminants, with known contaminant type labels and concentrations, is used. Near-infrared spectroscopy is performed on this batch of edible oils, and the acquired absorbance is combined with the contaminant concentrations and type labels to construct the training set.

[0071] Specifically, the mixture near-infrared spectrum set can be a set of real mixture near-infrared spectrum sets, obtained by: obtaining the near-infrared spectrum of pure edible oil and the near-infrared spectrum of edible oil containing contaminants, wherein the edible oil containing contaminants is obtained by adding at least one chemical oil to pure edible oil; and constructing near-infrared spectra of the mixture with different gradients based on the near-infrared spectra of the pure edible oil and the near-infrared spectra of the edible oil containing contaminants. Here, the real mixture spectrum set is established by adding chemical oils to the edible oil in a gradient. For example, the edible oil is soybean oil, and the chemical oil contaminant includes at least one of kerosene and gasoline, thereby obtaining the mixture spectrum set using the near-infrared spectrum of the soybean oil contaminated with the chemical oil. Specifically, the near-infrared spectrum of pure edible oil is obtained using a near-infrared spectrometer; one or more chemical oils are added to the pure edible oil to form edible oils contaminated with chemical oils of different masses, volumes, or molar concentrations; and the absorbance of the edible oils contaminated with chemical oils is obtained using a near-infrared spectrometer, thereby forming the real mixture spectrum set with different gradients.

[0072] In a specific embodiment of the present application, the net mass of edible oil is measured, the expected mass of chemical oil contaminants is calculated, the volume of chemical oil contaminants to be added is calculated using the mass-volume formula, the near-infrared spectrum of the mixture is measured, the net mass of the mixture is measured, and finally the above steps are repeated to obtain a spectrum set of chemical oil mixtures with different mass fractions doped in edible oil, wherein the mass fraction of chemical oil contaminants includes but is not limited to any one or any combination of 0.5, 1, 2, 4, 6, 8 and 10. For example, different concentrations of gasoline are doped into soybean oil, and the near-infrared spectrum of the mixture is measured, such as Figure 6The diagram shows the effect of different concentrations of gasoline on the near-infrared absorbance of soybean oil. Different concentrations of kerosene are added to rapeseed oil, and the near-infrared spectra of the resulting mixtures are measured, as shown in FIG. Figure 7 Schematic diagram showing the effect of different concentrations of kerosene on the near-infrared spectral absorbance of rapeseed oil.

[0073] In some embodiments of the present application, the edible oil containing contaminants includes a simulated mixture sample. In step S11, the content of the pollutant substance in the simulated mixture sample is set; the near-infrared spectrum of the pure edible oil is measured to obtain a corresponding first absorbance; the near-infrared spectrum of the simulated mixture sample is measured to obtain a corresponding second absorbance; the first absorbance and the second absorbance are vector-fitted to generate absorbance vectors of the near-infrared spectra of the mixture with different gradients, and a mixed near-infrared spectrum set is obtained.

[0074] Establish a near-infrared spectrum of a simulated mixture of edible oil contaminated by chemical oil to enrich the near-infrared spectrum set of the mixture. The simulated mixture spectrum set is established based on the near-infrared spectrum fitting of the pure edible oil corresponding to the chemical oil pollutant to be measured. Specifically, the near-infrared spectrum of the pure edible oil is measured using a near-infrared spectrometer to obtain the absorbance of the near-infrared spectrum of the pure edible oil as the first absorbance; the near-infrared spectrum of the chemical oil pollutant is measured using a near-infrared spectrometer to obtain the absorbance of the near-infrared spectrum of the chemical oil pollutant as the second absorbance; set the content of the chemical oil pollutant in the simulated mixture sample, for example, containing chemical oil pollutants with different mass fraction gradients such as 0% to 10%; fit the vector of the first absorbance with the vector of the second absorbance to form a simulated mixture spectrum set with different gradients.

[0075] For example, taking soybean oil and rapeseed oil as examples, a near-infrared spectrometer is used to obtain the near-infrared spectra of soybean oil and rapeseed oil; a near-infrared spectrometer is used to obtain the near-infrared spectra of gasoline and kerosene; the content of chemical oil pollutants in the simulated mixture sample is set; the near-infrared light absorbance vector of the chemical oil sample and the absorbance vector of the edible oil near-infrared spectrum are fitted to establish a simulated mixture spectrum set.

[0076] The fitting step satisfies the following formula:

[0077] Absor mix =x%×Absor EO +y%×Absor CO +c;

[0078] Among them, Absor mix Absor represents the absorbance vector of the near-infrared spectrum of the mixture. EO The vector representing the first absorbance, Absor COrepresents the vector of the second absorbance, x% represents the percentage content of each edible oil, y% represents the percentage content of each pollutant, and c is the compensation term to compensate for the error of linear fitting.

[0079] Two classification methods and two clustering methods were used to perform the mixture spectrum collection, and four different pollutant type identification models were obtained. The tested mixture can be accurately identified by any of the pollutant type identification models for the chemical oil pollutant in the edible oil. Taking soybean oil and rapeseed oil as an example, Figure 6 and Figure 7 As shown, in some bands, the absorbance vector of pure edible oil with different mass fractions of chemical oil pollutants changes significantly. The near-infrared spectrum of the mixture spectrum has the characteristics of high data dimension and large redundancy. In the embodiment of the present application, dimensionality reduction technology is used and decisions are made based on the distance between samples, which effectively discovers the characteristics of pollutants in the mixture spectrum from the essence.

[0080] The first classification method is as follows: the pollutant type label is used as the target feature space in the KNN model, the absorbance vector of each mixture is input into the KNN model to calculate the sample similarity; a decision is made based on the K value and the sample similarity, a KNN classification model is established, and the KNN classification model is used as a pollutant type identification model. Here, a K-nearest neighbor (KNN) model is used for classification analysis. The KNN model includes feature space, distance measurement, and K value (number of neighbor samples) selection; the KNN model uses the training set absorbance vector as input, the chemical oil pollutant label as the training target, the absorbance of the edible oil to be tested is input into the KNN model, and finally outputs the pollutant type label. The KNN model calculates the distance between the sample and each sample in the training set, selects the K training samples closest to the unlabeled sample as its "neighbors", and determines the category label of the unlabeled sample by majority voting based on the category labels of these K neighbors. A smaller K value will make the model complex and easily affected by noise; while a larger K value may ignore the inherent pattern of the sample. In the example of this application, the K value is continuously adjusted until a satisfactory category prediction label is obtained. The specific process is as follows: the spectral absorbance of the mixture is used as input and the pollutant type label is used as the target to construct a feature space, the sample similarity is judged by distance measurement, the decision is made based on the K value, and a KNN classification model is established. The KNN classification model is used as a pollutant type identification model, and the absorbance vector of the edible oil to be tested is input into the KNN classification model to determine the specific type of chemical oil pollutant from the output label.

[0081] The second classification method uses local linear embedding (LLE) for dimensionality reduction, and then combines it with a naive Bayes classifier for classification analysis; as follows: the absorbance vector of each mixture is subjected to dimensionality reduction using local linear embedding to obtain a low-dimensional vector; the low-dimensional vector and the pollutant type label are used to train a naive Bayes model to obtain a pollutant type identification model. Here, the absorbance vector of the mixture is first extracted and dimensionality reduced using LLE, and then the low-dimensional vector and the corresponding label, i.e., the pollutant type label, are used to train a naive Bayes model to obtain a classification model as a pollutant type identification model; the absorbance vector of the edible oil to be tested is subsequently input into the LLE dimensionality reduction and trained classification model to output labels to determine the corresponding chemical oil pollutants in the mixture sample. For example, if there are labels 1, 2, and 3 in the mixture sample, the output labels are labels 2 and 3. It can be seen that the edible oil to be tested also contains pollutants corresponding to the two labels in the mixture sample. For example, label 2 is diesel and label 3 is insulating oil, then the pollutants contained in the edible oil to be tested are diesel and insulating oil.

[0082] In a specific embodiment of the present application, taking rapeseed oil as an example, local linear embedding (LLE) is combined with a naive Bayes classification model as a classification method for analysis. The core idea of ​​LLE is to map high-dimensional data to a low-dimensional space by retaining the linear structure of the local neighborhood of the data; first, the absorbance vector is reduced in dimension using LLE, and then the reduced-dimensional characteristics are used as input, and the chemical oil pollutant label is used as the training target to establish a model, and finally the absorbance of the mixture to be tested is input into the naive Bayes model, and finally the type label of the pollutant is output.

[0083] Specifically, combining local linear embedding (LLE) with a naive Bayes classification model includes the following steps: Step 1, obtaining sample spectral data and constructing a spectral analysis experimental data set, wherein the data set is the spectral data of the sample, and the spectral data of the sample includes wavelength range, wavelength resolution, number of wavelength points and sample absorbance value; Step 2, constructing a sample weight coefficient matrix based on the sample reconstruction error; Step 3, solving the eigenvector of the weight coefficient matrix to obtain a low-dimensional projection matrix; Step 4, using the low-dimensional data to construct a naive Bayes classifier and verify its accuracy.

[0084] In step 2, constructing a sample weight coefficient matrix based on the sample reconstruction error includes the following steps:

[0085] Step S21: For the spectrum matrix X = {x1, x2, ..., x n}, for the sample x in the original high-dimensional space i They can be represented by their corresponding adjacent samples, where the reconstruction error between the real sample and the represented sample can be expressed as: Where x j is x iThe neighboring points, Q(i) is the neighboring sample set, w ij is the corresponding sample weight, satisfying ∑ j w ij =1.

[0086] Step S22: The weight coefficient vector can be obtained:

[0087]

[0088] From this we can get the weight matrix M = (IW)(IW) T , where W=[w1,…,w N ] T .

[0089] In step 3, solving the eigenvector of the weight coefficient matrix to obtain the low-dimensional projection matrix includes the following steps:

[0090] Step S31: For the weight matrix M, the eigenvalue and eigenvector can be obtained by the following formula:

[0091] MZ T =λZ T

[0092] Where Z=[z1,z2,…,z d ] T is the matrix composed of eigenvectors, and λ is the eigenvalue vector.

[0093] Step S32: Determine a low-dimensional projection matrix Z′ based on the d′ eigenvalues ​​with the smallest λ.

[0094] In step 4, building a naive Bayes classifier using low-dimensional data includes the following steps:

[0095] Step S41: For the data set T = {(z′1, y1), (z′2, y2), …, (z′ N ,y N )}, where z′ i ∈R d′ ,z′ ij ∈{α j1 ,α j2 ,…,α jL}, α jl is the lth possible value of the jth feature, j = 1, 2, ..., n, l = 1, 2, ..., L, y i ∈{c1,c2,…,c K}, the prior probability and conditional probability can be calculated:

[0096]

[0097] Step S42: For a given sample z′=(z′1, z′2,…, z′ d′ ) T ,calculate:

[0098]

[0099] Step S43: Determine sample z′=(z′1, z′2,…, z′ d′ ) T Categories:

[0100]

[0101] In some embodiments of the present application, the pollutant type identification model includes a clustering algorithm. In step S13, the absorbance vectors of each mixture and the absorbance of the edible oil to be tested are respectively reduced in dimension using a specified analysis method to obtain corresponding low-dimensional vectors, wherein the specified analysis method includes principal component analysis or sparse principal component analysis; the corresponding low-dimensional vectors are clustered using the K-means algorithm to determine the target cluster corresponding to the mixture and the cluster where the edible oil to be tested is located; the pollutant type of the target cluster is determined based on the pollutant type label in the target cluster; and the pollutant type is determined based on the cluster where the edible oil to be tested is located.

[0102] The first clustering method is as follows: cluster analysis is performed using principal component analysis (PCA) combined with the K-means algorithm; the vectors corresponding to the absorbance of the mixture and the absorbance of the edible oil to be tested are reduced in dimensionality using principal component analysis (PCA), and then the low-dimensional vectors are clustered using the K-means algorithm; the K-means algorithm divides the mixture samples into different clusters and finds the most representative cluster. For each data point (sample), its distance from each cluster center is calculated and assigned to the cluster represented by the nearest cluster center. Each cluster corresponds to a chemical oil pollutant category. The clustering results can be used to determine the cluster to which each data point belongs. The type of chemical oil pollutant in the cluster can be determined by the labels of the mixture spectrum set samples (known chemical oil pollutant categories) in the cluster, and the type of chemical oil pollutant substance can be determined by the cluster where the edible oil to be tested is located.

[0103] In the embodiment of the present application, when using PCA for dimensionality reduction, reducing it to two dimensions can well preserve sample information; the K-means algorithm is a distance-based clustering algorithm. First, K initial cluster center points are selected. Secondly, for each sample, its distance from the K cluster centers is calculated, and the sample is assigned to the cluster corresponding to the nearest cluster center. Furthermore, for each cluster, the mean of all its samples is calculated as the new cluster center. Finally, the above steps are repeated until the cluster center no longer changes.

[0104] The second clustering method is as follows: sparse principal component analysis (Sparse PCA) combined with the K-means algorithm is used for cluster analysis; first, sparse principal component analysis is performed on the absorbance of the mixture and the absorbance vector of the edible oil to be tested to extract a low-dimensional and interpretable absorbance vector and improve its sparsity; then, the K-means algorithm is applied to cluster the mixture using the low-dimensional absorbance vector as input; the type of chemical oil pollutant in the cluster is determined by the sample labels of the mixture spectrum set in the cluster, and the type of chemical oil pollutant is determined by the cluster where the edible oil to be tested is located.

[0105] For the second clustering method, the absorbance vectors of the real and simulated mixtures are reduced in dimension using sparse principal component analysis (Sparse PCA), and then the K-means algorithm is used to cluster the reduced-dimensional data. The clustering results can be used to determine the cluster to which each data point belongs, and the type of chemical oil pollutants in the cluster can be determined by the sample labels of the mixture spectrum set in the cluster, and the type of chemical oil pollutants can be determined by the cluster where the edible oil to be tested is located. Specifically, Sparse PCA is a dimensionality reduction technology that combines PCA and sparsity incorporation, which can extract principal components with sparse characteristics, so that each principal component has only a small number of features that have a significant impact on it; the regularization parameter (lambda) during dimensionality reduction is used to control sparsity. A larger lambda value will cause the principal components to become sparser, and more features will be set to zero; while a smaller value may result in denser principal components. Too high or too low values ​​may lead to loss of information or increased complexity of the model.

[0106] Furthermore, cross-validation is performed to evaluate the performance of different eigenvalue vector λ values. A double loop is used to verify the λ value. The outer loop traverses a series of candidate λ values. The inner loop divides the dataset into training set and test set, and trains the model using the current λ value. Then, the reconstruction error is calculated, and the λ value that leads to the minimum reconstruction error is found as the optimal λ value.

[0107] In some embodiments of the present application, the process of constructing a pollutant content regression model includes: performing interval partial least squares feature selection on the absorbance of the mixture to obtain an absorbance feature vector; training a partial least squares model based on the absorbance feature vector and the concentration of each pollutant to obtain a pollutant content regression model. Here, two regression methods are respectively performed on a set of mixture spectra of a known edible oil type and corresponding to a type of chemical oil pollutant to obtain a content regression model corresponding to a certain type of chemical oil pollutant in the edible oil; after the edible oil to be tested is identified by the pollutant type identification model, it is then passed through the regression model of the chemical oil pollutant content of the type to achieve accurate analysis of the content of chemical oil pollutants in the edible oil. Among them, the first regression method is as follows: using a partial least squares (PLS) model for regression analysis, the near-infrared original spectrum may have baseline drift and noise. In the embodiments of the present application, a variety of pre-processing methods such as smoothing and derivatives are used to process the original spectrum, thereby improving the modeling effect. The preprocessed spectra are subjected to interval partial least squares (iPLS) feature selection. The iPLS method divides the spectral data into several wavelength intervals, and performs a separate partial least squares (PLS) model on each wavelength interval to obtain a partial model for each interval. By analyzing the partial model parameters of each interval, the wavelengths that are highly correlated with the target variable are selected, that is, the wavelength points that are considered to have an important contribution to the model prediction performance; that is, the absorbance characteristic vector of the mixture is used as the independent variable and the pollutant concentration is used as the dependent variable, and the PLS model is established; finally, the absorbance vector of the edible oil to be tested is input, and the content of a chemical oil pollutant corresponding to the edible oil is determined based on the output of the PLS regression model.

[0108] The PLS regression model uses the input absorbance vector as the independent variable and the chemical oil pollutant content as the training target. Through projection, the predicted variables and the observed variables are mapped to a new space, thereby finding a linear regression model to accurately predict the pollutant content. This method can effectively process multivariate data and provide accurate prediction results, such as Figure 8 , which shows a schematic diagram of the degree of deviation between the predicted value and the true value of the soybean oil regression model.

[0109] In some embodiments of the present application, the process of constructing a pollutant content regression model includes: performing interval partial least squares feature selection on the absorbance of the mixture to obtain an absorbance feature vector; establishing a partial least squares model based on the absorbance feature vector and the concentration of each pollutant; calculating the training set prediction deviation in the partial least squares model, and calculating the absolute value of the regression coefficient to extract a new feature set; training an XGboost model based on the prediction deviation and the new feature set, and obtaining a pollutant content regression model based on the partial least squares model and the trained XGboost model. Here, the second regression method is as follows: use the partial least squares (PLS) model combined with the extreme gradient boosting (XGboost) model for regression analysis; first establish a partial least squares (PLS) regression model, then calculate the training set prediction deviation in the PLS regression model and find the number of latent variables with an explained variance greater than 85%, then calculate the absolute value of the regression coefficient of the PLS regression model, sort it in descending order, extract it into a new feature set, and use the new feature set as the independent variable and the prediction deviation as the dependent variable to establish the XGboost regression model; thus, the pollutant content regression model includes the PLS regression model and the trained XGboost regression model. Finally, the absorbance vector of the edible oil to be tested is input, and the content of a chemical oil pollutant corresponding to the edible oil is determined by the sum of the results of the PLS regression model and the XGboost regression model, such as Figure 9 Regression model results for rapeseed oil are shown.

[0110] The learning rate of the XGboost regression model controls the magnitude of the model parameter updates in each iteration to avoid overfitting. Typical learning rate values ​​can be set between 0.01 and 0.3. Increasing the number of iterations generally improves model performance, so a larger value can be set initially and then adjusted. In an embodiment of the present application, the XGboost model uses the prediction deviation of the PLS model training set as the training target, and then obtains the value of the validation set and adds it to the prediction result of the PLS model.

[0111] In a specific embodiment of the present application, the performance of the regression model is judged using the root mean square error (RMSEP) and the coefficient of determination (R-Squared). RMSEP is the root mean square error of the test set. The smaller the value, the better the fitting effect. RMSEP can be expressed as follows: Among them, y i is the true value, is the predicted value, and n is the number of test set samples. The coefficient of determination R-Squared is between 0 and 1. The closer R-Squared is to 1, the better the model is; the closer R-Squared is to 0, the worse the model is. The coefficient of determination R-Squared can be expressed as follows:

[0112]

[0113] Among them, y i is the true value, is the predicted value, The mean of all test set target values, where n is the number of test set samples.

[0114] like Figure 8 and Figure 9 In the illustrated embodiment, the root mean square error of prediction (RMSEP) of the regression models is less than 0.3, and the coefficient of determination R-Squared is greater than 0.99. It can be seen that the two regression models have small prediction errors and good fitting effects.

[0115] In a specific embodiment of the present application, taking the chemical oil pollutants including gasoline and kerosene as an example, the comparison results of the reference value and the predicted value of the types and contents of the pollutants based on the near infrared spectroscopy analysis method are as follows: Figure 10 As shown by Figure 10 It can be seen that the model has good classification accuracy and small pollutant content prediction error.

[0116] Compared with current edible oil pollutant detection technology, the method described in this application only needs to perform a spectral analysis on the contaminated edible oil to achieve qualitative and quantitative detection of pollutants. This simple operating process and fast analysis speed make this method have obvious advantages in practical applications. At the same time, this application can effectively solve the problems such as high similarity of pollutants and their spectral characteristics, too high data dimension and more redundant information, thereby efficiently extracting spectral characteristics and promoting the establishment of qualitative and quantitative models. In addition, the spectral database corresponding to the spectral data set constructed in this application has good scalability. In actual application, if a new type of pollutant is encountered, its spectral information can be added to the database, and then the qualitative and quantitative model can be expanded to enhance the scope of application of the method and system.

[0117] Figure 11 The system block diagram of the device for measuring pollutants in edible oil according to one embodiment of the present application is shown. Figure 11 As shown, the measuring device 100 for measuring pollutants in edible oil may include an internal communication bus 101, a processor 102, a read-only memory (ROM) 103, a random access memory (RAM) 104, a communication port 105, and a hard disk 107. The internal communication bus 101 can realize data communication between the device components of XXX. The processor 102 can make judgments and issue prompts. In certain embodiments, the processor 102 can be composed of one or more processors.

[0118] Communication port 105 enables data transmission and communication between the device for measuring contaminants in edible oil and external input / output devices. In some embodiments, the device for measuring contaminants in edible oil can send and receive information and data from a network via communication port 105. In some embodiments, the device for measuring contaminants in edible oil can transmit and communicate with external input / output devices via a wired connection via input / output port 106.

[0119] The device for determining contaminants in edible oil may also include various forms of program storage units and data storage units, such as a hard disk 107, a read-only memory (ROM) 103, and a random access memory (RAM) 104, capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor 102. The processor 102 executes these instructions to implement the main part of the method. The results processed by the processor 102 are transmitted to an external output device via a communication port 105 and displayed on the user interface of the output device.

[0120] For example, the implementation process file of the above-mentioned device for determining pollutants in edible oil may be a computer program, which is stored in the hard disk 107 and can be recorded in the processor 102 for execution to implement the method of the present application.

[0121] The present application also provides a computer-readable medium having computer instructions stored thereon, and the computer-readable instructions can be executed by a processor to implement the method for determining contaminants in edible oil as described above.

[0122] When the method for determining contaminants in edible oil is implemented as a computer program, it can also be stored in a computer-readable storage medium as an article of manufacture. For example, a computer-readable storage medium can include, but is not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memories (EPROMs), cards, sticks, key drives). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" can include, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain, and / or carry code and / or instructions and / or data.

[0123] It should be understood that the embodiments described above are merely illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or a combination thereof.

[0124] Some aspects of the present application can be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". The processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors or combinations thereof. In addition, various aspects of the present application may be expressed as computer products located in one or more computer-readable media, which include computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, tapes...), optical disks (e.g., compact disks CDs, digital versatile disks DVDs...), smart cards, and flash memory devices (e.g., cards, sticks, key drives...).

[0125] A computer-readable medium may include a propagated data signal embodying computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination thereof. A computer-readable medium may be any computer-readable medium other than a computer-readable storage medium that can be connected to an instruction execution system, apparatus, or device to communicate, propagate, or transmit the program for use. The program code on the computer-readable medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, radio frequency signal, or similar medium, or any combination of the above.

[0126] The basic concepts have been described above. It will be apparent to those skilled in the art that the above disclosures are merely illustrative and do not constitute limitations on this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and amendments to this application. Such modifications, improvements, and amendments are suggested in this application and remain within the spirit and scope of the exemplary embodiments of this application.

[0127] At the same time, this application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or multiple times in different locations in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0128] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of the present application are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

Claims

1. A method for determining pollutants in edible oil, characterized in that: The method comprises: Constructing a collection of near-infrared spectra of mixtures based on edible oils containing contaminants; Measuring the near infrared spectrum of the edible oil to be tested to obtain the absorbance of the edible oil to be tested; Inputting the absorbance of the edible oil to be tested into a pollutant type identification model to identify the types of pollutants in the edible oil to be tested, wherein the pollutant type identification model is trained by the near-infrared spectrum set of the mixture; The identified pollutant types in the edible oil to be tested are input into a pollutant content regression model to predict the content of each pollutant type, wherein the pollutant content regression model is trained by the near-infrared spectrum set of the mixture.

2. The method according to claim 1, characterized in that The mixture near-infrared spectrum set includes the absorbance of the mixture of edible oil containing pollutants, the concentration of each pollutant, and the type label of each pollutant.

3. The method according to claim 1, characterized in that A collection of NIR spectra of mixtures of contaminant-containing edible oils was constructed, including: Obtaining a near infrared spectrum of pure edible oil and a near infrared spectrum of edible oil containing contaminants, wherein the edible oil containing contaminants is obtained by adding at least one chemical oil to pure edible oil; The near infrared spectra of the mixture with different gradients are constructed according to the near infrared spectra of the pure edible oil and the near infrared spectra of the edible oil containing pollutants.

4. The method according to claim 1, wherein The edible oil containing pollutants includes a simulated mixture sample, and a mixture near-infrared spectrum set is constructed based on the edible oil containing pollutants, including: Set the content of pollutants in the simulated mixture sample; measuring a near infrared spectrum of the purified edible oil to obtain a corresponding first absorbance; measuring a near-infrared spectrum of the simulated mixture sample to obtain a corresponding second absorbance; Performing vector fitting on the first absorbance and the second absorbance to generate absorbance vectors of near-infrared spectra of the mixture with different gradients, thereby obtaining a mixed near-infrared spectrum set.

5. The method according to claim 4, characterized in that Perform vector fitting on the first absorbance and the second absorbance to generate absorbance vectors of the near-infrared spectrum of the mixture with different gradients, which satisfy the following formula: Absorber mix =x%×Absor EO +y%×Absorber CO +c; Among them, Absor mix Absor represents the absorbance vector of the near-infrared spectrum of the mixture. EO The vector representing the first absorbance, Absor CO represents the vector of the second absorbance, x% represents the percentage content of each edible oil, y% represents the percentage content of each pollutant, and c is the compensation term.

6. The method according to claim 2, characterized in that The construction process of the pollutant type identification model includes: The pollutant type label is used as the target feature space in the KNN model, and the absorbance vector of each mixture is input into the KNN model to calculate the sample similarity; A decision is made based on the K value and the sample similarity, and a KNN classification model is established, and the KNN classification model is used as a pollutant type identification model.

7. The method according to claim 2, characterized in that The construction process of the pollutant type identification model includes: Apply local linear embedding to reduce the dimension of the absorbance vector of each mixture to obtain a low-dimensional vector; The low-dimensional vector and the pollutant type label are used to train a naive Bayes model to obtain a pollutant type recognition model.

8. The method according to claim 2, characterized in that The pollutant type identification model includes a clustering algorithm, and the absorbance of the edible oil to be tested is input into the pollutant type identification model: Performing dimensionality reduction on the absorbance vectors of each mixture and the absorbance of the edible oil to be tested using a specified analysis method to obtain corresponding low-dimensional vectors, wherein the specified analysis method includes principal component analysis or sparse principal component analysis; Use the K-means algorithm to cluster the corresponding low-dimensional vectors to determine the target cluster corresponding to the mixture and the cluster where the edible oil to be tested is located; Determining the pollutant type of the target cluster according to the pollutant type label in the target cluster; The type of pollutant is determined based on the cluster where the edible oil to be tested belongs.

9. The method according to claim 2, characterized in that The construction process of the pollutant content regression model includes: Performing interval partial least squares feature selection on the absorbance of the mixture to obtain an absorbance feature vector; A partial least squares model is trained according to the absorbance characteristic vector and the concentration of each pollutant to obtain a pollutant content regression model.

10. The method according to claim 2, characterized in that The construction process of the pollutant content regression model includes: Performing interval partial least squares feature selection on the absorbance of the mixture to obtain an absorbance feature vector; Establishing a partial least squares model based on the absorbance characteristic vector and the concentration of each pollutant; Calculating the training set prediction deviation in the partial least squares model and calculating the absolute value of the regression coefficient to extract a new feature set; An XGboost model is trained according to the prediction deviation and the new feature set, and a pollutant content regression model is obtained according to the partial least squares model and the trained XGboost model.

11. A device for measuring pollutants in edible oil, characterized in that: The device comprises: one or more processors; and A memory storing computer readable instructions which, when executed, cause the processor to perform the operations of the method of any one of claims 1 to 10.

12. A computer-readable medium having computer instructions stored thereon, wherein the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 10.

Images