A difference compensation method based on spectral model transfer

Abstract

The present invention relates to near-infrared detection technology, and discloses a difference compensation method based on spectral model transfer, which solves the problems of large amount of spectral data, complex operation and low prediction efficiency in the existing spectral model transfer algorithm. The method first uses a miniaturized near-infrared spectral device to collect sample spectral data, obtains sample component absorption peaks, and reconstructs the spectrum in combination with the component absorption peaks, then divides the reconstructed spectral data, divides the host spectral data into a test set and a compensation set, and divides the slave spectral data into a compensation set with the same number of samples as the host compensation set; then uses the host training set samples to perform spectral modeling, and predicts the host and slave compensation sets, calculates and obtains the predicted mean, and then calculates the difference between the predicted mean of the host and slave compensation sets, and finally uses the host spectral model to predict the spectral data of the sample to be tested collected by the slave, and performs difference compensation on each predicted value one by one based on the difference.

Description

Technical Field

[0001] The invention relates to near infrared detection technology, and in particular to a difference compensation method based on spectral model transfer. Background Art

[0002] With the development of micro-electromechanical technology, in recent years, miniaturized near-infrared spectrometers have been widely used in various fields such as food, beverages, medicine, coal, etc. Compared with traditional large-scale Fourier transform spectroscopy analysis systems, they have the advantages of simple structure, low cost, easy to carry, and real-time visible results. It has become a popular product in the field of spectroscopy.

[0003] However, compared with large-scale near-infrared spectroscopic equipment in laboratories, miniaturized near-infrared spectrometers are affected by light sources, detectors, manufacturing processes, usage methods, environmental conditions, etc., and the performance of equipment consistency, stability, reproducibility and other indicators has been significantly reduced. Especially in terms of equipment consistency, different miniaturized near-infrared spectroscopic equipment of the same model has poor consistency, and the spectral models established by different equipment are not universal. Therefore, in actual miniaturized near-infrared spectroscopic analysis applications, for the same type of samples to be tested, each miniaturized near-infrared spectroscopic equipment needs to be modeled separately, which wastes a lot of manpower and material resources and greatly hinders the development of miniaturized near-infrared spectroscopic analysis technology.

[0004] At present, the conventional method used in the market to deal with the problem of non-universality of spectral models of miniaturized near-infrared spectroscopy equipment is spectral model transfer, and its algorithms include slope-intercept method (S / B), Shenks algorithm, direct standardization (DS) and piecewise standardization (PDS). Although these algorithms can solve the problem of non-universality of spectral models to a certain extent, these algorithms are usually applied to uniform samples, such as wheat powder, sorghum powder and other samples. For samples with poor uniformity and mixed materials such as distiller's grains and koji, the conventional model transfer algorithm plays a smaller role, and the prediction ability after the spectral model transfer is weak. At the same time, the conventional model transfer algorithm involves a large amount of spectral data, and the operation is complex and inefficient. Summary of the invention

[0005] The technical problem to be solved by the present invention is to propose a difference compensation method based on spectral model transfer to solve the problems of large amount of spectral data, complex operation and low prediction efficiency in existing spectral model transfer algorithms.

[0006] The technical solution adopted by the present invention to solve the above technical problems is:

[0007] A difference compensation method based on spectral model transfer comprises the following steps:

[0008] S1. For different near-infrared spectroscopic devices of the same type, one of the hosts is determined, and the rest are slaves; the host and the slaves are used to collect spectrum data of the same batch of samples to obtain the absorption peaks of the sample components;

[0009] S2, reconstructing sample spectrum data for the host and the slave respectively according to the peak points of the absorption peaks of the sample components;

[0010] S3, dividing the reconstructed sample spectral data of the host into a training set and a compensation set, and calibrating the spectral data in the training set; dividing the reconstructed sample spectral data of the slave into a compensation set having the same amount of spectral data as the compensation set of the host;

[0011] S4, using the training set of the host to perform spectral modeling, using the established host spectral model to predict the compensation set of the host and the compensation set of the slave, respectively, to obtain the predicted mean of the host compensation set and the predicted mean of the slave compensation set;

[0012] S5, calculating the difference between the predicted mean value of the master compensation set and the predicted mean value of the slave compensation set as the slave spectrum compensation value;

[0013] S6. When a slave machine is used to perform spectral sampling and prediction on the same type of samples to be tested, the sampled spectral data is predicted by the host spectral model to obtain a predicted value, and then the predicted value is difference compensated based on the slave machine spectral compensation value.

[0014] Furthermore, in step S1, the near-infrared spectroscopy device adopts an equally divided splicing type near-infrared spectroscopy device, and the method for obtaining the absorption peak of the sample component includes: selecting a wavelength point with the largest absorbance value in the band range of the near-infrared spectroscopy device, and the absorption peak at the wavelength point is the absorption peak of the sample component.

[0015] Furthermore, in step S2, the sample spectral data is reconstructed for the host and the slave respectively according to the peak points of the absorption peaks of the sample components, specifically including: only retaining a certain number of wavelength points around the component absorption peaks, and eliminating redundant wavelength points in the collected initial spectral data of the samples, thereby reconstructing the multi-sensor spectral data into single-sensor spectral data, that is, the number of wavelength points in the reconstructed spectral data is consistent with the number of wavelength points of the single-sensor spectral data in the near-infrared spectral device.

[0016] Furthermore, the method of retaining only a certain number of wavelength points around the component absorption peak and removing redundant wavelength points in the collected initial spectrum data of the sample specifically includes:

[0017] Set the wavelength range corresponding to the peak value of the component absorption peak as P. If P%10≠0, where % is the remainder symbol, round it off to get a new P value, replace the original P value, and take the wavelength range P as the center point, and take X3 wavelength points to the left and right for spectrum reconstruction. The calculation method of X3 is:

[0018] X3 = (X2-1) / 2, where X2 is the number of spectral wavelength points of a single sensor in the near-infrared spectroscopy device;

[0019] The reconstructed spectral data is actually represented as a set of absorbance values ​​at X2 wavelength points, where the wavelength ranges of each wavelength point are:

[0020] [(P-10*X3), (P-10*X3+10),…P,…(P+10*X3-10), (P+10*X3)];

[0021] When (P-10*X3)≤a, where a is the left boundary of the spectral band range of the near-infrared spectroscopy device, and the reconstructed spectral wavelength point overflows to the left, the wavelength point of the single sensor with the smallest band range in the near-infrared spectroscopy device is used as the wavelength point after spectral reconstruction;

[0022] When (P+10*X3)≥b, where b is the right boundary of the spectral band range of the near-infrared spectroscopy device, and the reconstructed spectral wavelength point overflows to the right, the wavelength point of the single sensor with the largest band range in the near-infrared spectroscopy device is used as the wavelength point after spectral reconstruction.

[0023] Further, in step S4, the method for respectively obtaining the predicted mean of the host compensation set and the predicted mean of the slave compensation set includes: assuming that the predicted values ​​of the component content prediction of the sample spectral data in the host compensation set using the established host spectral model are S1, S2, ... S n , the predicted values ​​of component content prediction for sample spectral data in the slave compensation set are C1, C2, ... C n ; then the host compensation set prediction mean Slave compensation set prediction mean n is the number of sample spectral data in the master compensation set and the slave compensation set.

[0024] Furthermore, in step S5, the calculation method of the slave spectrum compensation value is:

[0025] Slave spectrum compensation value

[0026] Furthermore, in step S6, performing difference compensation on the predicted value based on the slave spectrum compensation value includes:

[0027] Assume that the predicted values ​​of the component contents of the m samples to be tested are H1, H2, ... H m , then the predicted value after difference compensation is

[0028] The beneficial effects of the present invention are:

[0029] The method reconstructs spectral data according to the absorption peaks of sample components, reduces the amount of spectral data while retaining the sample component characterization capability, and improves the efficiency of spectral analysis. The method also utilizes the fact that the host and slave use the same spectral model to predict the mean values ​​of the predicted values ​​for the same batch of samples to be tested, and uses the difference between the predicted means of the compensation set of the host and the slave as the compensation value, thereby directly calling the host's spectral model to predict the spectral data collected by the slave, and using the compensation value to compensate and correct the predicted value to obtain the compensated predicted value. Since there is no need to establish a spectral model for each slave separately, the workload is reduced, and the accuracy of the prediction is also improved through difference compensation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a difference compensation method based on spectral model transfer in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention aims to propose a difference compensation method based on spectral model transfer to solve the problems of large amount of spectral data, complex operation and low prediction efficiency in the existing spectral model transfer algorithm. The method first uses a miniaturized near-infrared spectroscopy device to collect sample spectral data, obtains sample component absorption peaks, and reconstructs the spectrum in combination with the component absorption peaks. Then, the reconstructed spectral data is divided, the host spectral data is divided into a test set and a compensation set, and the slave spectral data is divided into a compensation set with the same number of samples as the host compensation set; then, the host training set samples are used for spectral modeling, and the host and slave compensation sets are predicted, the predicted mean is calculated, and then the difference between the predicted mean of the host and slave compensation sets is calculated, and finally, the host spectral model is used to predict the spectral data of the sample to be tested collected by the slave, and the difference compensation is performed one by one on each predicted value based on the difference.

[0032] Example:

[0033] like Figure 1 As shown, the difference compensation method based on spectral model transfer in this embodiment includes the following steps:

[0034] S101, spectrum data collection, obtain sample component absorption peak:

[0035] In this step, a miniaturized near-infrared spectrometer is used to collect spectral data of the same batch of samples to obtain the absorption peaks of the sample components. As a specific example, using a wavelength-averaged miniaturized near-infrared spectrometer to collect sample spectral data can make the spectral data uniform and reduce data errors.

[0036] At the same time, since the miniaturized near-infrared spectroscopy equipment is limited by the sensor performance, the band range covered by a single sensor is very limited. The commonly used single-sensor miniaturized near-infrared spectroscopy equipment covers a band range of 420 nanometers. The miniaturized near-infrared spectroscopy equipment used in this embodiment is a multi-sensor splicing type. The equipment includes 4 single sensors, and the band ranges are 790nm~1210nm, 1220nm~1640nm, 1650nm~2070nm, and 2080nm~2500nm, respectively. The use of a spliced ​​multi-sensor miniaturized near-infrared spectroscopy equipment can expand the single sensor equipment that initially only covers a band range of 420 nanometers to cover a band range of 1710 nanometers, which almost covers the entire near-infrared band range. In this way, the band range of the equipment can be effectively expanded to include as wide a band range as possible so as to cover the component absorption peaks of the sample to be tested as comprehensively as possible. The method for determining the absorption peak of a component is as follows: obtaining the wavelength point with the maximum absorbance value in the wavelength band range of 790nm to 2500nm, and the absorption peak at this wavelength point is the absorption peak of the sample component.

[0037] In specific implementation, a host can be determined from different near-infrared spectroscopy devices of the same type, and the rest are slaves; the method for determining the host can be randomly selected, or a standard device that can accurately collect spectra can be determined as the host based on experiments, which will serve as the basic device for modeling in this embodiment, and the spectral model of other devices directly adopts the spectral model of the host to maintain universality. The host and the slave are used to collect spectral data of the same batch of samples, so as to obtain spectral data collected by the host and spectral data collected by the slave respectively; the spectral data collected by the host and the spectral data collected by the slave are distributed to obtain the absorption peaks of the sample components.

[0038] S102, reconstructing sample spectrum data for the host and the slave respectively according to the peak point of the sample component absorption peak:

[0039] In this step, the wavelength points of the spectral data are reconstructed according to the wavelength range of the peak absorption peak of the sample component. Only a certain number of wavelength points around the component absorption peak are retained, and the redundant wavelength points in the initial multi-sensor spectral data are eliminated. The amount of spectral data is greatly reduced while retaining the sample spectral characteristic information, thereby improving the efficiency of spectral analysis.

[0040] In this embodiment, the resolution of the miniaturized splicing multi-sensor spectral device used is 10nm, and the band range is 790nm~2500nm. It can be calculated that the wavelength points contained in each multi-sensor spectral data are X1=1+(2500-790) / 10=172, and the band range of the single sensor is 420nm. It can be calculated that the wavelength points contained in each single-sensor spectral data are X2=1+420 / 10=43. The splicing multi-sensor spectral data is reconstructed for wavelength points, and the multi-sensor spectral data containing 172 wavelength points is reconstructed into single-sensor spectral data containing 43 wavelength points. Not only can the amount of spectral data be effectively reduced, but also the conventional single-sensor spectral data modeling method can be used for spectral modeling, which effectively improves the universality of spectral data and the efficiency of spectral analysis.

[0041] The initial multi-sensor spectral data wavelength point reconstruction is based on the single sensor wavelength point. The number of wavelength points contained in the reconstructed spectral data is the same as that of the single sensor spectral data. The wavelength range corresponding to the peak value of the component absorption peak is set to Pnm. If P%10≠0, where % is the remainder symbol, it is rounded off to get a new P value, and the original P value is replaced. With the wavelength range Pnm as the center point, X3 wavelength points are taken to the left and right for spectral reconstruction, where X3 is calculated as follows:

[0042] X3=(X2-1) / 2=21

[0043] The reconstructed spectral data is actually represented as a set of absorbance values ​​at 43 wavelength points, where the wavelength ranges of each wavelength point are [(P-210)nm, (P-200)nm, ... Pnm, ... (P+200)nm, (P+210)nm].

[0044] When (P-210)≤790, the wavelength points of the reconstructed spectrum overflow to the left and the spectrum reconstruction cannot be performed. When the above situation occurs, the leftmost wavelength range of the reconstructed spectrum is selected as 790nm. In this case, the wavelength ranges of each wavelength point after reconstruction are [790nm, 800nm, ... 1200nm, 1210nm].

[0045] When (P+210)≥2500, the wavelength points of the reconstructed spectrum overflow to the right and the spectrum reconstruction cannot be performed. When the above situation occurs, the rightmost wavelength range of the reconstructed spectrum is selected as 2500nm. It can be further known that in this case, the wavelength ranges of each wavelength point after reconstruction are [2080nm, 2090nm, ... 2490nm, 2500nm] respectively.

[0046] By processing all spectral data of the host and slaves according to the spectral reconstruction method of this step, new reconstructed spectral data are obtained, and these reconstructed spectral data are applied to modeling, prediction and compensation in subsequent steps. That is, all host and slave spectral data mentioned in subsequent steps are data after spectral reconstruction in this step.

[0047] S103, dividing the spectral data reconstructed by the host into a training set and a compensation set, and dividing the spectral data reconstructed by the slave into a compensation set:

[0048] In this step, the training set in the host sample set is a sample set with known calibration values ​​after calibration, which is used for spectral modeling; the host and slave compensation sets are sample sets with unknown calibration values. The samples contained in the sets are exactly the same and are used for difference compensation. There is no need to perform spectral modeling on the slave. The host spectral model is directly called for prediction, and the predicted values ​​are further compensated for differences for prediction and analysis of the component content of the samples to be tested.

[0049] In order to meet the requirements of miniaturized near-infrared spectroscopy analysis technology, the spectral model established by the host training set samples has high stability and strong prediction ability, and the host training set samples must not be less than 200. At the same time, in order to minimize the compensation value error, the number of samples of the host and slave compensation sets is selected to be 50. Too many compensation set samples will cause a waste of manpower and material resources, and the ability to improve the accuracy of spectral analysis is very limited. Too few compensation set samples will make the compensation error larger, affecting the accuracy of spectral analysis.

[0050] S104, using the training set of the host to perform spectral modeling, and using the established host spectral model to predict the compensation set of the host and the compensation set of the slave respectively:

[0051] In this step, the sample spectral data in the host training set are matched one by one with their component calibration values, and spectral modeling is performed to obtain the host spectral model M. The host model M is used to predict the component content of the 50 sample spectral data collected by the host in the host compensation set, and the component content prediction values ​​corresponding to the 50 samples are obtained, respectively (S1, S2, ... S 50 ), the predicted mean component content of 50 samples in the host compensation set is calculated as The host model M is used to predict the component content of 50 sample spectral data collected by the slave in the slave compensation set, and the component content prediction values ​​corresponding to the 50 samples are obtained respectively (C1, C2, ... C 50 ), the predicted mean of the component content of 50 samples in the slave compensation set is calculated as

[0052] S105. Calculate the difference between the predicted mean value of the master compensation set and the predicted mean value of the slave compensation set as the compensation value:

[0053] Because the samples contained in the compensation sets of the host and slave are exactly the same, their predicted values ​​should theoretically be exactly the same. However, due to the poor consistency of miniaturized near-infrared spectroscopy equipment, the spectral data of the samples collected by the host and slave have large deviations. The host model is used to predict the spectral data of the host and slave compensation sets, and there are large differences in the predicted value deviations. The difference between the mean predicted values ​​of the host and slave is used as the compensation value, which can well compensate and correct the predicted value of the slave and maximize the accuracy of the slave spectrum prediction.

[0054] In this embodiment, the average predicted value of 50 samples in the host compensation set is The mean predicted value of 50 identical samples in the slave compensation set is Calculate the difference between the mean of the predicted values ​​of the master compensation set and the mean of the predicted values ​​of the slave compensation set for:

[0055]

[0056] The above-mentioned difference is the slave compensation value. In actual spectral analysis applications, when the host spectral model is used to predict the component content of the spectral data of the sample to be tested collected by the slave, there is a large deviation between the slave prediction value and the actual content of the sample component. The slave prediction value needs to be compensated and corrected so that the slave prediction value is as close to the host prediction value as possible, that is, the actual component content of the sample, thereby improving the accuracy of the slave spectral prediction.

[0057] S106, using the host spectrum model to predict the spectrum data sampled by the slave, and performing difference compensation on the predicted value based on the compensation value:

[0058] In this step, there is no need to perform spectral modeling on the slave machine, and the host machine's spectral model can be directly called for prediction. Since the device consistency of the slave machine is significantly different from that of the host machine, there is a large deviation between its predicted value and the actual component content of the sample. Therefore, it is necessary to compensate the predicted value of the slave machine for the sample to be tested one by one to improve the accuracy of the slave machine's spectral prediction.

[0059] In this embodiment, if the spectral data contained in the sample to be tested collected by the slave is Q, the host spectral model is used to predict the Q sample spectral data collected by the slave, and the predicted values ​​of the component contents of the Q samples to be tested (H1, H2, ... H Q ), and then further perform difference compensation on the predicted values ​​of the component contents of the Q samples to be tested one by one, and obtain the difference compensation predicted values ​​of the component contents of the Q samples to be tested collected by the slave The above difference compensation prediction value is the actual prediction value of the component content of the sample to be tested by the slave.

[0060] Finally, it should be noted that the above embodiments are only preferred implementations and are not intended to limit the present invention. It should be pointed out that for those skilled in the art, several modifications, equivalent replacements, improvements, etc. can be made without departing from the scope of the present invention and the scope of protection of the claims, and all of these should be included in the protection scope of the present invention.

Claims

1. A difference compensation method based on spectral model transfer, characterized in that: The following steps are involved: S1. For different near-infrared spectroscopic devices of the same type, one of the hosts is determined, and the rest are slaves; the host and the slaves are used to collect spectrum data of the same batch of samples to obtain the absorption peaks of the sample components; S2. Reconstruct the sample spectrum data for the host and slave respectively according to the peak point of the sample component absorption peak: Only a certain number of wavelength points around the component absorption peak are retained, and redundant wavelength points in the collected initial spectral data of the sample are eliminated, so as to reconstruct the multi-sensor spectral data into single-sensor spectral data, that is, the number of wavelength points in the reconstructed spectral data is consistent with the number of wavelength points of the single-sensor spectral data in the near-infrared spectral device; the certain number is the number of wavelength points contained in the single-sensor spectral data; S3, dividing the sample spectral data reconstructed by the host into a training set and a compensation set, and calibrating the spectral data in the training set; Dividing the reconstructed sample spectral data of the slave into a compensation set having the same amount of spectral data as the compensation set of the master; S4, using the training set of the host to perform spectral modeling, using the established host spectral model to predict the compensation set of the host and the compensation set of the slave, respectively, to obtain the predicted mean of the host compensation set and the predicted mean of the slave compensation set; S5, calculating the difference between the predicted mean value of the master compensation set and the predicted mean value of the slave compensation set as the slave spectrum compensation value; S6. When a slave machine is used to perform spectral sampling and prediction on the same type of samples to be tested, the sampled spectral data is predicted by the host spectral model to obtain a predicted value, and then the predicted value is difference compensated based on the slave machine spectral compensation value; In step S2, only a certain number of wavelength points around the component absorption peak are retained, and redundant wavelength points in the collected initial spectrum data of the sample are eliminated, which specifically includes: Set the wavelength range corresponding to the peak value of the component absorption peak as P. If P%10≠0, where % is the remainder symbol, round it off to get a new P value, replace the original P value, and take the wavelength range P as the center point, and take X3 wavelength points to the left and right for spectrum reconstruction. The calculation method of X3 is: X3 = (X2-1) / 2, where X2 is the number of spectral wavelength points of a single sensor in the near-infrared spectroscopy device; The reconstructed spectral data is actually represented as a set of absorbance values ​​at X2 wavelength points, where the wavelength ranges of each wavelength point are: [(P-10*X3), (P-10*X3+10),…P,…(P+10*X3-10), (P+10*X3)]; When (P-10*X3)≤a, where a is the left boundary of the spectral band range of the near-infrared spectroscopy device, and the reconstructed spectral wavelength point overflows to the left, the wavelength point of the single sensor with the smallest band range in the near-infrared spectroscopy device is used as the wavelength point after spectral reconstruction; When (P+10*X3)≥b, where b is the right boundary of the spectral band range of the near-infrared spectroscopy device, and the reconstructed spectral wavelength point overflows to the right, the wavelength point of the single sensor with the largest band range in the near-infrared spectroscopy device is used as the wavelength point after spectral reconstruction.

2. A difference compensation method based on spectral model transfer as claimed in claim 1, characterized in that: In step S1, the near-infrared spectroscopy device adopts an equally divided spliced ​​near-infrared spectroscopy device, and the method for obtaining the absorption peak of the sample component includes: selecting a wavelength point with the largest absorbance value in the band range of the near-infrared spectroscopy device, and the absorption peak at the wavelength point is the absorption peak of the sample component.

3. A difference compensation method based on spectral model transfer as claimed in claim 1 or 2, characterized in that: In step S4, the method for respectively obtaining the predicted mean of the host compensation set and the predicted mean of the slave compensation set includes: assuming that the predicted values ​​of the component content prediction of the sample spectral data in the host compensation set using the established host spectral model are S1, S2, ..., S n , the predicted values ​​of component content prediction for sample spectral data in the slave compensation set are C1, C2, ... C n ; then the host compensation set prediction mean Slave compensation set prediction mean n is the number of sample spectral data in the master compensation set and the slave compensation set.

4. A difference compensation method based on spectral model transfer as claimed in claim 3, characterized in that: In step S5, the calculation method of the slave spectrum compensation value is: Slave spectrum compensation value 5. The difference compensation method based on spectral model transfer according to claim 4, characterized in that: In step S6, performing difference compensation on the predicted value based on the slave spectrum compensation value includes: Assume that the predicted values ​​of the component contents of the m samples to be tested are H1, H2, ... H m , then the predicted value after difference compensation is

Images