An evaluation method for spectral standardization cases
The principal component score error rate (PCSER) was calculated by principal component analysis and partial least squares method, which solved the problem of evaluating the effect of spectral standardization, achieved fast and accurate spectral standardization evaluation, improved model sharing and spectral similarity, and saved time and cost.
Patent Information
- Application Number
- CN202210067342.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-20
- Publication Date
- 2025-10-10
- Estimated Expiration
- 2042-01-20
AI Technical Summary
The existing technology lacks an effective method to evaluate the quality of spectral standardization in advance, which means that the spectral analysis model needs to be re-established when the instrument or environment changes, which consumes a lot of manpower and material resources. In addition, traditional evaluation methods are time-consuming and have many error factors.
The principal component loading matrix and score matrix were calculated by principal component analysis, the correction model was established using partial least squares method, and the principal component score error rate (PCSER) was calculated to evaluate the effect of spectral standardization. The smaller the PCSER value, the better.
It achieves rapid and accurate evaluation of spectral standardization effects, improves the shareability of models between different instruments, saves time costs, and improves spectral similarity.
Smart Images
Figure SMS_1 
Figure SMS_2 
Figure SMS_3
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of spectral analysis, and particularly relates to an evaluation method for spectral standardization. BACKGROUND
[0002] The near-infrared spectral analysis technology has the advantages of rapidness, accuracy and greenness, and is widely applied in the fields of food and medicine. However, when the detection conditions, detection environment or instrument equipment change, the absorbance of the near-infrared spectrum will be different, and thus the established analysis model is invalid, and it is necessary to establish a new model, which consumes a large amount of manpower and material resources.
[0003] In view of the above problems, the spectral standardization method is generally used to cope with them. The spectral standardization is to correct the spectral data collected by the same sample under different instruments or different conditions, so as to eliminate the difference caused by different instruments, environments and other external factors in the measurement process, so that the spectra collected under different conditions can be applied to the same model, and the model sharing is realized.
[0004] However, there is currently a lack of a good evaluation method for the execution of the spectral standardization. In the current application, the standardized spectrum is usually input into the spectral analysis model, and then the prediction result (equivalent to the measured value) is output by the model, and the prediction result is compared with the true result (i.e. the true value), and the error between the measured value and the true value is used to indirectly infer the good or bad of the spectral standardization. This evaluation method cannot be evaluated in advance, and it is necessary to complete all the work to know the good or bad, and thus a long time is required. Meanwhile, in addition to the standardization, there are many factors that affect the final prediction error, that is, the size of the prediction error cannot definitely indicate the good or bad of the standardization. SUMMARY
[0005] In view of the deficiencies in the prior art, the application provides an evaluation method for the spectral standardization, so as to solve the evaluation problem of the spectral standardization.
[0006] The application achieves the above technical purpose by the following technical means.
[0007] An evaluation method for the spectral standardization, comprising the following steps:
[0008] Step 1, calculating the score matrix: decomposing the master spectrum X m into the principal component loading matrix P m and the principal component score matrix T m , substituting the slave spectrum X' t and the P m into the formula X = TP T+E X , where E X is the residual matrix, and the principal component score matrix T′ of the slave machine is calculated t ;
[0009] Step 2: Calculate the principal component score error rate: m and the T′ t Substitute the following formula to calculate the principal component score error rate PCSER:
[0010]
[0011] Where T m,i T m The score rate of the i-th principal component in , T′ t,i T′ t The score rate of the i-th principal component, W i is the contribution rate of the i-th principal component in the spectral analysis model, and n is the number of principal components.
[0012] Furthermore, the quality of spectral standardization is judged by the size of the PCSER value. The smaller the PCSER value, the better the spectral standardization.
[0013] Furthermore, based on the principal component analysis method, the host spectrum X m Decompose.
[0014] Furthermore, multiple sets of data are tested, and the average PCSERave or the maximum value PCSERmax is taken from the multiple PCSER results.
[0015] Furthermore, the spectral band range is the near-infrared light band.
[0016] Furthermore, the spectral analysis model is established by partial least squares method.
[0017] Furthermore, the spectra were preprocessed by normalization and multivariate scattering correction.
[0018] The beneficial effects of the present invention are:
[0019] This paper provides an evaluation method for spectral standardization. Using a calibration model established using the partial least squares method, it can assess spectral differences and improve spectral similarity, thereby enabling model sharing between different instruments. Furthermore, compared to traditional evaluation methods, this method eliminates the need for a complete set of model predictions, thus saving significant time and costs. DETAILED DESCRIPTION
[0020] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown, and the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below by reference are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.
[0021] 1. Basic Concepts
[0022] There are two spectrometers, a master and a slave. The master's spectral data is used as a benchmark to establish a spectral analysis model. In order to enable the slave's spectral data to directly apply the above spectral analysis model (based on the master), the slave's spectral data needs to be standardized.
[0023] It should be noted that the above concepts of master and slave do not narrowly define them as two different machines; if the same machine is used in two different environments and the measured spectral data changes, the above concepts of master and slave can also be applied, that is, the machine in one of the environments is considered the master, and the machine in the other environment is considered the slave.
[0024] For the convenience of expression, let the subscripts m and t represent the master and slave respectively, and then let the master spectrum be X m , the original spectrum from the machine is X t , the spectrum after normalization from the machine is X′ t , specific m 、X t and X′ t The data form of is a matrix, so it can also be called a spectral matrix. Obviously, the most ideal standardized result is X′ t =X m Furthermore, the method for spectral normalization is not the subject of this invention; those skilled in the art may employ any known spectral normalization method to perform the corresponding spectral normalization operation. This invention merely provides a method for evaluating the quality of spectral normalization results.
[0025] In the field of spectral analysis, near-infrared light spectrum is most commonly used to detect and analyze the composition of substances. Therefore, the following embodiments are also based on near-infrared spectroscopy to illustrate the technical solutions of the present invention.
[0026] 2. Evaluation Method
[0027] Step 1: Calculate the score matrix
[0028] Based on the principal component analysis method, the spectrum matrix is decomposed to obtain the host spectrum X m The principal component loading matrix P in m And the principal component score matrix T m .
[0029] The following operational relationship exists between the principal component loading matrix P, the principal component score matrix T, and the spectral matrix X:
[0030] X=TP T +E X (1)
[0031] Among them E X is the residual matrix, which is also obtained by principal component decomposition, P T is the transpose of P.
[0032] Then the spectrum X′ after normalization from the machine will be t And the host's principal component load matrix P m Substitute into formula (1) and calculate the principal component score matrix T′ of the slave machine t .
[0033] Step 2: Calculate the principal component score error rate
[0034] The principal component score matrix T of the host m And the principal component score matrix T′ of the slave t Substitute the following formula:
[0035]
[0036] Wherein PCSER is the principal component score error rate defined in the present invention; T m,i is the principal component score matrix T of the host m The score rate of the i-th principal component in t,i T′ t The score rate of the i-th principal component in W i is the contribution rate of the i-th principal component in the spectral analysis model; n is the number of principal components.
[0037] Finally, the present invention judges the quality of spectral standardization by the size of the PCSER value. The smaller the PCSER value, the better the spectral standardization. To ensure the reliability of the results, multiple samples can be collected, multiple groups of data can be tested accordingly, and the average PCSERave or the maximum value PCSERmax can be taken from multiple PCSER results for judgment.
[0038] 3. Effect Test
[0039] 1) Collect samples
[0040] A total of 154 samples of wheat flour of different varieties or brands were collected, including high-gluten wheat flour, medium-gluten wheat flour, low-gluten wheat flour, self-raising flour, and whole-wheat flour. This test focused on the crude protein content of wheat flour.
[0041] 2) Near-infrared spectrum acquisition
[0042] Two spectrometers of different models were selected as the master and slave respectively. The master was a grating scanning S450 near-infrared spectrometer produced by Shanghai Lingguang Technology Co., Ltd., with a band range of 900-2500nm, a wavelength interval of 2nm, and a total of 801 wavelength points. In order to keep consistent with the band of the slave, the band range was trimmed to 1550-1950nm in actual use; the slave was a Fabry-Perot interferometer N500 near-infrared spectrometer produced by Jinan Haineng Instrument Co., Ltd., with a band range of 1550-1950nm, a wavelength interval of 2nm, and a total of 201 wavelength points.
[0043] The above-mentioned master and slave devices were used to collect near-infrared spectra of 154 portions of wheat flour.
[0044] 3) Determination of crude protein content in wheat flour
[0045] The crude protein content of 154 wheat flour samples was tested in accordance with the method specified in GB / T 31578-201 "Inspection of Cereals and Oils - Determination of Crude Protein in Cereals and Their Products - Dumas Combustion Method". The specific instrument used was the D500 Dumas Nitrogen Analyzer produced by Jinan Haineng Instrument Co., Ltd.
[0046] 4) Establish a spectral analysis model
[0047] Based on the spectral data collected by the host and the crude protein content data measured for the corresponding wheat flour, a spectral analysis model was established using the partial least squares method. In order to eliminate irrelevant information and noise interference in the spectral data, normalization + multivariate scattering correction was used to preprocess the spectrum.
[0048] 5) Spectral standardization
[0049] The slave spectra were standardized using three standardization methods: ① Direct Standardization (DS), ② Piecewise Direct Standardization (PDS), and ③ Simple Linear Regression Direct Standardization (SLRDS). Three sets of slave standardized spectra were obtained accordingly.
[0050] 6) Calculate the principal component score error rate
[0051] According to steps 1 and 2 above, the principal component score error rates (PCSERs) corresponding to the three normalization results are calculated respectively. For added comparison, the original spectrum of the slave without normalization is also calculated according to steps 1 and 2. The corresponding results are shown in the following table:
[0052] Table 1: Error rate of principal component scores
[0053]
[0054] 7) Model prediction
[0055] The slave spectra before standardization and the slave spectra after standardization in the three ways were applied to the established spectral analysis model, and the crude protein content of wheat flour was predicted using the model. The results were compared with the actual measured values, and the respective prediction errors were calculated. The final results are shown in Table 2, where Rp is the prediction correlation coefficient, RMSEP is the prediction standard deviation, and RPD is the relative prediction deviation. The closer the Rp value is to 1, the smaller the RMSEP value is, and the larger the RPD value is, the higher the prediction accuracy of the corresponding model is.
[0056] Table 2: Model prediction results
[0057]
[0058] A comprehensive comparison of the data in Tables 1 and 2 shows that both the standardization evaluation method of the present invention and the actual model prediction results show that the DS algorithm performed best in this experiment, followed by the PDS algorithm and then the SLRDS algorithm. This demonstrates that the standardization evaluation method of the present invention can effectively and accurately assess the quality of spectral standardization.
[0059] Based on the same inventive concept as the above-mentioned spectral standardization evaluation method, the present invention also provides an electronic device, which includes one or more processors and one or more memories, wherein a computer-readable code is stored in the memory, wherein the computer-readable code, when executed by one or more processors, can calculate the corresponding principal component score error rate PCSER. The memory may include a non-volatile storage medium and an internal memory; the non-volatile storage medium can store an operating system and a computer-readable code. The computer-readable code includes program instructions, which, when executed, enable the processor to perform test question similarity calculations. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory provides an environment for the operation of the computer-readable code in the non-volatile storage medium. When the computer-readable code is executed by the processor, the processor can execute the test question similarity calculation method based on problem-solving ideas and knowledge points of the present invention. It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.
[0060] In the description of the present invention, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.
[0061] The present invention is not limited to the above-mentioned embodiments. Any obvious improvement, replacement or modification that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the scope of protection of the present invention.
Claims
1. A method for evaluating spectral standardization, characterized in that: The steps include: Step 1, calculate the score matrix: from the host spectrum X m Decompose the principal component load matrix P m And the principal component score matrix T m , the slave spectrum X t ′ And the P m Substitute the formula X = TP T +E X , calculate the principal component score matrix T of the slave machine t ′ ; Among them E X is the residual matrix, T is the principal component score matrix, P is the principal component loading matrix, and P T is the transpose of P, X represents the spectrum matrix; when calculating, X t ′ Substitute X, P into the formula m Substitute P into the formula, and the calculated T is T t ′ ; Step 2: Calculate the principal component score error rate: m and the T t ′ Substitute the following formula to calculate the principal component score error rate PCSER: Where T m,i T m The score rate of the i-th principal component, T t ′ ,i T t ′ The score rate of the i-th principal component, W i is the contribution rate of the i-th principal component in the spectral analysis model, and n is the number of principal components.
2. The method for evaluating spectral standardization according to claim 1, wherein: The quality of spectral standardization is judged by the size of the PCSER value. The smaller the PCSER value, the better the spectral standardization.
3. The method for evaluating spectral standardization according to claim 1, wherein: Based on principal component analysis, the host spectrum X m Decompose.
4. The method for evaluating spectral standardization according to claim 2, wherein: Test multiple sets of data and take the average PCSERave or maximum PCSERmax among multiple PCSER results.
5. The method for evaluating spectral standardization according to claim 1, wherein: The spectral band ranges from the near-infrared band.
6. The method for evaluating spectral standardization according to claim 5, wherein: The spectral analysis model is established by partial least squares method.
7. The method for evaluating spectral standardization according to claim 6, wherein: The spectra were preprocessed by normalization and multivariate scatter correction.