A chemometric analysis method for spectra based on segmented intelligent optimization

Abstract

The present invention relates to a spectrum chemometric analysis method based on segmented intelligent optimization. The present invention uses all sample spectra and test analysis data and intelligently optimizes conditions to establish a main model. After the samples used for modeling are sorted according to physical property data, they are divided into several sections according to the characteristics of property distribution. Each section uses an intelligent algorithm to optimize the best spectrum pretreatment method and the spectrum area used for modeling, so as to respectively establish a sub-model of the index in each section, thereby improving the accuracy of the analysis result. In addition, after the samples are sorted according to the physical property data, according to the characteristics of property distribution, samples with similar types or structures and similar properties are classified into one category, and the sample spectra and data within the section are intelligently optimized for the pretreatment method, modeling area and other conditions to establish a corresponding segmented model; after segmenting according to the characteristics of property distribution, samples with similar types or structures can be classified into one category, and a separate sub-model can be established after the intelligent optimization conditions, which can effectively solve the problem that the accuracy of the analysis model is not high when the physical property variation range of the existing quick evaluation analysis method is large.

Description

Technical Field

[0001] The present invention relates to the technical field of quick evaluation and analysis of various raw materials and intermediate products in the petrochemical industry, and in particular to a spectrum chemometric analysis method based on segmented intelligent optimization. Background Art

[0002] For the petrochemical industry, the analysis and evaluation of raw materials and intermediate products plays a vital role in the production process. Enterprises need to use real-time analysis data of key indicators of raw materials and intermediate products to timely adjust the process conditions and parameters of each device in the production process to ensure that product quality meets the requirements while maximizing the economic benefits of the entire plant. Therefore, the accuracy and timeliness of the analysis data of each indicator are very important.

[0003] Usually, various raw materials or products need to analyze many indicators, and the types of analysis indicators for each material will also be different. In traditional laboratory analysis methods, each indicator generally requires professional personnel to use specific instruments to analyze separately. The entire analysis process requires multiple personnel and multiple instruments to be carried out simultaneously, resulting in high analysis costs, poor timeliness, and low efficiency, making it difficult to provide timely and accurate analysis data for production.

[0004] In recent years, many large petrochemical companies at home and abroad have gradually adopted modern instruments to achieve rapid evaluation and analysis of various raw materials or products, including near infrared spectroscopy (NIR), mid-infrared spectroscopy (MIR) and nuclear magnetic resonance spectroscopy (NMR). These rapid evaluation analysis methods first need to use instruments to scan samples to obtain the corresponding spectrum or spectrum of the sample, and then analyze multiple indicators of the sample at one time in a short time by analyzing the spectrum. The rapid evaluation analysis method greatly improves the analysis efficiency of samples, reduces the labor intensity and analysis cost of laboratory personnel, and provides timely analysis data of key indicators of each material to provide guidance for production.

[0005] For the quick-review analysis method, the most core process is the analysis of the sample spectrum. The analysis of the spectrum mainly includes two processes: (1) Establishment of the analysis model. Generally, before modeling, a certain amount of samples with laboratory analysis data obtained using standard analysis methods need to be accumulated, and the corresponding spectra of the samples need to be scanned using a quick-review analyzer. When modeling, the spectra of the accumulated samples are processed by a specific method and then associated with the laboratory analysis data to establish analysis models for each indicator; (2) Analysis of the spectrum. Using the established analysis model, the spectrum of the new sample scanned by the quick-review analyzer can be quickly analyzed to obtain the analysis data of each indicator of the sample. Therefore, the quality of each indicator analysis model directly determines the accuracy of the quick-review analysis data.

[0006] At present, the establishment process of each index analysis model is to first pre-process the sample spectrum by smoothing, derivative, appropriate normalization, multivariate scattering correction and other methods, and select the appropriate area according to the characteristics of each index and the relationship between the structure represented by each characteristic peak in the spectrum, and then associate it with the test analysis data through chemometric methods such as partial least squares method, and then establish the physical property analysis model of each index. This method is currently widely used, but overall, the accuracy of the model needs to be further improved. Summary of the invention

[0007] The present invention is to overcome the above-mentioned shortcomings, and the purpose is to provide a spectrum chemometric analysis method based on segmented intelligent optimization. The present invention uses all sample spectra and test analysis data and intelligent optimization conditions to establish a main model. Then, the samples used for modeling are sorted according to the physical property data and divided into several sections according to the property distribution characteristics. Each section is optimized by an intelligent algorithm to select the best spectrum pretreatment method and the spectrum area used for modeling, so as to establish a sub-model of the index in each section, thereby improving the accuracy of the analysis result. In addition, in addition to the main model, the present invention can also classify samples with similar types or structures and similar properties according to the property distribution characteristics after the samples are sorted according to the physical property data, and the corresponding segmented model is established after the sample spectra and data within the section are intelligently optimized for the pretreatment method, modeling area and other conditions; after segmenting according to the property distribution characteristics, samples with similar types or structures can be divided into one category, and a separate sub-model is established after the intelligent optimization conditions, so the model accuracy will be greatly improved, so it can effectively solve the problem that the analysis model accuracy is not high when the physical property change range of the existing quick evaluation analysis method is large.

[0008] The present invention achieves the above-mentioned purpose through the following technical scheme: a spectrum chemometric analysis method based on segmented intelligent optimization. The core algorithm involved in the present invention mainly includes three parts: intelligent optimization of the main model, intelligent optimization of the sub-model and analysis of the spectrum of new samples.

[0009] After collecting a certain number of sample spectra and corresponding laboratory analysis data in the early stage, the spectra can be divided into two parts: a calibration set and a validation set according to a certain algorithm. The establishment of the main model mainly involves intelligently selecting the preprocessing method and spectrum interval range for the spectra of all samples in the calibration set, and then correlating them with the conventional laboratory analysis data of the indicators using chemometric methods such as partial least squares to form the main model of the analysis process. The sample spectra in the calibration set can be preprocessed using mathematical methods such as smoothing, derivative, vector normalization, mean centering, mean variance conversion, standard normal variable transformation, and multivariate scattering correction. One or more of the preprocessing methods can be selected. After the preprocessing is completed, each data point in the calibration set spectrum can be correlated with the indicator laboratory analysis value, and the data points of the spectrum can be reordered according to the size of the correlation coefficient.

[0010] When the main model is established, the spectrum preprocessing method and the intelligent optimization of the interval range are mainly to process the spectrum according to a certain preprocessing method, and then arrange the spectrum data points in descending order according to the correlation coefficient with the modeling index, and then select the first 1, first 2, first 3...first n data points in this order to associate with the index test and analysis data (n is the total number of sample spectrum data points) to establish the main model of the index. After selecting the first point to complete the establishment of the main model, the main model is used to analyze and predict each spectrum in the verification set, and compare it with the actual test and analysis data, and calculate the average error. When the first two data points are taken for the second time, the spectrum in the verification set is also analyzed and predicted after the main model is established, the average error is calculated, and compared with the average error of the first time before, and the main model with the smaller average error is selected in the two times. By analogy, the average error of the main model analysis verification set established each time is calculated, and compared with the average error of the main model selected before. After the nth main model is established, the main model with the smallest average error will be retained as the best main model corresponding to the preprocessing method.

[0011] After completing the selection of the best master model under a certain spectrum preprocessing method, the calibration set spectrum preprocessing method can be adjusted according to the actual situation. Generally, one or more methods can be selected for spectrum preprocessing, or no selection can be made. When modeling each indicator, the type and quantity range of the spectrum preprocessing selection method can be artificially limited. Then, under the limited conditions, find the best master model corresponding to various preprocessing methods, and compare the average error of the analysis results when each master model analyzes the validation set spectrum. Select the master model with the smallest average error, which is the intelligently optimized master model.

[0012] When the range of physical property index values ​​varies greatly, in addition to the main model, segmented models can also be established for samples within different physical property ranges. To establish a segmented model, it is first necessary to sort the samples in the calibration set according to the physical property data and then classify the samples with similar types or structures and similar properties into one category based on the property distribution characteristics. Usually, the samples in the calibration set can be divided into one or more sections in areas with denser property distribution, but the number of samples in each section should be sufficient to ensure the accuracy of the segmented model.

[0013] After segmentation, the samples in each segment can be intelligently optimized for the spectrum preprocessing method and interval range. The specific optimization method is similar to the main model optimization process. The spectrum in the segmented interval must be processed according to a specific preprocessing method, and then the spectrum data points are arranged in descending order according to the correlation coefficient with the modeling index. Different data points are selected in this order to establish the sub-model of the index in the segment range, and the best sub-model is selected from them. Then, under limited conditions, the best sub-model corresponding to each preprocessing method is found, and a comprehensive comparison is made to select the sub-model with the smallest average error, which is the intelligently optimized sub-model.

[0014] When analyzing the spectrum of a new sample, you first need to use the main model to analyze and calculate the spectrum to see if the index prediction result is within the range of a certain physical property value when the sub-model is established. If the prediction result is not within any physical property value range, the main model prediction result is the final prediction result; if the prediction result is distributed in a certain physical property value range, you need to use the sub-model of that range to re-analyze and calculate the spectrum, and the sub-model prediction result is the final prediction result.

[0015] The operation steps of the method of the present invention mainly include seven processes: data collection, spectrum classification, spectrum preprocessing, interval sorting, main model intelligent optimization, sub-model intelligent optimization, spectrum analysis and the like.

[0016] (1) Data Collection

[0017] Before modeling, a certain amount of samples need to be accumulated. First, use the quick review analyzer to perform a spectral scan analysis on the accumulated samples to collect the sample's spectrum or spectral data set X (m*n), where m is the number of samples and n is the number of data points in the spectrum. Generally speaking, the more samples m there are, the higher the accuracy of the data and model. While collecting the spectral data, the macroscopic properties of the samples need to be analyzed using standard methods and traditional analytical instruments, and various physical property analysis data sets Y (m*n') are collected, where n' is the number of physical properties analyzed for this type of sample. When modeling one of the physical properties, the column vector of the property in the data set Y is extracted and merged into the last column of the spectral data set X to form a new data set Z (m*n+1) for the next step of model building.

[0018] (2) Spectral classification

[0019] After collecting the sample spectra and the corresponding analytical data, the spectra can be divided into calibration set Z according to the Kolmogorov-Smirnov algorithm. c (m1*n+1) and validation set Z v (m2*n+1) two parts, the sample ratio of the two data sets can be set manually, and the calibration set is generally set in the range of 60% to 80%. Among them, m1+m2=m.

[0020] (3) Spectrum preprocessing

[0021] Sample spectrum data set Z in the calibration set c The data except the last column can be preprocessed by mathematical methods such as smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, multivariate scatter correction, etc. to form the latest data set Z c—pre The preprocessing method can be selected from one or more of them, or it can be unselected. When modeling, the type and quantity range of the spectrum preprocessing selection method can be artificially restricted. v Select the same method as the calibration set for preprocessing to form the data set Z v—pre .

[0022] (4) Interval sorting

[0023] For dataset Z c—pre In the data set Z, the correlation analysis is performed from the 1st column to the nth column and the n+1th column respectively, and the 1st column to the nth column are rearranged in descending order according to the correlation coefficient to form a new data set Z. c—red , Z c—red Each column of data in the original data set Z c—pre The serial number in the dataset is recorded as data set D(1*n). At the same time, according to the sequence of the serial numbers in the dataset D, v—pre The data from the 1st to the nth columns are also rearranged to form a new data set Z v—red .

[0024] (5) Intelligent optimization of main model

[0025] Under a certain preprocessing method, the following operations are performed: Z is selected one by one c—red The first 1st column, the first 2nd column, the first 3rd column, ... the first nth column are respectively associated with the n+1th column of laboratory analysis data using partial least squares method to obtain regression coefficient data sets B1, B2, ... B n , which is the main model of the physical property under different data dimensions. Then, using data sets B1, B2, ... n For the validation set Z v—red The first 1, first 2, first 3, ... first n columns of the data set are analyzed and calculated to obtain the physical property analysis data of each m2 sample. Then, the analysis results of each sample are compared with the actual test analysis data, and the errors of each m2 sample and the overall average errors E1, E2, E3, ... E n If the minimum average error is E k , then data set B k This is the optimal main model under this preprocessing method.

[0026] Replace the calibration set spectra with other possible preprocessing methods within the constraints and calculate the minimum average error E in each case. k And the corresponding coefficient data set B k , and then comprehensively compare the E in all cases k value, select the smallest average error E k-best And the corresponding coefficient data set B k-best , which is the best main model for intelligent optimization.

[0027] (6) Intelligent Optimization of Models

[0028] In the calibration set, p segments of data sets are selected according to the distribution of sample physical property data, and are denoted as Z c1 (m 11 *n+1)、Z c2 (m 12 *n+1)……Z cp (m 1p *n+1), where m 11 +m 12 +……+m 12 ≤m1. Then repeat steps (3) to (5) for each region sample, and calculate the minimum average error E in each region according to the intelligent optimization method of the main model. k1-best 、E k2-best ……E kp-best And the corresponding coefficient data set B k1-bes , B k2-best ……B kp-best , which is the optimal sub-model for intelligent optimization of each segment.

[0029] (7) Spectrum analysis

[0030] The new sample spectrum dataset Z n (h*n) is processed according to the preprocessing method and interval sorting method selected by the main model to obtain the data set Z n—red Using the main model regression coefficient data set B k-best Z n—red The first k columns of data are analyzed and calculated to obtain the analytical result R of the new sample main model t .

[0031] If the main model analysis result is not within the range of the physical property values ​​of any data set in step (6), the final analysis result is R n If the prediction results are distributed in the jth range of physical property values ​​in step (6), the sub-model coefficient data set B of this range is required. kj-best Re-align Z n—red The first k columns of data are analyzed and calculated, and the model prediction results R nj This is the final prediction result.

[0032] The beneficial effects of the present invention are as follows: (1) The present invention adopts a main model + sub-model combined analytical method to predict the sample quick evaluation spectrum. When the range of physical property variation is large, the sample may contain multiple different types or structures. At this time, after all data are uniformly modeled, the model accuracy will be greatly affected; in this case, an additional sub-model is established for a specific area, which can greatly reduce the prediction error of the sample in the physical property section and improve the accuracy of the prediction result; (2) The present invention realizes the intelligent selection of the spectrum interval range, and the correlation analysis is performed between each data point of the spectrum and the physical property data during the modeling process. , and automatically selects the spectrum area involved in modeling according to the size of the correlation. While improving the accuracy of the model, it also reduces the manual operation and judgment process, and increases the convenience of the modeling process; (3) The spectrum preprocessing method of the present invention is richer and more intelligent. In the modeling process, spectrum preprocessing can select smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, multivariate scattering correction and other mathematical algorithms, and the types are richer; in addition, the modeling process can select the best preprocessing method according to the effect of the model under each processing method, so the selection process of the spectrum preprocessing method is also more intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a detection flow chart of the present invention;

[0034] Figure 2 is a crude oil sulfur content property distribution diagram of an embodiment of the present invention;

[0035] Figure 3 is a gasoline research octane property distribution diagram of an embodiment of the present invention;

[0036] Figure 4 is a comparison diagram of average errors under different pretreatment conditions of sulfur content in crude oil according to an embodiment of the present invention;

[0037] Figure 5 3 is a comparison diagram of average errors of gasoline octane numbers under different pretreatment conditions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto:

[0039] Example 1: The spectrum chemometric analysis method based on segmented intelligent optimization provided by the present invention is used to perform a quick evaluation and analysis on the infrared spectrum of crude oil from a refinery. The operation steps mainly include seven processes: data collection, spectrum classification, spectrum preprocessing, interval sorting, main model intelligent optimization, sub-model intelligent optimization, and spectrum analysis.

[0040] (1) Data Collection

[0041] Before modeling, a certain amount of samples need to be accumulated. First, a mid-infrared analyzer is used to perform spectral analysis on the accumulated 412 crude oil samples, and the mid-infrared spectrum data set X (412*1804) of the samples is collected. The wave number range of the mid-infrared spectrum of each crude oil is about 650~4000cm -1 , decomposed into 1804 data points within this range. While collecting the spectral data, 28 macroscopic properties of crude oil samples, such as density, sulfur content, nitrogen content, residual carbon, acid value, viscosity, water content, salt content, distillation range, etc., are analyzed using standard methods and traditional analytical instruments, and the physical property analysis data set Y (412*28) is collected. When modeling one of the physical properties, the column vector of the physical property in the data set Y is extracted and merged into the last column of the spectral data set X to form a new data set Z (412*1805) for the next step of model building. Let's take the sulfur content of crude oil as an example to illustrate.

[0042] (2) Spectral classification

[0043] The mid-infrared spectrum of crude oil is divided into a calibration set Z according to the Kolmogorov-Smirnov algorithm. c (330*1805) and validation set Z v (82*1805) two parts, the calibration set ratio is 80% of the total samples.

[0044] (3) Spectrum preprocessing

[0045] For the sample spectrum data set Z in the calibration set c The types of preprocessing methods for the first 1804 items are set to smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, and multivariate scatter correction. The number of preprocessing methods is set to 0 or 1. Then Z c There are 8 possible preprocessing schemes, and the processed calibration set is recorded as Z c—pre (330*1805). At the same time, after each calibration set selects the preprocessing method, the validation set Z v Select the same method as the calibration set for preprocessing to form the data set Z v—pre (82*1805).

[0046] (4) Interval sorting

[0047] Under the condition of no preprocessing method, the data set Z c—pre The correlation analysis is performed on the sulfur content physical property values ​​from column 1 to column 1804 and column 1805, and columns 1 to 1804 are rearranged in descending order according to the correlation coefficient to form a new data set Z. c—red (330*1805), Zc—red Each column of data in the original data set Z c—pre The serial number in the dataset is recorded as dataset D (1*1805). At the same time, according to the sequence of the serial numbers in dataset D, v—pre The data from columns 1 to 1804 are also rearranged to form a new data set Z v—red (82*1805).

[0048] (5) Intelligent optimization of main model

[0049] Select Z one by one c—red The first 1, first 2, first 3, ... first 1804 columns are respectively associated with the sulfur content values ​​of the 1805th column using the partial least squares method to obtain the regression coefficient data sets B1, B2, ... B 1804 , which is the main model of the physical property under different data dimensions. Then, using data sets B1, B2, ... 1804 For the validation set Z v—red The first 1, first 2, first 3, ... first 1804 columns of the data set are analyzed and calculated to obtain the physical property analysis data of the 82 validation set samples. Then the analysis results of each sample are compared with the actual test analysis data, and the individual errors of the 82 samples and the overall average errors E1, E2, E3, ... E 1804 The minimum average error is E 1709 , that is, when the first 1709 columns of data with a larger correlation coefficient are selected for modeling with sulfur content, the average error is the smallest. 1709 This is the optimal main model.

[0050] The calibration set was preprocessed using seven preprocessing methods: smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, and multivariate scatter correction. The minimum average error E in each case was calculated. k And the corresponding coefficient data set B k Table 1 shows the minimum average error of each method under the eight preprocessing schemes. It can be seen from the data in Table 1 that when the eighth preprocessing scheme (multiple scatter correction method) is selected to process the calibration set, the prediction result of the validation set is the best. The coefficient data set B under this method best , which is the best main model for intelligent optimization.

[0051]

[0052]

[0053] Table 1

[0054] (6) Intelligent Optimization of Models

[0055] The distribution of sulfur content analysis data in the sample set is as follows: Figure 2 According to the distribution of sample physical property data, two data sets of 0-1.2 and 2.6-4.2 are selected and recorded as Z c1 (169*1805), Z c2 (146*1805). Then repeat steps (3) to (5) for the two segment area samples, and calculate the minimum average error E in each segment area according to the intelligent optimization method of the main model. 1-best 、E 2-best And the corresponding coefficient data set B 1-bes , B 2-best , which is the optimal sub-model for intelligent optimization of each segment.

[0056] (7) Spectrum analysis

[0057] After the main model and sub-models are established, the spectrum of new crude oil samples can be analyzed and predicted. n (q*1804) is preprocessed according to the multivariate scattering correction method. When q is small, the new spectrum data set Z n and the calibration set Z used for modeling c , validation set Z v After merging and preprocessing together, separate them to get the prediction sample set Z n-pre Then, the first 1804 columns of data are rearranged according to the order in data set D to obtain data set Z. n—red Using the main model regression coefficient data set B best Z n—red The first k columns of data are analyzed and calculated to obtain the analytical result R of the new sample main model n .

[0058] If the main model analysis result is not within the range of 0 to 1.2 and 2.6 to 4.2, the final analysis result is R n ; If the prediction results are distributed in the jth range of physical property values, the sub-model coefficient data set B of this range is required j-best Re-align Z n—red The first k columns of data are analyzed and calculated, and the model prediction results R nj This is the final prediction result.

[0059] Example 2: The spectrum chemometric analysis method based on segmented intelligent optimization provided by the present invention is used to quickly evaluate and analyze the near-infrared spectrum of 92# gasoline in a refinery tank area. The operation steps mainly include seven processes: data collection, spectrum classification, spectrum preprocessing, interval sorting, main model intelligent optimization, sub-model intelligent optimization, and spectrum analysis.

[0060] (1) Data Collection

[0061] Before modeling, a certain amount of samples need to be accumulated. First, a near-infrared analyzer is used to perform spectral analysis on the accumulated 357 gasoline samples, and the near-infrared spectrum data set X (357*1298) of the samples is collected. The wave number range of each gasoline near-infrared spectrum is about 4000~14000cm -1 , decomposed into 1298 data points within this range. While collecting the spectral data, 12 macroscopic properties of gasoline samples, such as density, research octane number, motor octane number, distillation range, vapor pressure, benzene content, aromatic content, olefin content, oxygen content, etc., are analyzed using standard methods and traditional analytical instruments, and the physical property analysis data set Y (357*12) is collected. When modeling one of the physical properties, the column vector of the physical property in the data set Y is extracted and merged into the last column of the spectral data set X to form a new data set Z (357*1299) for the next step of model building. Now take the research octane number as an example to illustrate.

[0062] (2) Spectral classification

[0063] The gasoline near infrared spectrum is divided into the calibration set Z according to the Kolmogorov-Smirnov algorithm. c (286*1299) and validation set Z v (71*1299) parts, the calibration set ratio is 80% of the total samples.

[0064] (3) Spectrum preprocessing

[0065] For the sample spectrum data set Z in the calibration set c The types of preprocessing methods for the first 1298 items are set to smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, and multivariate scatter correction. The number of preprocessing methods is set to 0 or 1. Then Z c There are 8 possible preprocessing schemes, and the processed calibration set is recorded as Z c—pre (286*1299). At the same time, after each calibration set selects the preprocessing method, the validation set Z v Select the same method as the calibration set for preprocessing to form the data set Z v—pre (71*1299).

[0066] (4) Interval sorting

[0067] Under the condition of no preprocessing method, the data set Z c—pre The correlation analysis is performed on the research octane number values ​​of column 1 to column 1298 respectively, and the columns 1 to 1298 are rearranged in descending order according to the correlation coefficient to form a new data set Z. c—red(286*1299), Z c—red Each column of data in the original data set Z c—pre The serial number in the dataset is recorded as dataset D (1*1299). At the same time, according to the sequence of the serial numbers in dataset D, v—pre The data from columns 1 to 1298 in the dataset are also rearranged to form a new dataset Z v—red (71*1299).

[0068] (5) Intelligent optimization of main model

[0069] Select Z one by one c—red The first 1, first 2, first 3, ... first 1298 columns are respectively correlated with the research octane number value of the 1299th column using the partial least squares method to obtain the regression coefficient data sets B1, B2, ... B 1298 , which is the main model of the physical property under different data dimensions. Then, using data sets B1, B2, ... 1298 For the validation set Z v—red The first 1, first 2, first 3, ... first 1298 columns of the data set are analyzed and calculated to obtain the octane number analysis data of the 71 validation set samples. Then, the analysis results of each sample are compared with the actual test analysis data, and the errors of the 71 samples in each case and the overall average errors E1, E2, E3, ... E 1298 The minimum average error is E 1211 , that is, when the first 1211 columns of data with larger correlation coefficients are selected for modeling with the research octane number, the average error is the smallest. 1211 This is the optimal main model.

[0070] The calibration set was preprocessed using seven preprocessing methods: smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, and multivariate scatter correction. The minimum average error E in each case was calculated. k And the corresponding coefficient data set B k Table 2 shows the minimum average error of each method under the eight preprocessing schemes. From the data in Table 2, it can be seen that when the first preprocessing scheme (no preprocessing method) is selected, the prediction result of the validation set is the best. The coefficient data set B under this method best , which is the best main model for intelligent optimization.

[0071] Program No. Preprocessing methods Minimum average error Solution 1 No pre-processing 0.228145697 Solution 2 smooth 0.248177205 Solution 3 Derivative 0.247426024 Solution 4 Vector Normalization 0.268616612 Solution 5 Mean Centering 0.251932265 Solution 6 Mean variance 0.267560076 Solution 7 Standard normal variable transformation 0.251588032 Solution 8 Multivariate Scattering Correction 0.273317586

[0072] Table 2

[0073] (6) Intelligent Optimization of Models

[0074] The distribution of the research octane number analysis data in the sample set is as follows Figure 3According to the distribution of sample physical property data, we select the data set in the 92.1~94.1 region to establish a sub-model, denoted as Z c1 (279*1299). Then repeat steps (3) to (5) for the two segment area samples, and calculate the minimum average error E in the area according to the intelligent optimization method of the main model. 1-best And the corresponding coefficient data set B 1-bes , which is the optimal sub-model for intelligent optimization of each segment.

[0075] (7) Spectrum analysis

[0076] After the main model and sub-models are established, the new sample spectra can be analyzed and predicted. n (s*1298) Without any processing, the first 1298 columns of data are rearranged in the order of data set D to obtain data set Z n—red Using the main model regression coefficient data set B best Z n—red The first k columns of data are analyzed and calculated to obtain the analytical result R of the new sample main model n .

[0077] If the main model analysis result is not in the range of 92.1 to 94.1, the final analysis result is R n ; If the prediction results are distributed in the range of 92.1 to 94.1, the sub-model coefficient data set B in this range is required 1-best Re-align Z n—red The first k columns of data are analyzed and calculated, and the model prediction results R n1 This is the final prediction result.

[0078] The above embodiments use a spectrum chemometric analysis method based on segmented intelligent optimization, which greatly improves the accuracy of crude oil mid-infrared spectrum analysis results and gasoline near-infrared spectrum analysis results and the convenience of modeling process operation. Figure 4 and Figure 5 It can be seen that for the two material analysis indicators of crude oil and gasoline, when the method of the present invention is used to analyze the spectra of new samples, the average error of the prediction results of the validation set is significantly smaller than the average error when the conventional partial least squares method is used for analysis, which proves that the method of the present invention can greatly improve the accuracy of the analysis results of various samples in the field of petrochemical industry during quick evaluation analysis. In addition, for the pretreatment method and interval range selected in the modeling process, the method of the present invention can automatically select the best conditions, so the method has a high application value.

[0079] The above description is a specific embodiment of the present invention and the technical principles used. If the changes made according to the concept of the present invention do not exceed the spirit covered by the description and drawings, they should still fall within the scope of protection of the present invention.

Claims

1. A spectrum chemometric analysis method based on segmented intelligent optimization, characterized in that: It includes seven processes: data collection, spectrum classification, spectrum preprocessing, interval sorting, main model intelligent optimization, sub-model intelligent optimization, and spectrum analysis. The details are as follows: (1) collecting data on the accumulated samples to obtain a sample spectrum data set; (2) Spectral classification is performed based on the sample spectrum data set, which is divided into a calibration set and a validation set; (3) Preprocessing the calibration set and validation set after spectrum classification respectively; (4) performing interval sorting on the preprocessed calibration set and validation set respectively to obtain sorted calibration set and validation set; (5) Based on the calibration set and the validation set data set, the main model is intelligently selected; specifically, under a certain preprocessing method, the following operations are performed: Z is selected one by one c—red The first 1st column, the first 2nd column, the first 3rd column, ... the first nth column are respectively associated with the n+1th column of laboratory analysis data using partial least squares method to obtain regression coefficient data sets B1, B2, ... B n , which is the main model of sample properties under different data dimensions; then using data sets B1, B2...B n For the validation set Z v—red The first 1, first 2, first 3, and first n columns of the data set are analyzed and calculated to obtain the physical property analysis data of m2 samples; Then compare the analysis results of each sample with the actual test analysis data, and calculate the individual errors of m2 samples and the overall average errors E1, E2, E3...E n ; If the minimum average error is E k , then data set B k This is the optimal main model under this preprocessing method; among them, various other possible preprocessing methods of the calibration set spectrum are replaced within the constraints, and the minimum average error E in each case is calculated respectively. k And the corresponding coefficient data set B k , and then comprehensively compare the E in all cases k value, select the smallest average error E k-best And the corresponding coefficient data set B k-best , which is the best main model for intelligent optimization; (6) Based on the calibration set and the validation set data set, intelligent optimization of the sub-model is performed; specifically, p segments of data sets are selected in the calibration set according to the distribution of sample physical property data, and are denoted as Z c1 (m 11 *n+1)、Z c2 (m 12 *n+1)……Z cp (m 1p *n+1), where m 11 +m 12 +……+m 12 ≤m1; then repeat steps (3) to (5) for each area sample, and calculate the minimum average error E in each area according to the intelligent optimization method of the main model. k1-best 、E k2-best ……E kp-best And the corresponding coefficient data set B k1-bes , B k2-best ……B kp-best , which is the best sub-model for intelligent optimization of each segment; (7) Perform spectral analysis on the new sample spectral data set based on the main model and sub-model obtained by intelligent optimization, and output the analysis results.

2. The spectrum chemometric analysis method based on segmented intelligent optimization according to claim 1, characterized in that: The step (1) is specifically as follows: using a quick evaluation analyzer to perform a spectrum analysis on the accumulated samples, and collecting the spectrum or spectral data set X (m*n) of the samples, wherein m is the number of samples and n is the number of data points in the spectrum; while collecting the spectrum data, the macroscopic properties of the samples need to be analyzed using standard methods and traditional analytical instruments, and various physical property analysis data sets Y (m*n') are collected, wherein n' is the number of physical properties analyzed for this type of sample; when modeling one of the physical properties, the column vector of the sample physical property in the data set Y is extracted and merged into the last column of the spectrum data set X to form a new data set Z (m*n+1) for the next step of model building.

3. The spectrum chemometric analysis method based on segmented intelligent optimization according to claim 1, characterized in that: The step (2) is specifically as follows: the spectra in the data set are divided into a calibration set Z according to the Kolmogorov-Smirnov algorithm. c (m1*n+1) and validation set Z v (m2*n+1) two parts, the sample ratio of the two data sets can be set manually, among which the calibration set is preferably set in the range of 60% to 80%; wherein, m1+m2=m.

4. The spectrum chemometric analysis method based on segmented intelligent optimization according to claim 1, characterized in that: The step (3) is specifically as follows: c The data except the last column are preprocessed to form the latest data set Z c—pre ; The preprocessing method is selected from any one or more of smoothing, derivative, vector normalization, mean centering, mean variance, standard normal variable transformation, and multivariate scatter correction mathematical methods; it is also possible not to select a preprocessing method, and the type and quantity range of the spectrum preprocessing selection method can be manually limited during modeling; similarly, the validation set Z v Select the same method as the calibration set for preprocessing to form the data set Z v—pre .

5. The spectrum chemometric analysis method based on segmented intelligent optimization according to claim 1, characterized in that: The step (4) specifically comprises: c—pre In the data set Z, the correlation analysis is performed from the 1st column to the nth column and the n+1th column respectively, and the 1st column to the nth column are rearranged in descending order according to the correlation coefficient to form a new data set Z. c—red , Z c—red Each column of data in the original data set Z c—pre The serial number in is recorded as data set D(1*n); similarly, according to the serial number order in data set D, the preprocessed validation set Z v—pre The data from the 1st to the nth columns are also rearranged to form a new data set Z v—red .

6. The method for chemometric analysis of spectra based on segmented intelligent optimization according to claim 1, characterized in that: The step (7) is specifically as follows: (7) The new sample spectrum dataset Z n (h*n) is processed according to the preprocessing method and interval sorting method selected by the main model to obtain the data set Z n—red ; Using the main model regression coefficient data set B k-best Z n—red The first k columns of data are analyzed and calculated to obtain the analytical result R of the new sample main model t ; If the main model analysis result is not within the range of the physical property values ​​of any data set in step (6), the final analysis result is R n If the prediction results are distributed in the jth range of physical property values ​​in step (6), the sub-model coefficient data set B of this range is required. kj-best Re-align Z n—red The first k columns of data are analyzed and calculated, and the model prediction results R nj This is the final prediction result.

Images