A refined oil tracing method based on spectral features and PCA-KNN

Through the method based on spectral characteristics and PCA-KNN, the portable hyperspectral oil detection equipment and K nearest neighbor algorithm are used to solve the problem of timeliness and cost of quality detection of refined oil, real-time traceability and quality verification of refined oil are achieved. Reduces the risk of pollution and goods collection.

CN118758880BActive Publication Date: 2024-11-22SHANGHAI SHENGGUI INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411253605.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-22
Estimated Expiration
2044-09-09

AI Technical Summary

Technical Problem

The existing quality inspection of refined oils is not time-sensitive, has high cost, and cannot monitor the quality of oil products in every batch in real time, affecting consumer safety and environment.

Method used

Using a method based on spectral characteristics and PCA-KNN, the spectral characteristic data of refined oil is collected through portable hyperspectral oil rapid detection equipment, and a spectral characteristic model library is established, and the K nearest neighbor algorithm is used to trace the refined oil, real-time detection and verification are achieved. .

Benefits of technology

It realizes the collection and detection of refined oil in low-cost and real-time under on-site operation of non-professional personnel, reducing the risk of pollution and cargo, and improving the timeliness and safety of oil quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118758880B_ABST
    Figure CN118758880B_ABST
Patent Text Reader

Abstract

The present invention relates to a finished oil source tracing method based on spectral features and PCA-KNN, which adopts a hyperspectral oil rapid detection device to obtain the unique spectral features of each batch of finished oil produced by a refinery, performs machine learning on the obtained spectral features of the finished oil through PCA and KNN algorithms, trains a model and establishes a spectral feature database, and a terminal customer uses the same hyperspectral oil detection device to perform real-time detection on the obtained finished oil, and the obtained spectral features are classified in the established spectral feature database through principal component feature recognition, data classification and regression algorithm, so as to achieve the purpose of tracing. The finished oil source tracing method provided by the present invention makes full use of the uniqueness of spectral features, does not need to go through a laboratory detection link, can achieve on-site real-time tracing of the refinery origin, category, batch and factory time of the finished oil, and improves the control of finished oil quality with low cost and simple operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of refined oil tracing, and in particular to a refined oil tracing method based on spectral features and PCA-KNN. Background Art

[0002] Finished oil, including gasoline, diesel and lubricating oil, is a resource product with special properties. It is widely used in all aspects of social production and people's lives, providing power for transportation, industrial production, agricultural machinery, etc. The quality of oil products is not only a concentrated reflection of the manufacturer's oil refining process level, but also the foundation for oil sales terminals such as gas stations and lubricant sales companies to provide consumers with safe, reliable and quality oil products.

[0003] However, affected by comprehensive factors such as the refinery's technical level, equipment and process flow, each production batch of oil products will have varying degrees of sulfur content, high olefin content or low octane number components and cetane number. Differences in additives and pollution caused by irregular transportation will also cause pollution. In addition, with the continuous and rapid development of social economy, the demand for refined oil resources is increasing. The recycled and filtered refined oil is put into the market as new oil, which not only damages the rights and interests of consumers, but the recycled and filtered refined oil may contain excessive sulfur, lead and other harmful substances. Discharging it into the atmosphere will seriously pollute the environment and affect air quality. It will also disrupt the normal market order and damage the development of the entire industry.

[0004] At present, the quality inspection of refined oil can only be carried out by sending sampling to professional laboratories for inspection. This is not only time-consuming and not timely, but the inspection fees of professional laboratories are high, which increases the cost of manufacturers. In addition, in the process of transporting oil samples to the laboratory, there will be problems such as improper storage that affect the accuracy of the test results. Professional laboratory inspections are also unable to synchronize every batch of the same type of refined oil. Summary of the invention

[0005] In view of this, the present invention provides a refined oil traceability method based on spectral features and PCA-KNN, which effectively solves the problems of lack of timeliness and high cost of refined oil quality detection in the prior art.

[0006] To achieve the above object, the present invention provides a method for tracing the source of refined oil based on spectral features and PCA-KNN, comprising the following steps:

[0007] S1. Spectral feature data set collection: a portable hyperspectral oil rapid detection device is used to obtain the unique spectral feature data of each production batch of finished oil, including reflectance and DN value;

[0008] S2. Use principal component analysis to establish a spectral feature model library, including:

[0009] S201, spectral feature data standardization;

[0010] S202, establishing a covariance matrix S;

[0011] S203, calculating the eigenvectors and eigenvalues ​​of the covariance matrix S;

[0012] S204, select the main component, sort the eigenvalues ​​obtained in step S203 in descending order, and select the top The eigenvalues ​​are taken as principal components;

[0013] S205, completing the establishment of the spectral feature model library;

[0014] S3. Establishing a refined oil prediction and tracing model: Using the K nearest neighbor algorithm, the new spectral feature vectors generated in step S2 are classified and trained in the spectral feature model library to establish a refined oil tracing model;

[0015] S301, machine learning training stage, storing the feature vectors of finished oil samples and their corresponding oil sample labels, and using the K nearest neighbor algorithm to classify and label the known feature vectors of finished oil samples;

[0016] S302, in the prediction and tracing stage, the K nearest neighbor algorithm calculates the distance between the finished oil sample to be traced and all the finished oil samples in the training set, identifies the nearest neighbor cluster, and classifies and labels the finished oil sample to be traced according to the nearest neighbor cluster to complete the tracing;

[0017] S4. Collect spectral characteristic data of the finished oil to be traced according to the method of step S1. The finished oil prediction and tracing model obtained by step S3 can quickly complete the tracing of the finished oil to be tested.

[0018] Preferably, it is characterized in that the spectral characteristic data standardization first establishes a matrix X of the finished oil sample and its spectral spectrum segment for the spectral characteristic data collected in step S1, and the dimension size is ,

[0019] ;

[0020] Among them, n represents the number of refined oil samples, p represents the spectral band contained in each refined oil sample, each row represents a refined oil sample, each column represents a spectral variable, and the spectral feature data is standardized:

[0021] ;

[0022] in, Represents the mean value of each column.

[0023] Preferably, the covariance matrix S is expressed as:

[0024] ;

[0025] in, express The transposed matrix of .

[0026] Preferably, the spectral feature model library is established by projecting the original spectral data collected in step S1 onto the principal component selected in step S204 to obtain Dimensionality reduction means:

[0027] ;

[0028] ;

[0029] Among them, Y represents the spectral feature model library, V represents the selected main component is a matrix of eigenvalues.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The hyperspectral oil rapid detection equipment used in the refined oil traceability method based on spectral features and PCA-KNN provided by the present invention can collect and detect oil samples in real time at a very low cost under on-site operation by non-professionals, can trace the refined oil to the refinery and its exit batch in real time, verify the quality of the refined oil in real time, and can reduce the risk of contamination, mixed goods and other unpredictable factors affecting the quality of the refined oil during transportation between the refined oil leaving the factory and the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a flowchart of the PCA algorithm of the present invention for establishing a spectral feature model library;

[0033] Figure 2 It is a spectral characteristic reflectance data diagram of the finished oil to be tested in the present invention;

[0034] Figure 3 It is a spectrum characteristic DN value diagram of the finished oil to be tested in the present invention;

[0035] Figure 4 It is the spectral characteristic model library diagram of 16 oil samples of the present invention;

[0036] Figure 5 The categories and clustering area diagrams of 5 different oil samples of the present invention are shown in FIG.

[0037] Figure 6 Establishing a model library diagram of feature vectors and feature values ​​of refined oil products for the PCA and KNN algorithms of the present invention;

[0038] Figure 7 It is the oil sample spectrum characteristic curve diagram of the present invention;

[0039] Figure 8 The spectral feature model library of oil sample A and the spectral feature model classification diagram of oil sample B of the present invention;

[0040] Fig. 9 The characteristic vector and characteristic value model library diagram established for 16 oil samples numbered D01-D16 of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0042] The present invention provides the following technical solutions:

[0043] A finished oil source tracing method based on spectral features and PCA-KNN uses the principal component analysis algorithm PCA (Principal Component Analysis) and the K nearest neighbor data classification and regression algorithm KNN (K Nearest Neighbors Algorithm) to establish a finished oil feature vector and feature value model library. The finished oil source tracing is determined by obtaining the classification of its feature vector and feature value in the geometric space of the model library;

[0044] The following steps are involved:

[0045] S1. Spectral feature data set collection: Use portable hyperspectral oil rapid detection equipment to obtain unique spectral feature data of at least 300 spectral segments in the 400nm-1000nm spectral region from each production batch of finished oil, including reflectance and DN value;

[0046] like Figure 2 , 3 As shown, high-dimensional data is used to reflect the composition characteristics of finished oil, the element data of finished oil that changes with time and use, and the differences between the same type of hyperspectral oil rapid detection equipment;

[0047] The hyperspectral oil rapid detection equipment uses hyperspectral quantitative detection of metal dust content and chemical composition in oil, breaking the traditional emission spectrum analysis method in the near infrared and infrared spectrum (800nm-2300nm), such as atomic spectrometer technology, simplifying the detection equipment, easy to carry (1kg), and all test results can be obtained in 5-6 seconds. The detection speed is fast, the operation is simple, and no professional personnel are required, which is convenient for refineries to test each batch of finished oil and establish a spectral feature library; at the same time, the hyperspectral oil rapid detection equipment only needs to be started and calibrated once a day, does not require consumables, and has low-cost operation and maintenance;

[0048] The principle of hyperspectral oil detection is to use a 400nm-1000nm halogen light source. The oil sample to be tested is placed in a rectangular, 3.5ml, two-sided transparent cuvette, which is inserted into the darkroom of the equipment for one-button operation. The hyperspectral oil rapid detection equipment obtains a spectrum of a specific wavelength through a reflected light path, which is called a hyperspectral characteristic spectrum line. The hyperspectral characteristic spectrum line is cut into 300 spectrum segments by a spectrometer, and the accuracy of each spectrum segment is 2nm. The reflected light is generally represented by the characteristic spectrum line of ion groups (molecules), and the half-peak full-wave width FWHM (Full width at half peak) of the spectrum segment peak is 2nm. maximum) is relatively wide, and the characteristic spectrum of the detected element (atom) is very narrow and approximately straight in the spectrum. Since there is no complete overlap of characteristic spectrum lines of two different elements in nature, the element spectrum line is extracted from the characteristic spectrum line to realize the identification and quantitative analysis of the detected element. When there is more than one detected element or its components in the detected oil sample, such as additives, a series of spectrum lines with various wavelengths corresponding to each element will appear in the spectrum, generally within the range of dozens of spectrum lines, and these spectrum lines are mixed with or even overlapped with the spectrum lines of other elements. These spectrum lines must be split to extract the target element spectrum line to realize the identification and quantitative analysis of the element, or reflect the characteristics of a certain component of the oil product based on multiple spectrum lines. In the process of spectral analysis, multiple spectrum lines with main characteristics within the spectrum line range are usually selected for model calculation to realize the detection of the composition and content of the target element. The entire spectral analysis calculation modeling is completed on the cloud server. Since the reflection principle is used to obtain the spectral characteristic spectrum line, the spectral characteristics obtained by putting the same finished oil into the colorimetric dish have very high consistency in fixed light path and darkroom structure, which is the key to the traceability of finished oil by high-spectral oil rapid detection equipment.

[0049] S2. Use principal component analysis to help identify the most important spectral bands or wavelengths by reducing the data dimension while retaining most of the variance, so as to establish a spectral feature model library, including:

[0050] S201, spectral feature data standardization;

[0051] According to the spectral feature data collected in step S1, a matrix X of the finished oil sample and its spectral spectrum is established. The dimension of the matrix X of the spectral spectrum of the finished oil sample is , where n represents the number of finished oil samples, p represents the spectral bands contained in each finished oil sample, each row represents a finished oil sample, each column represents a spectral variable, and with each additional oil sample, the matrix X adds a dimension with p spectral band variables;

[0052] In this embodiment, assuming that the number of oil samples is 16 and each sample has 300 spectral segments, the dimension size of the spectral segment matrix X of the finished oil sample is ,

[0053] ;

[0054] Standardize the spectral feature data:

[0055] ;

[0056] in, It represents the average value of 300 spectral bands;

[0057] S202, establish a covariance matrix S centered on the mean value, the covariance matrix provides information about the relationship between different spectral variables:

[0058] ;

[0059] in, express The transposed matrix of

[0060] S203. Calculate the eigenvectors and eigenvalues ​​of the covariance matrix S. The eigenvectors represent the direction of the maximum variance in the data, that is, the principal component, and the corresponding eigenvalues ​​represent the amount of variance explained by each principal component.

[0061] S204, select the main component, arrange the eigenvalues ​​obtained in step S203 in descending order, and select the top The eigenvalues ​​are used as principal components to capture most of the variance in the data. ;

[0062] S205: Complete the establishment of the spectral feature model library, and project the original spectral data collected in step S1 onto the principal component selected in step S204 to obtain Wei said:

[0063] ;

[0064] ;

[0065] Among them, Y represents the spectral feature model library, V represents the selected main component eigenvalues; Figure 4 The spectral feature model library Y of 16 lubricating oil products containing different metal components is shown. At this time, the data of the spectral feature model library has been reduced to dimensional space;

[0066] The PCA algorithm transforms the dimension of the original spectral band data matrix X from Reduced to the dimension size of the spectral feature model library matrix Y , which significantly reduces the dimension of the finished oil sample and its spectral matrix X, while retaining most of the important physical characteristic information, making the sample data easy to visualize, cluster and analyze; the principal component corresponds to the linear relationship of the original spectral band variables, and the principal component eigenvectors are orthogonal, indicating that there is no relationship between the eigenvectors. At the same time, the importance of the principal component eigenvectors decreases in sequence, which shows that the first principal component eigenvector is the most important.

[0067] S3. Establishing a refined oil prediction and tracing model: Using the K nearest neighbor algorithm, the new spectral feature vectors generated in step S2 are classified and trained in the spectral feature model library to establish a refined oil tracing model;

[0068] The "K" in the KNN algorithm refers to the number of adjacent points of the traceable point calculated after the known n oil samples are classified and separated. The value of K is less than or equal to the number of model training times. The KNN algorithm learns and optimizes in each step and iteration, eliminating the need for any specific learning stage or empirical threshold setting. The KNN algorithm is to classify all known spectral feature points stored in the spectral feature model library (multidimensional space) and the new spectral vectors generated by PCA in the model library, and complete the classification according to the proximity score between the nearest spectral feature model and the new spectral feature vector in the multidimensional space. This classification method is based on the majority vote (K times) of the spectral feature vectors in the multidimensional space or in the multidimensional space composed of the model training points.

[0069] S301, machine learning training stage, KNN algorithm stores the feature vector of finished oil sample and its corresponding oil sample label, and KNN algorithm classifies and labels the known feature vector of finished oil sample according to the input;

[0070] S302, in the prediction and tracing stage, the K nearest neighbor algorithm calculates the distance between the finished oil sample to be traced and all the finished oil samples in the training set, identifies the nearest neighbor cluster, and classifies and labels the finished oil sample to be traced according to the nearest neighbor cluster to complete the tracing;

[0071] Figure 5The figure is a binary classification diagram of the KNN algorithm. The same oil sample is tested (modeled) at different times, environmental conditions and equipment. Clustering areas are automatically established according to the distance. D1-D5 represent the categories and clustering areas of 5 different oil samples. The finished oil sample to be traced is traced by the KNN algorithm in the binary spectral feature vector space④. K depends on the number of training finished oil samples. K is generally defined as an odd number less than the number of training samples to facilitate majority voting to determine the category. In this embodiment, K=3, that is, in the binary spectral feature vector space, the three closest points are used to determine which category the finished oil sample to be traced belongs to. Figure 5 In the figure, three solid lines with directions are used to represent the distance from the traced point to any three nearest points, and the dotted lines with directions represent the distance from the traced point to each clustering area. The distance of the clustering area participates in the voting weight.

[0072] S4. Collect spectral characteristic data of the finished oil to be traced according to the method of step S1. The finished oil prediction and tracing model obtained by step S3 can quickly complete the tracing of the finished oil to be tested.

[0073] Classification based on the spectral characteristics of two different oil samples:

[0074] The hyperspectral oil rapid detection equipment uses a 400nm-1000nm halogen light source. The finished oil sample A is placed in a rectangular container with a volume of 3.5ml and a colorimetric dish with two transparent sides. It is inserted into the darkroom of the hyperspectral oil rapid detection equipment, and the spectrum of a specific wavelength is obtained through the reflected light path, such as Figure 7 The broken line ① is the characteristic line of the hyperspectral spectrum. Insert oil sample A into the darkroom of different hyperspectral oil rapid detection equipment to obtain the spectrum of a specific wavelength, such as Figure 7 ② The broken line in the middle; Repeat the operation of obtaining the spectrum of the specific wavelength of oil sample A to establish and train the spectral feature model library; The obtained spectral feature model library is as follows Figure 8 As shown in ①, insert oil sample B into the darkroom of the hyperspectral oil rapid detection equipment to obtain a spectrum of a specific wavelength, such as Figure 7 As shown in ③, the spectral characteristic model of oil sample B is as follows Figure 8 As shown in ②;

[0075] like Figure 8 As shown, the feature vectors of oil sample A and oil sample B in the spectral feature model library are significantly different, and there is an obvious distance between the clustering areas of oil sample B and oil sample A, so they are divided into different categories.

[0076] Modeling and tracing were performed using 16 known lubricant oil samples with different components:

[0077] Using a hyperspectral oil rapid detection device and a feature vector and feature value model library of 16 finished lubricating oil samples with different metal component contents, randomly select one of the 16 lubricating oil samples to simulate the traceability test to see if the result given by the spectral feature model library is consistent with the actual test oil sample;

[0078] The 16 oil samples were defined as D01-D16 and inserted into the hyperspectral oil rapid detection equipment. Each oil sample was tested 5 times in different environments, time and different cuvette containers for machine learning, model training and model library establishment. Fig. 9 As shown;

[0079] Among the 16 oil samples, one oil sample is randomly selected and inserted into the hyperspectral oil rapid detection equipment. The software in the computer will accurately feedback the number of the tested oil sample to achieve the purpose of equivalent traceability.

[0080] The refinery inputs each batch of finished oil into the model, and gas stations and other terminals conduct traceability:

[0081] Each refinery uses a hyperspectral oil rapid detection device to test each batch of finished oil products leaving the factory. Due to the low test cost, short time and simple operation, 5 oil samples are taken from each batch of finished products for testing (the test time is 5-6 seconds for one oil sample, plus the preparation work, the time required for this stage should be less than 10 minutes), which meets the purpose of machine learning, model training and model library establishment; the model library will update each test result input (the cloud server will recalculate the spectral feature model library), and the spectral feature model library can be classified according to the type of finished oil (gasoline, diesel or lubricating oil), which helps to reduce the amount of calculation for each spectral feature model library establishment;

[0082] Gas stations, oil depots and other terminals use hyperspectral oil rapid detection equipment to sample the received finished oil for real-time traceability verification. Any contaminated finished oil or finished oil from batches not produced by designated refineries during transportation will produce differences in the spectral feature model library, and cannot be traced back to the cluster space generated by the modeling of the designated batch of finished oil from the refinery, thereby achieving the purpose of controlling the quality of finished oil.

[0083] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A refined oil tracing method based on spectral features and PCA-KNN, characterized in that: The following steps are involved: S1. Spectral feature data set collection: a portable hyperspectral oil rapid detection device is used to obtain the unique spectral feature data of each production batch of finished oil, including reflectance and DN value; S2. Use principal component analysis to establish a spectral feature model library, including: S201, spectral feature data standardization; First, the matrix X of the finished oil sample and its spectral spectrum band is established for the spectral feature data collected in step S1, and the dimension size is , Where n represents the number of refined oil samples, p represents the spectral band contained in each refined oil sample, each row represents a refined oil sample, and each column represents a spectral variable. The spectral feature data is standardized: in, represents the mean value of each column; S202, establishing a covariance matrix S; S203, calculating the eigenvectors and eigenvalues ​​of the covariance matrix S; S204, select the main component, sort the eigenvalues ​​obtained in step S203 in descending order, and select the top The eigenvalues ​​are taken as principal components; S205, completing the establishment of the spectral feature model library; S3. Establishing a refined oil prediction and tracing model: Using the K nearest neighbor algorithm, the new spectral feature vectors generated in step S2 are classified and trained in the spectral feature model library to establish a refined oil tracing model; S301, machine learning training stage, storing the feature vectors of finished oil samples and their corresponding oil sample labels, and using the K nearest neighbor algorithm to classify and label the known feature vectors of finished oil samples; S302, in the prediction and tracing stage, the K nearest neighbor algorithm calculates the distance between the finished oil sample to be traced and all the finished oil samples in the training set, identifies the nearest neighbor cluster, and classifies and labels the finished oil sample to be traced according to the nearest neighbor cluster to complete the tracing; S4. Collect spectral characteristic data of the finished oil to be traced according to the method of step S1. The finished oil prediction and tracing model obtained by step S3 can quickly complete the tracing of the finished oil to be tested.

2. The method for tracing the source of refined oil based on spectral features and PCA-KNN according to claim 1, characterized in that: The covariance matrix S is expressed as: in, express The transposed matrix of .

3. The method for tracing the source of refined oil based on spectral features and PCA-KNN according to claim 2, characterized in that: The spectral feature model library is established by projecting the original spectral data collected in step S1 onto the principal component selected in step S204. Dimensionality reduction means: Among them, Y represents the spectral feature model library, V represents the selected main component is a matrix of eigenvalues.