Insulating oil Raman spectrum baseline subtraction method and system based on iterative adaptive weighted algorithm
Through iterative adaptive weighting algorithm and sparse matrix technology, the parameters are automatically adjusted, and the problem of fluorescence background interference in transformer oil Raman spectroscopy is solved, the signal-to-noise ratio and analysis accuracy are improved, and the deduction process is simplified.
Patent Information
- Application Number
- CN202210530321.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-05-16
AI Technical Summary
In the prior art In transformer oil Raman spectroscopy analysis, fluorescence background interference leads to baseline deviation, signal-to-noise ratio decreases, and the deduction process is complicated and errors are easily introduced, affecting the accuracy of the analysis.
The baseline deduction method of insulating oil Raman spectral based on iterative adaptive weighting algorithm is adopted. By constructing a balance function between accuracy and smoothness, combining sparse matrix technology and iterative adaptive weighting algorithm, parameters are automatically adjusted to realize the deduction of background signals. .
Effectively remove background interference from spectral data, improve signal-to-noise ratio, is simple and fast, avoid user intervention, and ensure analysis accuracy, especially when signal-to-noise is relatively low, the effective signal remains unchanged.
Smart Images

Figure CN114965425B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of transformer oil Raman spectrum data analysis, and relates to an insulating oil Raman spectrum baseline subtraction method and system based on an iterative adaptive weighted algorithm. Background Art
[0002] Raman spectroscopy is a spectral analysis method that uses molecular vibration-rotation information. Due to its high sensitivity to subtle chemical and biochemical changes and its adaptability to non-invasive applications, this technology has been widely used in structural detection and performance analysis of solid and liquid materials.
[0003] In the Raman spectroscopy analysis of transformer oil, the most important interfering factor is fluorescence. Due to the fluorescence influence of organic molecules or pollutants in the sample, the Raman spectrum often produces a fluorescent background signal, which manifests as a typical inclined wide background, causing the baseline to deviate and the signal-to-noise ratio to decrease, affecting the further analysis and processing of the data. It must be deducted in advance before analysis.
[0004] By using the background subtraction algorithm, the baseline can be lowered, and the background can be subtracted without losing useful information, so as to obtain a more ideal analysis result. Since Raman light is scattered light, it cannot be fully applied to the Lambert-Beer law for quantitative analysis. In order to better perform quantitative analysis on the sample, it is necessary to pre-process the data with appropriate methods before analysis to obtain better quantitative analysis results.
[0005] At present, the main methods for subtracting the Raman spectrum baseline include wavelet transform method, polynomial fitting method, displacement excitation Raman difference method, etc. The above methods can indeed effectively realize the Raman spectrum baseline subtraction, but to a certain extent, they also manipulate the pure Raman spectrum required for measurement. Usually, the above methods also rely on human intervention to select adjustable input parameters. Therefore, the applied technology and input parameter values must be selected very carefully to ensure the minimum operation of the required pure Raman spectrum. The implementation is complex and inefficient, and it is easy to introduce errors, affecting the accuracy of Raman spectrum data analysis. Summary of the invention
[0006] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide an insulating oil Raman spectrum baseline subtraction method and system based on an iterative adaptive weighted algorithm, which can effectively remove the baseline signal caused by the insulating oil fluorescence background when measuring spectral data.
[0007] In order to achieve the above object, the present invention adopts the following technical solution:
[0008] The insulating oil Raman spectrum baseline subtraction method based on iterative adaptive weighting algorithm includes the following steps:
[0009] Step 1: Collect an oil sample of the actual operating transformer insulating oil to obtain the original Raman spectrum analysis signal X of the transformer insulating oil;
[0010] Step 2: Let Z be the fitting vector obtained by deducting the baseline from X, and establish the accuracy calculation model F between Z and X and the smoothness calculation model R of Z;
[0011] Step 3: Based on the calculation models F and R, a smoothing adjustment parameter α of Z is introduced to construct a balance function Q describing the balance between the accuracy and smoothness;
[0012] Step 4: After introducing the iterative weighting factor into the balance function Q, the expression of the fitting vector Z is derived;
[0013] Step 5: Based on the expression of the fitting vector Z, the sparse matrix technology and iterative adaptive weighted algorithm are used to solve the fitting vector Z to achieve the baseline subtraction of the insulating oil Raman spectrum.
[0014] The present invention further includes the following preferred embodiments:
[0015] Preferably, in step 1, an oil sample of the actually operating transformer insulating oil is collected, and an original Raman spectroscopy analysis signal X of the transformer insulating oil is obtained through a laser Raman spectroscopy detection platform.
[0016] Preferably, in step 2, the accuracy calculation model F between Z and X is established using the sum of squared errors, specifically:
[0017]
[0018] Where m is the length of Z, x i 、z i are the values of the i-th data of the original Raman spectrum analysis signal X and the fitting vector Z respectively.
[0019] Preferably, in step 2, the smoothness calculation model R of Z is established by using the sum of the square differences of two adjacent items, specifically:
[0020]
[0021] Preferably, in step 3, a penalized least squares algorithm is used to construct a balance function Q describing the balance between the accuracy and smoothness, specifically:
[0022] Q=F+αR=||XZ|| 2 +α||DZ|| 2 (3)
[0023] Among them, α is the smoothing adjustment parameter of Z;
[0024] DZ = ΔZ, ΔZ is Δz iThe matrix formed.
[0025] Preferably, step 4 is specifically:
[0026] Derivative the equilibrium function Q and make it equal to 0, that is The following linear equation is obtained:
[0027] (I+αD′D)Z=X (4) Where D is the difference matrix, D′ is the transposed matrix of D, and I represents a matrix with a matrix value of 1;
[0028] By introducing the iterative weighting factor w i , change the precision between Z and X to:
[0029]
[0030] Where W is w i diagonal matrix on the diagonal;
[0031] Then equation (4) becomes equation (6):
[0032] (W+αD′D)Z=WX (6) According to formula (6), the expression of the fitting vector Z is obtained:
[0033] Z=(W+αD′D) -1 WX (7).
[0034] Preferably, step 5 specifically includes:
[0035] Step 501: Initialize the diagonal matrix W, D and calculate αD′D;
[0036] Step 502: Calculate W using sparse matrix technology t +αD′D;
[0037] Step 503: According to W t +αD′D solves the fitting vector of the tth iteration;
[0038] Step 504: Determine whether the number of iterations or convergence conditions are met. If so, the number of iterations is increased by one and the process returns to step 502. Otherwise, the fitting vector Z is output to achieve the baseline subtraction of the insulating oil Raman spectrum.
[0039] Preferably, in step 502, W is calculated t +αD′D, W t In each iteration According to formula (9), we can obtain:
[0040]
[0041] Among them, the vector dt From X and the fitted value Z t-1 The difference is negative.
[0042] Fitted value Z t-1 is the baseline value of the previous iteration;
[0043] t represents the iteration number;
[0044] The W matrix obtained by formula (9) is a sparse matrix. If x i Greater than or equal to It is regarded as part of the Raman spectrum peak, and its iterative weighting factor is set to zero to ignore its effect in the next iterative fitting, thereby automatically and step by step eliminating the peak part and saving the baseline part.
[0045] Preferably, in step 504, the convergence condition is:
[0046] |d t |=0.0001×|X| (10).
[0047] The present invention also provides an insulating oil Raman spectrum baseline subtraction system based on an iterative adaptive weighted algorithm, which is used to implement the insulating oil Raman spectrum baseline subtraction method based on an iterative adaptive weighted algorithm.
[0048] The beneficial effects of the present invention are as follows:
[0049] The present invention can effectively deduct the background interference signal of Raman spectral data, highlight the characteristic of the component signal to be measured, ensure the validity of the spectral data, and improve the signal-to-noise ratio of the original spectral signal;
[0050] The present invention is simple and fast, does not require user intervention, and can subtract the background while keeping the effective signal unchanged in the case of low signal-to-noise ratio, thereby avoiding the introduction of errors in the algorithm itself during the calculation process and improving the accuracy of Raman spectroscopy data analysis.
[0051] 1. The iterative adaptive weighted algorithm proposed in the present invention is a step-by-step approximation background fitting algorithm, which does not require user intervention and can subtract the background while keeping the effective signal unchanged in the case of low signal-to-noise ratio;
[0052] 2. The present invention utilizes sparse matrix technology, which can quickly complete the background subtraction of a large number of modeling spectra;
[0053] 3. The present invention gradually approaches the background through an iterative adaptive weighted algorithm, introduces parameters to adjust the smoothness of the curve, and subtracts the fitted background from the original signal to obtain a spectral signal after the background is subtracted. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a flow chart of the insulating oil Raman spectrum baseline subtraction method based on iterative adaptive weighted algorithm of the present invention;
[0055] Figure 2 It is a flow chart of obtaining the fitting vector Z by using an iterative adaptive weighted algorithm in an embodiment of the present invention;
[0056] Figure 3 It is the comparison result of Raman spectra before and after iterative adaptive weighting algorithm processing in the embodiment of the present invention. DETAILED DESCRIPTION
[0057] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0058] like Figure 1 As shown, Example 1 of the present invention provides a method for Raman spectrum baseline subtraction of insulating oil based on an iterative adaptive weighted algorithm. In a preferred but non-limiting embodiment of the present invention, the method comprises the following steps 1-5:
[0059] Step 1: Collect an oil sample of the actual operating transformer insulating oil to obtain the original Raman spectrum analysis signal X of the transformer insulating oil;
[0060] Further preferably, according to the collected actual operating transformer oil sample, original Raman spectrum data of the transformer oil is obtained on a laser Raman spectrum detection platform.
[0061] The oil fluorescence background causes the Raman signal of the transformer oil to have a higher baseline, and as the aging degree increases, the baseline also moves upward, which in turn affects the accuracy of the spectral data analysis. In order to eliminate or weaken the influence of the baseline, the original spectral data needs to be baseline subtracted.
[0062] Step 2: Let Z be the fitting vector obtained by deducting the baseline from X, and establish the accuracy calculation model F between Z and X and the smoothness calculation model R of Z;
[0063] That is, X is the original spectral signal, and the fitting vector Z refers to Figure 3 The output spectrum signal in .
[0064] Further preferably, assuming that X is the measured transformer oil Raman spectrum analysis signal, Z is the fitting vector, and both have a length of m;
[0065] The accuracy of Z and X can be expressed by the sum of their squared errors, as shown in formula (1):
[0066]
[0067] xi 、z i are the values of the i-th data of the original Raman spectrum analysis signal X and the fitting vector Z respectively.
[0068] The smoothness of vector Z can be expressed by the sum of the square differences of its two adjacent terms, as shown in formula (2):
[0069]
[0070] Step 3: Based on the calculation models F and R, a penalty least squares algorithm is used to construct a balance function Q describing the balance between the accuracy and smoothness;
[0071] Further preferably, the balance between accuracy and smoothness can be achieved by adding the accuracy term to the term after the roughness penalty, as shown in formula (3):
[0072] Q=F+αR=||XZ|| 2 +α||DZ|| 2 (3)
[0073] Among them, α is the smooth adjustment parameter of Z, which can be adjusted by the operator;
[0074] The larger the α is, the smoother the fitted Z is.
[0075] That is, the balance between smoothness and accuracy can be adjusted by the operator himself using the α parameter.
[0076] D corresponds to the difference matrix, that is, DZ = ΔZ. ΔZ is Δz i The matrix formed is denoted as DZ=ΔZ, that is, ΔZ is expressed as the product of D and Z, and D is called the difference matrix.
[0077] Step 4: After introducing the iterative weighting factor into the balance function Q, the expression of the fitting vector Z is derived;
[0078] Further preferably, the balance function Q (3) is derived and made equal to 0, that is, A linear equation can be obtained, as shown in formula (4);
[0079] The destructive process is:
[0080]
[0081] Further
[0082] -X+Z+αD′DZ=0
[0083] Finally, we can get:
[0084] (I+αD′D)Z=X (4) Where D is the difference matrix, D′ is the transposed matrix of D, and I represents a matrix with a matrix value of 1;
[0085] By introducing weights, i.e. iterative weighting factors w i , set any value, such as 0, the formula (1) of Z accuracy for spectral signal X can be changed as follows:
[0086]
[0087] Where W is w i A diagonal matrix on the diagonal.
[0088] Equation (4) becomes equation (6):
[0089] (W+αD′D)Z=WX (6) Solving the above linear equation (6), the fitting vector Z can be easily obtained, as shown in equation (7):
[0090] Z=(W+αD′D) -1 WX (7)
[0091] Formula (7) is used to solve the fitting vector in step 503.
[0092] Step 5: Based on the expression of the fitting vector Z, the sparse matrix technology and iterative adaptive weighted algorithm are used to solve the fitting vector Z to achieve the baseline subtraction of the insulating oil Raman spectrum.
[0093] Further preferably, in order to correctly determine the peak position and gradually approach the background, a sparse matrix and an iterative weighting factor are used, specifically;
[0094] Introducing the iterative weighting factor w into the balance function Q (3) i , can be changed to:
[0095]
[0096] Among them, the iterative weighting factor w i It is obtained by using an adaptive iterative method, and an initial value w is given at the beginning 0 =1, after assignment, calculate W t +αD′D, W t In each iteration According to formula (9), we can obtain:
[0097]
[0098] Among them, the vector d t By X and Z t-1 The difference is negative.
[0099] Fitted value Zt-1 is the baseline value from the previous iteration.
[0100] The W matrix obtained by formula (9) is a sparse matrix. If x i Greater than or equal to is considered as part of the Raman spectrum peak (i.e., x i for Fitted background ), set its iterative weighting factor to zero to ignore its role in the next iterative fitting (i.e. subtract the original signal from Fitted background , the spectral signal after background subtraction can be obtained), thereby automatically eliminating the peak part step by step and saving the baseline part.
[0101] The iterative method stops when it reaches the maximum number of iterations or the convergence condition. The convergence condition is given as follows:
[0102] |d t |=0.0001×|X| (10)
[0103] Specifically, Figure 2 As shown, step 5 specifically includes:
[0104] Step 501: Initialize the diagonal matrix W, D, set α, and calculate αD′D;
[0105] W is usually initialized to a matrix with all diagonal values set to 1 and all other values set to 0. D is usually initialized to the identity matrix; α is recommended to be set to 10e 4 ;
[0106] Step 502: Calculate W using sparse matrix technology t +αD′D;
[0107] Step 503: According to W t +αD′D solves the fitting vector of the tth iteration;
[0108] Step 504: Determine whether the number of iterations or convergence conditions are met. If so, the number of iterations is increased by one and the process returns to step 502. Otherwise, the fitting vector Z is output to achieve the baseline subtraction of the insulating oil Raman spectrum.
[0109] In a specific embodiment of the present invention, the Raman spectra comparison results before and after the iterative adaptive weighting algorithm are as follows: Figure 3 shown.
[0110] The insulating oil Raman spectrum baseline subtraction system based on iterative adaptive weighted algorithm is used to implement the insulating oil Raman spectrum baseline subtraction method based on iterative adaptive weighted algorithm.
[0111] The beneficial effects of the present invention are as follows:
[0112] The present invention can effectively deduct the background interference signal of Raman spectral data, highlight the characteristic of the component signal to be measured, ensure the validity of the spectral data, and improve the signal-to-noise ratio of the original spectral signal;
[0113] The present invention is simple and fast, does not require user intervention, and can subtract the background while keeping the effective signal unchanged in the case of low signal-to-noise ratio, thereby avoiding the introduction of errors in the algorithm itself during the calculation process and improving the accuracy of Raman spectroscopy data analysis.
[0114] 1. The iterative adaptive weighted algorithm proposed in the present invention is a step-by-step approximation background fitting algorithm, which does not require user intervention and can subtract the background while keeping the effective signal unchanged in the case of low signal-to-noise ratio;
[0115] 2. The present invention utilizes sparse matrix technology, which can quickly complete the background subtraction of a large number of modeling spectra;
[0116] 3. The present invention gradually approaches the background through an iterative adaptive weighted algorithm, introduces parameters to adjust the smoothness of the curve, and subtracts the fitted background from the original signal to obtain a spectral signal after the background is subtracted.
[0117] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.
Claims
1. A method for subtracting the baseline of insulating oil Raman spectrum based on an iterative adaptive weighted algorithm, characterized in that: The method comprises the following steps: Step 1: Collect an oil sample of the actual operating transformer insulating oil to obtain the original Raman spectrum analysis signal X of the transformer insulating oil; Step 2: Let Z be the fitting vector obtained by deducting the baseline from X, and establish the accuracy calculation model F between Z and X and the smoothness calculation model R of Z; Step 3: Based on the calculation models F and R, a smoothing adjustment parameter α of Z is introduced to construct a balance function Q describing the balance between the accuracy and smoothness; Step 4: After introducing the iterative weighting factor into the balance function Q, the expression of the fitting vector Z is derived; Step 5: Based on the expression of the fitting vector Z, the sparse matrix technology and iterative adaptive weighting algorithm are used to solve the fitting vector Z to achieve the baseline subtraction of the insulating oil Raman spectrum, which includes: Step 501: Initialize the diagonal matrix W, D and calculate αD′D; Step 502: Calculate W using sparse matrix technology t +αD′D; Calculate W t +αD′D, W t In each iteration According to formula (9), we can obtain: Among them, the vector d t From X and the fitted value Z t-1 The difference is negative. Fitted value Z t-1 is the baseline value of the previous iteration; t represents the iteration number; The W matrix obtained by formula (9) is a sparse matrix. If x i Greater than or equal to It is regarded as a part of the Raman spectrum peak, and its iterative weighting factor is set to zero to ignore its role in the next iterative fitting, thereby automatically and step by step eliminating the peak part and saving the baseline part; Step 503: According to W t +αD′D solves the fitting vector of the tth iteration; Step 504: determine whether the number of iterations or convergence conditions are met. If so, the number of iterations is increased by one and the process returns to step 502. Otherwise, the fitting vector Z is output to achieve the baseline subtraction of the insulating oil Raman spectrum. The convergence conditions are: |d t |=0.0001×|X| (10)。 2. The method for insulating oil Raman spectrum baseline subtraction based on iterative adaptive weighted algorithm according to claim 1, characterized in that: In step 1, an oil sample of the actually running transformer insulating oil is collected, and an original Raman spectroscopy analysis signal X of the transformer insulating oil is obtained through a laser Raman spectroscopy detection platform.
3. The method for insulating oil Raman spectrum baseline subtraction based on iterative adaptive weighted algorithm according to claim 1, characterized in that: In step 2, the accuracy calculation model F between Z and X is established using the sum of squared errors, specifically: Where m is the length of Z, x i 、z i are the values of the i-th data of the original Raman spectrum analysis signal X and the fitting vector Z respectively.
4. The method for insulating oil Raman spectrum baseline subtraction based on iterative adaptive weighted algorithm according to claim 3 is characterized in that: In step 2, the smoothness calculation model R of Z is established by using the sum of the square differences of two adjacent items, specifically:
5. The method for insulating oil Raman spectrum baseline subtraction based on iterative adaptive weighted algorithm according to claim 4, characterized in that: In step 3, a penalized least squares algorithm is used to construct a balance function Q describing the balance between the accuracy and smoothness, specifically: Q=F+αR=||X-Z|| 2 +α||DZ|| 2 (3) Among them, α is the smoothing adjustment parameter of Z; DZ = ΔZ, ΔZ is Δz i The matrix formed.
6. The method for insulating oil Raman spectrum baseline subtraction based on iterative adaptive weighted algorithm according to claim 5, characterized in that: Step 4 is as follows: Derivative the equilibrium function Q and make it equal to 0, that is The following linear equation is obtained: (I+αD′D)Z=X (4) Where D is the difference matrix, D′ is the transposed matrix of D, and I represents a matrix with a matrix value of 1; By introducing the iterative weighting factor w i , change the precision between Z and X to: Where W is w i diagonal matrix on the diagonal; Then equation (4) becomes equation (6): (W+αD′D)Z=WX (6) According to formula (6), the expression of the fitting vector Z is obtained: Z=(W+αD′D) -1 WX (7)。 7. Insulating oil Raman spectrum baseline subtraction system based on iterative adaptive weighted algorithm, characterized by: The system is used to implement the insulating oil Raman spectrum baseline subtraction method based on the iterative adaptive weighted algorithm as described in any one of claims 1 to 6.
