A gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine
By calculating the projection matrix in subspace learning and iterative training using soft labels, adaptively balancing edge distribution and conditional distribution, the problems of unsatisfactory classification performance and high label cost in the prior art are solved, and the high accuracy of gas sensors is achieved Rate drift compensation.
Patent Information
- Application Number
- CN202210522582.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-13
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2042-05-13
AI Technical Summary
Existing drift compensation methods based on subspace learning reduce distribution differences only by aligning the edge distribution between the source domain and the target domain, without evaluating the conditional distribution between the two domains, resulting in poor classification performance and increasing Tag cost.
Using a gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine, the projection matrix is calculated in the subspace, the source domain and target domain data are projected to the common subspace, and iteratively trained using soft labels. Adaptively balance edge distribution and conditional distribution, reduce label costs and improve classification accuracy.
The dual drift compensation effect at the feature level and decision-making level is realized, the classification accuracy is improved, the label cost is reduced, and the gas classification tasks can be adapted to long-term and short-term drift scenarios, and the calculation is simple and easy to implement.
Smart Images
Figure CN114925762B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a gas sensor drift compensation method and belongs to the field of pattern recognition. Background Art
[0002] A gas sensor is a gas analysis system that converts the volume fraction of a certain gas into a corresponding electrical signal. Due to its advantages such as low detection cost, simple operation, and small size, gas sensors have been used in many industrial fields, including food analysis, environmental monitoring, medical diagnosis, and explosive detection. However, due to the effect of drift, the feature distribution of the data collected by the gas sensor will change over time. In this case, traditional machine learning methods are not competent for classification tasks with drift effects.
[0003] At present, existing drift compensation techniques usually use data processing, subspace learning or learning a robust classifier to suppress sensor drift, but the classification accuracy of most methods is not ideal (accuracy is less than 90%). On the other hand, drift compensation methods based on migration samples require the use of labeled target domain samples in the training phase of the model. In fact, the samples in the target domain will continue to increase over time. In order to adapt to the changes in gas sensor drift, drift compensation methods based on migration samples must collect more labeled samples. However, in the real world, collecting enough labeled samples is expensive and time-consuming. In addition, most drift compensation methods based on subspace learning only reduce the distribution difference by aligning the marginal distribution between the source domain and the target domain, without evaluating the conditional distribution between the two domains, which will reduce the classification performance of the drift compensation method. Summary of the invention
[0004] In order to solve the problem that most drift compensation methods based on subspace learning only reduce the distribution difference by aligning the marginal distribution between the source domain and the target domain without evaluating the conditional distribution between the two domains, which reduces the classification performance of the drift compensation method and increases the label cost, a gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine is proposed.
[0005] The technical solution adopted by the present invention to solve the above problems is: the specific steps of the method of the present invention are as follows:
[0006] Step 1: Collect source domain and target domain datasets;
[0007] Step 2: Establish feature set;
[0008] Step 3: Subspace learning;
[0009] Step 4: Build and train a gas classification model;
[0010] Step 5: Predict the label of the target domain data.
[0011] Furthermore, in step 1, the same gas sensor is used to collect raw data of gas in different time periods, or different gas sensors with the same configuration are used to collect raw data of gas at the same time, to establish a source domain dataset S and a target domain dataset T.
[0012] Furthermore, in step 2, multiple features are extracted based on the collected raw data, and a feature set is established, namely:
[0013]
[0014]
[0015] In formula (1) and formula (2), X S represents the source domain feature set, X T represents the target domain feature set, D represents the feature set dimension, and N S Represents the number of samples in the source domain, N T represents the number of samples in the target domain, represents the i-th sample in the source domain, i=1,2,...,N S , represents the jth sample in the target domain, j = 1, 2, ..., N T .
[0016] Furthermore, in step 3, the projection matrix P is calculated * , and the source domain X S and the target domain X T Projected to the common subspace, the source domain of the projected subspace is Y S , the target domain is Y T .
[0017] Furthermore, the projection matrix P * The calculation formula is:
[0018] P * =[P 1 ,P 2 , ..., P d ](3),
[0019] In formula (3), P represents the eigenvector corresponding to the dth smallest eigenvalue, d = 1, 2, 3, ...
[0020] Furthermore, in step 4, in the subspace, a guidance sample set is established, and the guidance sample set and source domain data are input into CAELM for training;
[0021] In the first iteration, CAELM selects ELM as the classifier, and only source domain samples participate in the training of the classifier, and the classification results of the classifier on the target domain are used as soft labels; these soft labels are used for subspace learning and gas classification model training in the next iteration;
[0022] In the subsequent iterations, CAELM selects DAELM as the classifier using the soft labels obtained in the previous iteration.
[0023] Furthermore, in step five, after the subspace and gas classification models have been trained for multiple iterations and meet the requirements, the trained models are used to classify the target domain samples to obtain the final classification results.
[0024] The beneficial effects of the present invention are as follows: the method of the present invention designs drift compensation strategies at both the feature level and the decision level, thereby achieving the effect of dual suppression of sensor drift with minimal label cost and improving classification accuracy; the present invention proposes a subspace learning method, which can minimize the feature distribution difference between the source domain and the target domain in the subspace while retaining the useful information of the original data; the present invention solves the problem that most drift compensation methods based on subspace learning only reduce the distribution difference by aligning the marginal distribution between the source domain and the target domain, but do not evaluate the conditional distribution between the two domains, thereby reducing the classification performance of the drift compensation method; for the continuously evolving drift (the feature distribution is constantly changing), the proposed subspace learning method can adaptively balance the importance of the marginal distribution and the conditional distribution, and can adaptively balance the energy of the source domain and the target domain in the subspace, which enables the present invention to simultaneously solve the gas classification task in the long-term drift scenario and the short-term drift scenario; the proposed subspace learning method is solved by eigenvalue decomposition, and the calculation is simple and easy to implement; the present invention uses the soft label of the target domain to calculate the conditional distribution between the source domain and the target domain, and at the decision level, uses the soft label of the target domain to improve the cross-domain migration ability of the model. Therefore, in the model training stage, the present invention does not need to use the real labels of the target domain samples, thereby reducing the label cost; the subspace and classifier are continuously optimized by iteratively refining the soft labels, thereby improving the classification accuracy. Therefore, the present invention achieves dual drift compensation effects at the feature level and the decision level. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a schematic diagram of the process of the present invention;
[0026] Figure 2 It is a schematic diagram of the present invention. DETAILED DESCRIPTION
[0027] Specific implementation method 1: Combination Figure 1 and Figure 2This embodiment is described. The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment is implemented by the following steps:
[0028] Step 1: Collect source domain and target domain datasets;
[0029] Step 2: Establish feature set;
[0030] Step 3: Subspace learning;
[0031] Step 4: Build and train a gas classification model;
[0032] Step 5: Predict the label of the target domain data.
[0033] Specific implementation method 2: Combination Figure 1 and Figure 2 To illustrate this embodiment, in step 1 of the gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment, the same gas sensor is used to collect raw data of gas in different time periods, or different gas sensors with the same configuration are used to collect raw data of gas at the same time to establish a source domain dataset S and a target domain dataset T.
[0034] Specific implementation method three: Combination Figure 1 and Figure 2 To illustrate this embodiment, in step 2 of the gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment, multiple features are extracted according to the collected original data, and a feature set is established, namely:
[0035]
[0036]
[0037] In formula (1) and formula (2), X S represents the source domain feature set, X T represents the target domain feature set, D represents the feature set dimension, and N S Represents the number of samples in the source domain, N T represents the number of samples in the target domain, represents the i-th sample in the source domain, i=1,2,...,N S , represents the jth sample in the target domain, j = 1, 2, ..., N T .
[0038] Specific implementation method four: Combination Figure 1 and Figure 2In this embodiment, the projection matrix P is calculated in step 3 of the gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment. * , and the source domain X S and the target domain X T Projected to the common subspace, the source domain of the projected subspace is Y S , the target domain is Y T .
[0039] In the first iteration of this embodiment, since the soft labels of the target domain samples cannot be obtained, α 1 = 0. In other words, during the first iteration, subspace learning only aligns the marginal distributions without considering the conditional distributions.
[0040] Specific implementation method five: Combination Figure 1 and Figure 2 The projection matrix P of the gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment is described as follows: * The calculation formula is:
[0041] P * =[P 1 ,P 2 , ..., P d ](3),
[0042] In formula (3), P represents the eigenvector corresponding to the dth smallest eigenvalue, d = 1, 2, 3, ...
[0043] Specific implementation method six: Combination Figure 1 and Figure 2 This embodiment is described. In step 4 of the gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment, in the subspace, a guidance sample set is established, and the guidance sample set and source domain data are input into CAELM for training;
[0044] In the first iteration, CAELM selects ELM as the classifier, and only source domain samples participate in the training of the classifier, and the classification results of the classifier on the target domain are used as soft labels; these soft labels are used for subspace learning and gas classification model training in the next iteration;
[0045] In the subsequent iterations, CAELM selects DAELM as the classifier using the soft labels obtained in the previous iteration.
[0046] Specific implementation method seven: Combination Figure 1 and Figure 2To illustrate this embodiment, in step five of the gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine described in this embodiment, after the subspace and gas classification models have been trained for multiple iterations and meet the requirements, the trained model is used to classify the target domain samples to obtain the final classification result.
[0047] How it works
[0048] The present invention proposes a subspace learning method, which aims to minimize the feature distribution difference between the source domain and the target domain in the subspace while retaining the useful information of the original data, that is, to achieve gas sensor drift compensation at the feature level;
[0049] Assumptions is the source domain dataset, where D represents the sample dimension, N S represents the number of samples in the source domain, represents the i-th sample in the source domain, i = 1, 2, ..., N S ; represents the target domain, where N T represents the number of samples in the target domain, represents the jth sample in the target domain, j = 1, 2, ..., N T ;set up is the projection matrix, which can project data from the D-dimensional original space to the d-dimensional common subspace (d≤D), ||·|| 2 represents the 2-norm, tr(·), (·) -1 and(·) T denote the trace, inverse, and transpose of a matrix, respectively.
[0050] In the common subspace based on P, minimizing the distance of the marginal distribution can reduce the data distribution difference between the source domain and the target domain. The calculation formula is as follows:
[0051]
[0052] In formula (4), μ S represents the center of the source domain in the subspace, μ T represents the center of the target domain in the subspace, represents the i-th sample of the source domain in the subspace, that is, represents the jth sample of the source domain in the subspace, that is, Therefore, formula (4) can be rewritten as:
[0053]
[0054] In formula (5), u S represents the center of the source domain in the original space, that is, represents the center of the target domain in the original space, that is,
[0055] Similarly, minimizing the distance of the conditional distribution can also reduce the distribution difference between the source domain and the target domain. The calculation formula is as follows:
[0056]
[0057] In formula (6), c∈{1,2,…,C} represents the class labels of the source domain and the target domain. represents the number of samples belonging to category c in the source domain, represents the number of samples belonging to class c in the target domain, represents the center of the c-class sample in the source domain in the original space, Represents the center of the c-class sample in the target domain in the original space;
[0058] However, in the real world, it is difficult to collect enough labeled target domain samples and it is impossible to accurately calculate the conditional distribution distance. In order to estimate the conditional distribution distance, the present invention uses a classifier to predict the label of the target domain sample. The predicted label is called the soft label of the target domain sample. Finally, the target domain with soft labels is used to estimate the conditional distribution distance. Therefore, formula (6) can be rewritten as:
[0059]
[0060] In formula (7), (^) represents the classification result of the classifier. represents the jth sample in the subspace belonging to the target domain c class, and Indicates the number of samples belonging to class c. represents the jth sample belonging to the target domain c in the original space, Represents the center of the sample belonging to class c in the original space;
[0061] In order to adaptively balance the importance of marginal distribution and conditional distribution, the present invention uses the balance factor α 1 (α 1 ∈[0,1]) combines (5) and (4) together:
[0062]
[0063] In addition, maximizing the energy of the source and target domains in the subspace can prevent the data from being distorted after projection, thereby retaining more available information; the calculation formula is as follows:
[0064]
[0065]
[0066] In fact, the number of samples in the source domain and the target domain is usually unbalanced; therefore, the present invention uses the balance factor α 2 ∈[0,1] to balance the energy of (9) and (10). So combining (9) and (10) is:
[0067]
[0068] Subspace learning aims to optimize (8) and (11) simultaneously, that is, the present invention seeks to minimize the distribution difference (marginal distribution and conditional distribution) between the source domain and the target domain while maintaining as much discriminative information as possible; therefore, it can be expressed as:
[0069]
[0070] Obviously, the minimization problem in formula (19) has a non-unique solution; therefore, an equality constraint is imposed to transform (12) into:
[0071]
[0072] In formula (13) represents the unit matrix;
[0073] In order to solve the minimization problem with equality constraints in (13), the Lagrangian function is introduced:
[0074]
[0075] In formula (14) represents the Lagrange multiplier;
[0076] Set the partial derivative of L with respect to P to 0, that is,
[0077]
[0078] Therefore, the projection matrix P can be solved by eigenvalue decomposition; for ease of observation, (15) is rewritten as:
[0079] AP=PΦ (16)
[0080] In formula (16)
[0081] From equation (16), we can see that P is a matrix composed of the eigenvectors of A. Furthermore, since equation (12) is a minimization problem, the optimal projection matrix P is * It is a matrix composed of the eigenvectors corresponding to the first d smallest eigenvalues, expressed as:
[0082] P * =[P 1 ,P2 ,...,P d ] (17).
[0083] 2. Traditional Extreme Learning Machine (ELM)
[0084] In the multi-classification task, assuming [x 1 ,x 2 ,…,x N ] represents the training set (source domain), where N represents the number of samples in the training set, represents the input vector, represents the corresponding label, where n and m are the dimensions of the input layer and output layer of ELM respectively. In the traditional ELM, the hidden layer has L neurons, and its activation function is h(·), so the output of the hidden layer is represents the output weight of the hidden layer;
[0085] The training process of ELM consists of two steps:
[0086] In the first step, ELM constructs a feature map from the input layer to the hidden layer and randomly generates parameters in the hidden layer;
[0087] The second step is to calculate the output weight β. The calculation formula is as follows:
[0088]
[0089] In formula (18) Represents x i The error, C represents the penalty factor, and N represents the number of samples in the training set;
[0090] By substituting the constraint term in equation (18) into the objective function, (18) can be rewritten as:
[0091]
[0092] In formula (19)
[0093] The regularized least squares problem in formula (19) can be solved by replacing L ELM The gradient relative to β is set to 0 to solve, that is,
[0094]
[0095] In formula (20) and Represents the unit matrix.
[0096] 3. Domain Adaptive Extreme Learning Machine (DAELM)
[0097] DAELM is a domain adaptation algorithm based on traditional ELM. It improves the cross-domain migration capability of the model without losing computational efficiency. The core of DAELM is to label some samples in the target domain and use them as the regularization term of the algorithm. The expression of DAELM is:
[0098]
[0099] In formula (21), S represents the source domain and T represents the target domain. Indicates that it comes from the source domain. represents the prediction error of the target domain sample, C S and C T represents the penalty factor, N S Represents the number of source domain samples, N T represents the number of migration samples, where the migration samples represent the labeled samples selected from the target domain, and represents the output of the hidden layer, represents the source domain, Represents the label of the migration sample. Finally, the output weight β is as follows:
[0100]
[0101] In formula (22), represents the source domain, represents the hidden layer output weight corresponding to the target domain, I S and I T Unit matrix, T S represents the source domain, T T Represents the label matrix corresponding to the migration samples.
[0102] 4. Cross-domain adaptive extreme learning machine (CAELM) based on guidance samples
[0103] Further observing formula (20) and formula (22), when the migration sample set is empty, DAELM is simplified to the traditional ELM, that is, formula (22) is simplified to the solution of formula (20). In order to improve the cross-domain transfer capability at a lower label cost, the present invention proposes a method combining traditional ELM and DAELM, namely, a domain adaptive extreme learning machine (CAELM) based on guidance samples; if the guidance sample set is empty, the proposed CAELM uses the traditional ELM as a classifier; that is, the ELM is trained using source domain samples, and then the labels of the target domain samples are predicted; if the guidance sample set is non-empty, the proposed CAELM uses DAELM as a classifier; that is, the CAELM is trained using source domain samples and guidance samples, and then the labels of the target domain samples are predicted; therefore, it is expressed in CAELM as:
[0104]
[0105] Essentially, CAELM uses guidance samples to replace migration samples, which not only achieves drift compensation at the decision level but also reduces label costs. The method for establishing a guidance sample set is as follows:
[0106] Step 1: Calculate the center of the source domain belonging to class c in the subspace. The calculation formula is as follows
[0107]
[0108] In formula (24), μ represents the center of the sample in the subspace where the source domain belongs to class c, S represents the source domain, represents the number of samples belonging to category c in the source domain, represents the i-th sample of the source domain in the subspace;
[0109] Step 2: Use CAELM to predict the labels of samples in the target domain. The predicted labels are called soft labels of target domain samples. If Exceeds a preset threshold The sample set that belongs to class c in the subspace comes from the target domain with soft labels Selected to enter step 3, otherwise, return to step 1, and c = c + 1;
[0110] Step 3: Calculation Each sample in The Euclidean distance of The sample with the smallest distance is used as the guidance sample, and then return to step 1, and guide c = C;
[0111] If the number of samples belonging to class c is small, it cannot be guaranteed that these soft labels are consistent with the true results; therefore, step 2 uses th 1 Remove unreliable samples; in step three, use th 2 to select guidance samples because they are closer to the source domain in the subspace.
[0112] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. A gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine, characterized in that: The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine is implemented by the following steps: Step 1: Collect source domain and target domain datasets; Step 2: Establish feature set; Step 3: Subspace learning; Step 4: Build and train the gas classification model; in the subspace, establish a guidance sample set, and input the guidance sample set and source domain data into CAELM for training; CAELM refers to a cross-domain adaptive extreme learning machine based on guidance samples; In the first iteration, CAELM selects ELM as the classifier, only source domain samples participate in the training of the classifier, and the classification results of the classifier on the target domain are used as soft labels; These soft labels are used for subspace learning and gas classification model training in the next iteration; In the subsequent iterations, using the soft labels obtained in the previous iteration, CAELM selects DAELM as the classifier; DAELM refers to the domain adaptive extreme learning machine; Step 5: Predict the label of the target domain data.
2. The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine according to claim 1 is characterized in that: In step 1, the same gas sensor is used to collect raw data of gas in different time periods, or different gas sensors with the same configuration are used to collect raw data of gas at the same time, to establish a source domain dataset S and a target domain dataset T.
3. The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine according to claim 1 is characterized in that: In step 2, multiple features are extracted based on the collected raw data, and a feature set is established, namely: In formula (1) and formula (2), X S represents the source domain feature set, X T represents the target domain feature set, D represents the feature set dimension, and N S Represents the number of samples in the source domain, N T represents the number of samples in the target domain, represents the i-th sample in the source domain, i = 1, 2, ..., N S , represents the jth sample in the target domain, j = 1, 2, ..., N T .
4. The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine according to claim 1, characterized in that: In step 3, the projection matrix P is calculated * , and the source domain X S and the target domain X T Projected to the common subspace, the source domain of the projected subspace is Y S , the target domain is Y T .
5. The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine according to claim 4 is characterized in that: Projection Matrix P * The calculation formula is: P * =[P1,P2,...,P d ](3), In formula (3), P d Represents the eigenvector corresponding to the dth smallest eigenvalue, d = 1, 2, 3...
6. The gas sensor drift compensation method based on subspace learning and cross-domain adaptive extreme learning machine according to claim 1, characterized in that: In step 5, after the subspace and gas classification models have been trained for multiple iterations and meet the requirements, the trained models are used to classify the target domain samples to obtain the final classification results.