A method for preprocessing sensor data to provide a training feature for training a machine learning model

Abstract

The invention relates to a method (100) for preprocessing sensor data to provide a training feature for training a machine learning model, comprising the following steps: - Providing (101) the sensor data, the sensor data resulting from an acquisition of at least one sensor (1), - Extracting (102) features from the sensor data using a wavelet decomposition of the sensor data, the extracted features being invariant to a die-to-die variation of the at least one sensor (1), - Providing (103) the training feature based on the extracted features. Furthermore, the invention relates to a computer program, an apparatus, and a storage medium for this purpose.

Description

[0001] Description

[0002] Title

[0003] A method for preprocessing sensor data to feature for a machine learning model

[0004] The invention relates to a method for preprocessing sensor data to provide a training feature for training a machine learning model. Furthermore, the invention relates to a computer program, an apparatus, and a storage medium for this purpose.

[0005] State of the art

[0006] Gas sensors detecting a wide range of gases, especially Volatile Organic Compounds (VOC), Hydrogen (H2), Carbon Monoxide (CO) or Hydrocarbons (CxHy) may use the technology of metal oxides (MOX) as gas sensing material on a chip, where the material can be actively heated to defined temperatures for certain operation times. The electrical resistance changes depending on the oxygen content on its surface when semiconductor particles (typically tin dioxide) are heated in air at high temperature. The electrical characteristics (e.g., conductivity or resistance) of the sensing material are read out by an applicationspecific integrated circuit (ASIC) and then processed by software and algorithms. In addition to that a temperature, a humidity and a pressure sensor can optionally be considered in the system to measure the environmental data in which gas sensor is operating.

[0007] One of the main challenges of MOX gas sensor technology is die-to-die variation. In MOX sensors, additives known as dopants are employed to enhance sensing properties. These dopants can significantly affect the sensing behaviour of gas sensors. Die-to-die variation is caused mainly due to the accuracy of the doping process. Any variations in the doping process can lead to differences in sensor characteristics like offset, sensitivity, response time and recovery time of sensors. Consequently, two sensors from the same production lot placed in the same environment very likely have different sensor signals. This causes a major problem for the development of algorithms and if not considered can heavily impact the overall accuracy of the gas sensor output.

[0008] Gas sensors have a wide range of applications including Machine Learning or Artificial Intelligence (Al) enabled solutions such as classification (classifying different gas compositions with different combinations and concentrations of gases) or quantification (classifying a gas and estimate a value of its concentration) of gas composition present in the environment. However, one of the major challenges for such solutions is die-to-die variation and consequent signal variation of sensors, which reduces the accuracy and prevents the generalizability of Al based machine learning models. This is even worse in the case of embedded software where machine learning models need to be small and thus inherently lack generalization capabilities.

[0009] There are two major directions in the state of the art to reduce die-to-die variation which are hardware improvements or software improvements. Hardware based solutions include improving the accuracy of ink injection during the doping process. This process is known to be expensive. Software-based solution vary depending upon use case. Some commonly used methods are available for reducing die-to-die deviation, but they have some major disadvantages which are detailed below. One method is modelling, where an algorithm or model is developed, capable of intrinsically considering variations. This requires expert knowledge and plenty of costly development time or a large Al model trained on plenty of data which is not suitable for embedded applications. Another possibility are Baseline Processing Methods. These methods compensate for the response of each sensor by tracking the sensor sensitivity over time based on a baseline estimation. However, for Al based applications, such compensation is not practical and due to the complexity of the causes of drifting, these methods can only reduce some of the side effects caused by drift. Another possibility is calibration, where the algorithms are calibrated for each individual sensor before using the sensor in a use case. There are different calibration techniques such as cross sensor calibration or reference gas calibration, which require a specific calibration setup. These calibration techniques are time consuming and expensive for either sensor manufacturer or user.

[0010] Disclosure of the invention

[0011] According to aspects of the invention a method with the features of claim 1 , a computer program with the features of claim 9, a data processing apparatus with the features of claim 10 as well as a computer-readable storage medium with the features of claim 11 are provided. Further features and details of the invention are disclosed in the respective dependent claims, the description and the drawings. Features and details described in the context to the inventive method also correspond to the inventive computer program, the inventive data processing apparatus as well as the inventive computer-readable storage medium, and vice versa in each case.

[0012] According to an aspect of the invention a method for preprocessing sensor data to provide a training feature for training a machine learning model is provided, the method comprising the following steps, wherein the steps may be performed repeatedly and / or in a specific order. The machine learning model may be a neural network.

[0013] In a first possible step, the sensor data may be provided, the sensor data resulting from an acquisition of at least one sensor. The at least one sensor may be a gas sensor and the sensor data may represent a resistance of the gas sensor. However, the method according to the invention is not limited to gas sensors and may also be used for other use cases involving variations in time series signals from different sensor modalities, e.g., accelerometers, gyroscopes, magnetometers as well as sensors for measuring a pressure, temperature and / or humidity.

[0014] In a next possible step, features are extracted from the sensor data using a wavelet decomposition of the sensor data, the extracted features being invariant to a die-to-die variation of the at least one sensor. The wavelet decomposition may be a Discrete Wavelet Transform (DWT), which particularly is a mathematical technique used in signal processing to analyse various frequencies and their locations in a given signal. The Discrete Wavelet Transform (DWT) may allow for the analysis of both frequency and location simultaneously, which may make it particularly useful for signals where sudden changes occur. Another possible decomposition may be a Fourier Transform.

[0015] The term “die-to-die variation” in the context of the invention may refer to a variability in performance or characteristics seen between different sensors even when they are produced from the same semiconductor wafer using the same manufacturing processes. These variations may arise due to slight differences in the fabrication process across the wafer, such as inconsistencies in material deposition, lithography, doping levels, and etching. The die-to-die variations can affect critical parameters such as sensitivity, noise levels, and response time. The term may also include variations within the same sensor over time, for example due to aging.

[0016] In a next possible step, the training feature is provided based on the extracted features.

[0017] By means of the method according to the invention, an effect of the die-to-die variations in sensor signal values may thus be compensated and consequently a performance and robustness of the machine learning model may be improved when using the provided training feature in a training of the machine learning model.

[0018] In an example, the method further comprises the following step:

[0019] Performing a data analysis of the sensor data by testing the at least one sensor to determine effects of the die-to-die variation of the at least one sensor.

[0020] The data analysis may comprise measurements with different input data and may be performed at different temperature levels to create a temperature profile. The effects of the die-to-die variation may be given between two sensors or within the same sensor, for example due to aging effects.

[0021] In another example, the Extracting of the features comprises the following steps: Passing the sensor data through a low pass filter and sampling down an output of the low pass filter to estimate approximate coefficients of the sensor data,

[0022] Passing the sensor data through a high pass filter and sampling down an output of the high pass filter to estimate detailed coefficients of the sensor data,

[0023] Concatenating the approximate coefficients and the detailed coefficients to provide the training feature.

[0024] A low pass filter may be a type of electronic filter designed to allow signals with frequencies lower than a specific cutoff frequency to pass through with little to no attenuation, while attenuating signals with frequencies higher than the cutoff frequency. A high pass filter may perform the opposite function to the low pass filter. It may allow signals with frequencies higher than a certain cutoff frequency to pass through with minimal loss, while attenuating signals with frequencies below the cutoff frequency. The respective outputs may be down sampled by 2, which means only every second sample may be kept. This may reduce the dataset size. The approximate coefficients and the detailed coefficients may be concatenated by joining the end of the approximate coefficients with the beginning of the detailed coefficients, resulting in a continuous signal. This may be done by adding the two together, or by using a concatenation function in a programming language.

[0025] In another example, the Extracting further comprises the following step: Defining a specific wavelet function that determines the shape of the low pass filter and / or the high pass filter.

[0026] The wavelet function may be chosen and defined based on properties of the sensor data and specific requirements of a present use case. Common wavelet functions include Daubechies, Haar, and Morlet.

[0027] In another example, the wavelet decomposition is performed as a multilevel decomposition, wherein the wavelet decomposition is applied iteratively to a respective result of a previous application of the wavelet decomposition. This may advantageously allow for a hierarchical representation of the sensor data, capturing both coarse and fine details across multiple resolution levels. For example, four levels of decomposition may be used. A number of levels of decomposition may be chosen depending on a present sensor and sensor data.

[0028] In another example, the method further comprises the following step:

[0029] Utilising the provided training feature in a training of the machine learning model for classifying the sensor data.

[0030] The training of the machine learning model may be a supervised training, wherein the provided training feature may be used as training data. Supervised training of a machine learning model may involve using a dataset that contains input-output pairs, where the outputs are known labels or values associated with the inputs. The training may begin with splitting the dataset into a training dataset and a test dataset. The training dataset may be used to teach the machine learning model how to predict the output from the input data. During training, the model may make predictions based on the inputs, and adjustments may be made to the machine learning model’s parameters, such as weights in neural networks, through a process called optimization, typically using algorithms like gradient descent. The objective may be to minimize the difference between the predicted outputs and the actual outputs in the training data, a process quantified by a loss function. After the machine learning model is trained, it may be evaluated using the test dataset, which was not seen by the machine learning model during training, to assess its performance on new, unseen data.

[0031] In another example, the method further comprises the following steps:

[0032] Providing new sensor data, the new sensor data resulting from an acquisition of the at least one sensor and / or at least one additional sensor, Extracting features from the new sensor data using the wavelet decomposition of the new sensor data, the extracted features being invariant to the die-to-die variation of the at least one sensor and / or the at least one additional sensor,

[0033] Classifying the new sensor data by means of the trained machine learning model in an inference phase based on the extracted features.

[0034] The classifying may involve determining a gases’ class and / or concentration, if the at least one sensor and / or the at least one additional sensor is a gas sensor. The inference phase of the machine learning model may refer to the process of using the trained machine learning model to make predictions or decisions based on new, unseen data, i.e. the new sensor data. During inference, the machine learning model’s learned parameters may be used to compute outputs for new inputs, without any further adjustments to its parameters.

[0035] In another aspect of the invention, a computer program may be provided, in particular a computer program product, comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to the invention. Thus, the computer program according to the invention can have the same advantages as have been described in detail with reference to a method according to the invention.

[0036] In another aspect of the invention, an apparatus for data processing may be provided, which is configured to execute the method according to the invention. As the apparatus, for example, a computer can be provided which executes the computer program according to the invention. The computer may include at least one processor that can be used to execute the computer program. Also, a nonvolatile data memory may be provided in which the computer program may be stored and from which the computer program may be read by the processor for being carried out.

[0037] According to another aspect of the invention a computer-readable storage medium may be provided which comprises the computer program according to the invention and / or instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to the invention. The storage medium may be formed as a data storage device such as a hard disk and / or a non-volatile memory and / or a memory card and / or a solid state drive. The storage medium may, for example, be integrated into the computer.

[0038] Furthermore, the method according to the invention may be implemented as a computer-implemented method. Alternatively or additionally, at least one of the disclosed method steps may be computer-implemented and / or automated.

[0039] Further advantages, features and details of the invention will be apparent from the following description, in which embodiments of the invention are described in detail with reference to the drawings. In this context, the features mentioned in the claims and in the description may each be essential to the invention individually or in any combination. Showing:

[0040] Fig. 1 : A method, a sensor and an additional sensor, a computer program, a storage medium and apparatus according to embodiments of the invention,

[0041] Fig. 2: A flowchart of training the machine learning model using the training feature according to embodiments of the invention,

[0042] Fig. 3: A flowchart of estimating a gas composition / intensity using the trained machine learning model according to embodiments of the invention.

[0043] Fig. 1 shows a method 100, a sensor 1 and an additional sensor 2, a computer program 20, a storage medium 15 and apparatus 10 according to embodiments of the invention.

[0044] Fig. 1 particularly shows an embodiment of a method 100 for preprocessing sensor data to provide a training feature for training a machine learning model. In a first step 101 , the sensor data is provided, the sensor data resulting from an acquisition of at least one sensor 1. In a second step 102, features are extracted from the sensor data using a wavelet decomposition of the sensor data, the extracted features being invariant to a die-to-die variation of the at least one sensor 1. In a third step 103, the training feature is provided based on the extracted features.

[0045] According to embodiments of the invention, a method using Discrete Wavelet Transform for feature extraction as a preprocessing step is provided. The method according to the invention may improve negative effects of die-to-die variations.

[0046] According to embodiments of the invention, a (software) method is described, that solves the selection of appropriate features for MOX gas sensors based on DWT features. These extracted features may be selected in a way to be invariant to die-to-die variations. Passing them as input to machine learning models or any other signal processing models may reduce the effect of die-to-die variations and thus improve overall performance and robustness. The method according to the invention may significantly improve the performance of various gas-sensor based use-case solutions. Use cases such as quantification and classification of gases may heavily benefit from this improvement. This approach may not be limited to gas sensor use cases, and may also be used for a wide range of other use cases involving variations in time series signals from different sensor modalities (e.g., accelerometer, gyroscope, magnetometer, pressure, temperature, humidity etc.) involving Al, Machine Learning (ML) or classical signal processing applications.

[0047] Advantages that may arise by applying the method according to the invention may include the following. The die-to-die variation may be reduced and consequently the performance and robustness of the underlying machine learning model may allow to use large set of sensors in different use cases. An adaptive adjustment of compensation based on the sensed gas resistance reading may be provided. Therefore, there may be no restriction for porting the compensation model to another sensor. An ease of use for the sensor user may be provided as there may be no need to additionally compensate die-to-die deviations for developing robust use case solutions. Further, there may be no need for hardware compensation.

[0048] An application of the present invention into a use case for a gas sensor may be divided into two main steps. In the first step, DWT feature extraction may be applied on the gas sensor signal, i.e. the sensor data. To set up the DWT feature extraction, first a lab data analysis of the sensor data may be performed to understand the effects of die-to-die variation onto the gas sensor resistance signals. Based on the gained insights the details of DWT signal decomposition may be investigated. Here following knowledge about DWT and gas sensor may be considered: DWT particularly divides a signal into high pass filter signal or detailed coefficients and low pass filter signal or approximate coefficients. Low pass filter signal or approximate coefficients may represent the smoothened form of the signal. This may capture broader, more general features of a signal and may denoise the signal and give out a smoothened signal. This may also contain information about the amplitude changes in the signal. High pass signal or detailed coefficients may represent frequency domain representation of the signal. This may be a part of the signal which is independent of the offset drift and sensitivity drift due to die-to-die variations and may capture smaller details of the signal. Low level detailed features like variation with respect to heater steps, changes in gas resistance response, response time, recovery time, transition state and specimens may be captured in this feature. In this step, once approximate and detailed coefficients are available, these features may be appended or concatenated together to form the training feature according to the invention.

[0049] In the second step, once the die-to-die variations in the gas resistance value are eliminated by the DWT feature extraction, the specific use case algorithm may take the die-to-die invariant features and process them to a degree such that a user can interpret them. The use case algorithm may be a supervised training of a machine learning model. Such a use case algorithm may also be a signal processing model that may also heavily improve in performance and robustness.

[0050] The method according to embodiments of the invention may be performed once during initialization of the at least one sensor for a given use case, or may be continuously performed during an operation of the at least one sensor. The training of the machine learning model may also be performed once and the trained machine learning model may then be used for several various use cases.

[0051] An example of such a solution is training a machine learning model for a gas classification or regression as shown in Fig. 2. Machine learning techniques may be used to train a classification or regression model, based on available labeled data 3. The DWT feature extraction may be used as a preprocessing step and the feature extracted gas resistance value 4, i.e. the provided training feature, along with optional temperature 5, humidity 6 and pressure values 7, may be provided as an input to the method estimating the machine learning model 8. An output may be given as the optimized machine learning model 9 for classification or regression.

[0052] Fig. 3 displays the use of such a die-to-die robust machine learning model 8 for a prediction of a gas composition 12 in a sensor environment. The machine learning model 9 trained as shown in Fig. 2 may be used as an input to a prediction function 11 which provides the probability of occurrence of gases or its concentration 12, based on the DWT feature extraction. As further inputs to the prediction function 11 , the feature extracted gas resistance value 4 along with optional temperature 5, humidity 6 and pressure values 7 may be provided.

[0053] Without using feature extraction as preprocessing step, by directly using gas resistance values as input to the ML model, a performance of a machine learning model may decrease drastically when testing on new sensors. With the method according to the invention, the performance may be improved significantly and effects of die-to-die variations may be reduced drastically.

[0054] In the following, a brief insight into the mathematics and considerations of the method according to embodiments of the invention is provided.

[0055] The Approximate and Detailed DWT coefficients of the sensor data are particularly estimated by passing the sensor data through a respective single or a series of low pass and high pass filters. First the sensor data may be passed through the low pass filter resulting in where x[k] may represent the sensor data at a given time and n, g may represent the low pass filter. y_low may represent an output from the low pass filter, which may then be down sampled to provide the approximate coefficients.

[0056] The sensor data may also be passed through the high pass filter, particularly at the same time: where, x[k] may represent the sensor data at a given time and n, h may represent the high pass filter. y_high may represent an output from the high pass filter, which may then be down sampled to provide the detailed coefficients. The high pass filter and low pass filter may be derived from wavelets. The choice of the specific wavelet function (e.g., Haar, Daubechies, etc.) may determine the shape of the filters (h, g) and, consequently, the characteristics of the extracted coefficients. The wavelet may change depending on the sensor variant, data, and other such conditions.

[0057] The above equations particularly represent a single level decomposition. Multilevel decomposition may involve applying the DWT iteratively to the approximation coefficients obtained at each level of the decomposition. This process may allow for a hierarchical representation of the input signal, capturing both coarse and fine details across multiple resolution levels. For example, four levels of decomposition may be used. However, this level may change depending on a given sensor and sensor data.

[0058] These wavelets and levels may depend only upon the gas sensor’s sensing element. So, once the sensor device design is finalized, these compensation wavelet and level may be suitable for any gas sensor having the same sensing element. Once approximate coefficients and detailed coefficients are approximated, these may be concatenated to form the training feature invariant to die-to-die variation.

[0059] The above explanation of the embodiments describes the present invention in the context of examples. Of course, individual features of the embodiments can be freely combined with each other, provided that this is technically reasonable, without leaving the scope of the present invention.

Claims

Claims1 . A method (100) for preprocessing sensor data to provide a training feature for training a machine learning model, comprising the following steps:Providing (101) the sensor data, the sensor data resulting from an acquisition of at least one sensor (1),Extracting (102) features from the sensor data using a wavelet decomposition of the sensor data, the extracted features being invariant to a die-to-die variation of the at least one sensor (1), Providing (103) the training feature based on the extracted features.

2. The method (100) of claim 1 , characterized in that the method (100) further comprises the following step:Performing a data analysis of the sensor data by testing the at least one sensor (1) to determine effects of the die-to-die variation of the at least one sensor (1).

3. The method (100) of any one of the preceding claims, characterized in that the Extracting (102) of the features comprises the following steps:Passing the sensor data through a low pass filter and sampling down an output of the low pass filter to estimate approximate coefficients of the sensor data,Passing the sensor data through a high pass filter and sampling down an output of the high pass filter to estimate detailed coefficients of the sensor data,Concatenating the approximate coefficients and the detailed coefficients to provide the training feature.

4. The method (100) of claim 3, characterized in that the Extracting further comprises the following step:Defining a specific wavelet function that determines the shape of the low pass filter and / or the high pass filter.

5. The method (100) of any one of the preceding claims, characterized in that the wavelet decomposition is performed as a multilevel decomposition, wherein the wavelet decomposition is applied iteratively to a respective result of a previous application of the wavelet decomposition.

6. The method (100) of any one of the preceding claims, characterized in that the method (100) further comprises the following step:Utilising the provided training feature in a training of the machine learning model for classifying the sensor data.

7. The method (100) of claim 6, characterized in that the method (100) further comprises the following steps:Providing new sensor data, the new sensor data resulting from an acquisition of the at least one sensor (1) and / or at least one additional sensor (2),Extracting features from the new sensor data using the wavelet decomposition of the new sensor data, the extracted features being invariant to the die-to-die variation of the at least one sensor (1) and / or the at least one additional sensor (2),Classifying the new sensor data by means of the trained machine learning model in an inference phase based on the extracted features.

8. The method (100) of any one of the preceding claims, characterized in that the at least one sensor (1) is a gas sensor and the sensor data represents a resistance of the gas sensor.

9. A computer program (20), comprising instructions which, when the computer program (20) is executed by a computer (10), cause the computer (10) to carry out the method (100) of any one of the preceding claims.

10. A data processing apparatus (10), comprising means for carrying out the method (100) of any one of claims 1 to 8.

11. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer (10) to carry out the steps of the method (100) of any one of claims 1 to 8.

Images