METHOD FOR DEVELOPING A CLASSIFICATION MODEL
A computer-implemented method for developing classification models optimizes acquisition parameters by simulating time series signals to evaluate performance, reducing costs and time through efficient parameter selection.
Patent Information
- Application Number
- FR2023006062
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-14
- Publication Date
- 2024-12-20
AI Technical Summary
Existing methods for developing classification models for time series signals require time-consuming and expensive trial-and-error searches for optimal acquisition parameters, such as sampling frequency and data quantity, which do not guarantee optimal performance.
A computer-implemented method that involves obtaining initial time series signals, simulating groups of signals with varied parameters, developing test classification models, evaluating their performance, and selecting optimal parameters for final model development based on simulated data analysis.
This method simplifies the determination of acquisition parameters, reducing costs and time by avoiding multiple actual data acquisitions and ensuring optimal performance criteria are met.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: METHOD FOR DEVELOPING A CLASSIFICATION MODEL
[0001] Embodiments and implementations relate to the classification of time series signals.
[0002] Computers can use classification models to classify data into categories or groups based on certain characteristics. There are several types of classification models. Each type of classification model has its own advantages and disadvantages depending on the type of data to be classified.
[0003] Classification models include decision trees, neural networks, neighbor-based classification models such as k-nearest neighbors (KNN), and support vector machines (SVM).
[0004] Classification models can be used for time series signals. Time series signals are data that evolve over time and contain a measurement or observation at each acquisition time.
[0005] A time series signal may, for example, correspond to an audio signal, a video signal, a vibration signal or a physiological signal.
[0006] In particular, the time series signals may be acquired according to different acquisition parameters. For example, the time series signals may be acquired according to a sampling frequency and a data quantity defined by a user.
[0007] The sampling frequency corresponds to a number of data samples of the time series signal per unit of time. The higher the sampling frequency for the same amount of data, the higher the resolution of the time series signals in the time domain, this resolution corresponding to the value of the sampling frequency divided by the amount of data. Conversely, a lower sampling frequency for the same amount of data can distort the time series signals.
[0008] The amount of data corresponds to the number of data of the time series signal acquired during the acquisition. This amount of data is defined by indicating a buffer size of the acquisition device to be used for the acquisition of the time series signals. The greater the amount of data for the same sampling frequency, the more it is possible to observe finer details of the time series signal in the frequency domain.
[0009] Furthermore, the amount of data should be defined as a function of the sampling frequency. Indeed, if the sampling frequency is too high and the amount of data is too low, then the acquisition duration may be too short to acquire enough information in the time series signal.
[0010] At a given sampling frequency, the higher the amount of data, the longer the acquisition time. This allows more behaviors to be observed in the acquired time series signal, but it also results in greater power consumption, a longer waiting time to acquire the time series signal, and greater memory occupation to store the acquired data.
[0011] A classification model is trained from acquired time series signals. The sampling frequency and the amount of data have an impact on the trained classification model.
[0012] In particular, the classification model exhibits performance that may vary depending on the sampling frequency and the amount of data used for acquiring the time series signals provided to train this classification model.
[0013] The performance of a classification model is defined in particular with regard to an accuracy of the classification model, an acquisition duration of the time series signal to be used for the classification model (which impacts the energy consumption of the acquisition and the waiting time to acquire the time series signal) as well as the quantity of data defined for the acquisition of the time series signal (which impacts the occupation of the memory for the storage of the acquired data).
[0014] Thus, it is advantageous to define the sampling frequency and the quantity of data making it possible to obtain a classification model presenting optimal performance with regard to the user's needs.
[0015] Conventionally, the user performs several acquisitions at different sampling frequencies and with different amounts of data in order to test which combination of sampling frequency and amount of data makes it possible to obtain a classification model with optimal performance.
[0016] The user therefore carries out a search by trial and error to determine the combination of sampling frequency and quantity of data allowing to obtain a classification model with optimal performance.
[0017] This trial-and-error search requires performing several time series signal acquisitions. These time series signal acquisitions have the disadvantages of being time-consuming and costly. In addition, trial-and-error search does not ensure obtaining a classification model with performance optimal.
[0018] Thus, there is a need to propose a solution to facilitate the search for acquisition parameters for the acquisition of time series signals used to train a classification model.
[0019] According to one aspect, there is provided a computer-implemented method of developing a classification model, called the final classification model, the method comprising: - obtaining at least one group of initial time series signals, said initial time series signals being associated with at least one initial acquisition parameter, - a development of at least one group of simulated time series signals from said at least one group of initial time series signals, said simulated time series signals of each group being associated with at least one simulated acquisition parameter different from said at least one initial acquisition parameter, - a development of different test classification models, each test classification model being developed from said at least one group of initial time series signals or said at least one group of simulated time series signals, each test classification model being associated with said at least one acquisition parameter corresponding to said at least one initial or simulated acquisition parameter associated with the initial or simulated time series signals used to develop this test classification model, - a performance evaluation of each test classification model, said performance being associated with said at least one acquisition parameter of this test classification model, then - obtaining at least one group of final time series signals associated with at least one final acquisition parameter selected from an analysis of said evaluated performances of each test classification model, then - a development of said final classification model from said at least one group of final time series signals.
[0020] Such a method makes it possible to simplify the determination of the acquisition parameters to be applied for the acquisition of the final time series signals used for the development of the final classification model.
[0021] Indeed, the acquisition of initial time series signals is carried out by an acquisition device, such as a sensor, according to at least one initial acquisition parameter. The performances of the test classification models are then evaluated from simulated initial time series signals and not acquired by the acquisition device.
[0022] The acquisition parameters to be applied for the acquisition of serial signals The final time series are therefore determined from an analysis of the performances related to the acquisition parameters associated with the simulated time series signals.
[0023] In particular, the acquisition parameters to be applied for the acquisition of the final time series signals can be chosen by analyzing which acquisition parameters allow optimal performance to be obtained with regard to user-defined performance criteria.
[0024] This avoids performing an acquisition of time series signals with different acquisition parameters to determine which acquisition parameters allow developing a classification model having optimal performance with regard to user-defined performance criteria.
[0025] By simplifying the search for said at least one final acquisition parameter, such a method thus makes it possible to reduce the overall cost of developing the final classification model because the latter can be obtained more simply and more quickly.
[0026] In an advantageous embodiment, said at least one initial acquisition parameter comprises a combination of an initial sampling frequency and an initial data quantity of the initial time series signals.
[0027] Advantageously, said initial sampling frequency corresponds to a maximum sampling frequency permitted by an acquisition device used to acquire said initial time series signals.
[0028] Preferably, said initial data quantity corresponds to a maximum data quantity permitted by an acquisition device used to acquire said initial time series signals.
[0029] In an advantageous embodiment, said at least one simulated acquisition parameter of said simulated time series signals of each group comprises a simulated sampling frequency less than or equal to said initial sampling frequency and a simulated data quantity less than or equal to said initial data quantity.
[0030] Advantageously, each group of initial time series signals is associated with a class indicated during said obtaining of the initial time series signals.
[0031] In an advantageous embodiment, each group of simulated time series signals is associated with the class indicated for the group of initial time series signals from which this group of simulated time series signals is developed.
[0032] Preferably, the method further comprises extracting feature values from the initial and simulated time series signals, each test classification model being developed from an analysis of the extracted feature values and the class associated with each group of time series signals. initial or simulated used to develop this test classification model.
[0033] In an advantageous embodiment, the method further comprises extracting feature values from the final time series signals, the final classification model being developed from an analysis of the extracted feature values and the class associated with each group of final time series signals.
[0034] Preferably, the method further comprises an indication of the performance of each test classification model in relation to said at least one acquisition parameter associated with this test classification model.
[0035] Advantageously, said indication of the performance of each classification model comprises a display on a screen of a performance graph comprising said performance of each classification model according to said at least one associated acquisition parameter.
[0036] In an advantageous embodiment, said evaluated performances of each test classification model comprise an accuracy, an acquisition time of a time series signal and an amount of data acquired for this time series signal.
[0037] Advantageously, the method further comprises developing a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the final classification model.
[0038] According to another aspect, there is provided a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method of developing a classification model as described above.
[0039] According to another aspect, there is provided a computer system comprising: - a memory comprising a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method of developing a classification model as described above, - a processing unit configured to execute said computer program product.
[0040] Other advantages and characteristics of the invention will appear on examining the detailed description of embodiments, which are in no way limiting, and the appended drawings in which:
[0041] [Fig.l]
[0042] [Fig.2]
[0043] [Fig.3] illustrate embodiments and implementations of the invention.
[0044] [Fig.l] illustrates an embodiment of a computer system SYS configured to implement a method for developing a classification model.
[0045] The computer system SYS comprises a processing unit UT and a memory MEM. Such a computer system SYS can be a personal computer or a server for example.
[0046] The memory MEM comprises a computer program PRG comprising instructions which, when the program PRG is executed by the processing unit UT of the computer system SYS, cause the latter to implement the method for developing a classification model MDL, as described below.
[0047] The MEM memory is also configured to store time series signals. These time series signals serve as training data for developing the MDL classification model.
[0048] The time series signals can be provided by a user of the SYS computer system wishing to define a classification model from these time series signals.
[0049] Time series signals may be acquired by sensors and then provided to the SYS computer system. A time series signal may, for example, correspond to an audio signal, a vibration signal, or a physiological signal. The acquired time series signals comprise a set of data acquired over time. The time series signals may comprise several lines of data.
[0050] Time series signals are associated with different classes. A class is a group of data that have similar characteristics and are grouped together based on those characteristics.
[0051] Thus, the time series signals are divided into several groups of time series signals, each group of time series signals being associated with a class indicated by the user.
[0052] For example, one group of time series signals may be associated with a class corresponding to signals corresponding to normal operation, and another group of time series signals may be associated with a class corresponding to anomaly signals.
[0053] The processing unit UT is configured to develop an MDL classification model by implementing said development method. In particular, the processing unit UT is configured to develop an MDL classification model by executing the computer program PRG by taking time series signals as input.
[0054] The time series signals are acquired by an acquisition device (not shown). This acquisition device is configured to perform an acquisition according to at least one user-defined acquisition parameter, such as a sampling rate and data amount of the time series signals.
[0055] The sampling frequency corresponds to a number of data samples of the time series signal per unit of time. The higher the sampling frequency for the same amount of data, the higher the resolution of the time series signals in the time domain, this resolution corresponding to the value of the sampling frequency divided by the amount of data. Conversely, a lower sampling frequency for the same amount of data can distort the time series signals.
[0056] The amount of data corresponds to the number of data of the time series signal acquired during the acquisition. This amount of data is defined by indicating a buffer size of the acquisition device to be used for the acquisition of the time series signals. The greater the amount of data for the same sampling frequency, the more it is possible to observe finer details of the time series signal in the frequency domain.
[0057] Furthermore, the amount of data should be defined as a function of the sampling frequency. Indeed, if the sampling frequency is too high and the amount of data is too low, then the acquisition duration may be too short to acquire sufficient information in the time series signal.
[0058] At a given sampling frequency, the higher the amount of data, the longer the acquisition time. This makes it possible to observe more behaviors in the acquired time series signal, but it also results in greater power consumption, a longer waiting time to acquire the time series signal, and greater memory occupation to store the acquired data.
[0059] The acquisition device allows the user to choose a sampling frequency less than or equal to a maximum sampling frequency permitted by this acquisition device. The acquisition device allows the user to choose a quantity of data less than or equal to a maximum quantity of data permitted by this acquisition device.
[0060] [Fig.2] illustrates an embodiment of a computer-implemented method for developing a time series signal classification model.
[0061] The method aims to assist in a choice of at least one acquisition parameter for the acquisition of the time series signals to be used for the development of the MDL classification model.
[0062] For example, the method aims to assist in choosing a combination of sampling frequency and data quantity for the acquisition of these serial signals. temporal.
[0063] The method comprises obtaining 20 at least one group of initial time series signals.
[0064] The initial time series signals are acquired with a maximum sampling frequency permitted by the acquisition device. The initial time series signals are also acquired with a maximum amount of data permitted by the acquisition device.
[0065] In this way, the initial time series signals are sufficiently precise to distinguish the different classes associated with them.
[0066] The method then comprises a development 21 of at least one group of simulated time series signals from said initial time series signals. Simulated time series signals are thus obtained for the different classes studied. The simulated time series signals make it possible to simulate acquisitions of time series signals with acquisition parameters different from those used to acquire the initial time series signals.
[0067] In particular, the simulated time series signals are produced with different sampling frequencies less than or equal to said maximum sampling frequency.
[0068] The simulated time series signals are also produced with different amounts of data which are less than or equal to said maximum amount of data.
[0069] More particularly, each simulated time series signal takes data from the initial time series signal. Using a sampling frequency lower than the maximum sampling frequency allows only a portion of the data from the time series signal to be retained. Using a quantity of data lower than the maximum quantity of data allows the quantity of data in the simulated time series signal to be reduced relative to the initial time series signal. Each simulated time series signal may have data from the initial time series signal that is repeated in the simulated time series signal. Each simulated time series signal may also have a different number of lines from the initial time series signal. Each line then has the same quantity of data.In order to evaluate the test classification models described in the following, it may be important to have a sufficient number of rows for each class. For example, it may be advantageous to use a minimum of one hundred rows per class.
[0070] The simulated time series signals are used to evaluate the impact of sampling frequencies and data amounts on classification accuracy.
[0071] In particular, the method comprises an extraction 22 of values of the characteristics taken into account for the classification of the initial time series signals and the simulated time series signals.
[0072] For example, characteristics of time series signals may include amplitudes, frequencies, durations, minimums, maximums, averages, standard deviations, etc.
[0073] The method then comprises, for each group of time series signals, a development 23 of a test classification model. Each test classification model is developed from the values extracted from the characteristics of the time series signals used to develop this classification model.
[0074] The test classification model is preferably developed using relatively fast methods. For example, it is possible to develop a test classification model using a decision tree, a random forest classifier, a support vector machine (SVM) or a k-nearest neighbors method (also referred to by the acronym "KNN"). The test classification models can thus be developed without performing cross-validation or without dividing the initial time series signals into a set of training signals and a set of test signals.
[0075] Each test classification model is associated with a sampling rate and data quantity corresponding to the sampling rate and data quantity of the time series signals used to develop that test classification model.
[0076] Next, the method comprises an evaluation 24 of the performance of each test classification model. In particular, the evaluation of the performance of each test classification model comprises a determination of the accuracy of this test classification model.
[0077] The accuracy of the test classification model is determined by comparing the class identified by the test classification model to the class entered by the user when acquiring the initial time series signals.
[0078] The evaluation of the performance of each test classification model also includes a determination of the acquisition time of the time series signals.
[0079] The evaluation of the performance of each test classification model may comprise a calculation of a score. This score is then calculated according to the different performance criteria to be evaluated. For example, the performance score of each test classification model may be calculated according to the accuracy of this test classification model, the acquisition time and the quantity of data associated with this test classification model. This performance score may in particular be defined by a weighting of the different performance criteria evaluated. The user can also define test constraints such as a maximum acquisition duration and / or a maximum quantity of data per line.
[0080] The method then comprises an indication 25 of the performance of each test classification model developed in relation to the sampling frequency and the amount of data associated with this test classification model.
[0081] For example, said performances can be displayed on a screen of the computer system SYS. A performance graph can for example be developed by the computer system and then displayed on said screen.
[0082] An example of such a performance graph is illustrated in [Fig.3]. The performance graph shows the performance versus the PRC accuracy and the ACQ_T acquisition time of the time series signals according to the SPL_R sampling frequencies (in Hz) and the DAT_L data quantities (in number of bits) associated with these models.
[0083] From the indicated performances, the user can select the acquisition parameters to be used to acquire final time series signals.
[0084] In particular, the user can choose the sampling frequency and the quantity of data associated with the classification model having obtained optimal performance with regard to the user's needs. Preferably, this selection is carried out automatically from criteria entered by the user. Alternatively, it is possible to suggest to the user certain combinations of sampling frequency and quantity of data obtaining the best performance.
[0085] The user can then use the acquisition device to acquire groups of final time series signals associated with the different defined classes. These final time series signals are acquired using the selected combination of sampling frequency and data amount.
[0086] The method then comprises obtaining 26 of these final time series signals. In particular, these final time series signals are provided as input to the PRG computer program.
[0087] The method then comprises an extraction 27 of values of the characteristics of the final time series signals.
[0088] The method then comprises developing 28 a final classification model from these final time series signals.
[0089] In particular, the final classification model is developed from the extracted values of the characteristics of the final time series signals used to develop this classification model. The classification model can be obtained by different methods than that used to develop the test classification models. For example, the final classification model can be developed from a more advanced method than that used to develop test classification models. This allows for a more accurate final classification model. In particular, the final classification model can be an artificial neural network.
[0090] The method then comprises a development 29 of a computer program comprising instructions which, when the program is executed by a computer, cause the latter to implement the final classification model.
[0091] Such a computer program can then be used by a computer, for example by a microcontroller to perform a classification of time series signals of the same type as those used to train the final classification model using the final classification model.
[0092] More particularly, the user-defined performance criteria used to analyze the performance of the test classification models may be chosen based on the computer that will run the final classification model.
[0093] Such a method makes it possible to simplify the search for a combination of sampling frequency and amount of data for the acquisition of the final time series signals used for the development of the final classification model.
[0094] Indeed, the acquisition of initial time series signals is carried out according to a single combination of sampling frequency and amount of data. The performances of the test classification models are then evaluated from simulated time series signals and not acquired by said acquisition device.
[0095] By simplifying the search for the combination of sampling frequency and data quantity for the acquisition of the final time series signals, such a method also makes it possible to reduce an overall cost of developing the final classification model because the latter can be obtained more quickly.
[0096] Such a method also has the advantage of being fully automated. In this case, the final selection of the combination of sampling frequency and data quantity is carried out automatically according to performance criteria entered by the user.
Claims
Claims
1. A computer-implemented method for developing a classification model, called the final classification model, the method comprising: - obtaining (20) at least one group of initial time series signals, said initial time series signals being associated with at least one initial acquisition parameter, - developing (21) at least one group of simulated time series signals from said at least one group of initial time series signals, said simulated time series signals of each group being associated with at least one simulated acquisition parameter different from said at least one initial acquisition parameter, - developing (23) different test classification models, each test classification model being developed from said at least one group of initial time series signals or said at least one group of simulated time series signals,each test classification model being associated with said at least one acquisition parameter corresponding to said at least one initial or simulated acquisition parameter associated with the initial or simulated time series signals used to develop this test classification model, - an evaluation (24) of performances of each test classification model, said performances being associated with said at least one acquisition parameter of this test classification model, then - obtaining (26) at least one group of final time series signals associated with at least one final acquisition parameter selected from an analysis of said evaluated performances of each test classification model, then - developing (28) said final classification model from said at least one group of final time series signals.,
2. The method of claim 1, wherein said at least one initial acquisition parameter comprises a combination of an initial sampling rate and an initial data amount of the initial time series signals.
3. The method of claim 2, wherein said initial sampling frequency corresponds to a maximum sampling frequency permitted by an acquisition device used to acquire said initial time series signals.
4. A method according to either of claims 2 or 3, wherein said initial data quantity corresponds to a maximum data quantity permitted by an acquisition device used to acquire said initial time series signals.
5. A method according to one of claims 2 to 4, wherein said at least one simulated acquisition parameter of said simulated time series signals of each group comprises a simulated sampling frequency less than or equal to said initial sampling frequency and a simulated data quantity less than or equal to said initial data quantity.
6. A method according to one of claims 1 to 5, wherein each group of initial time series signals is associated with an indicated class when said obtaining the initial time series signals.
7. The method of claim 6, wherein each group of simulated time series signals is associated with the class indicated for the group of initial time series signals from which that group of simulated time series signals is derived.
8. The method of claim 7, further comprising extracting (22) feature values from the initial and simulated time series signals, each test classification model being developed from an analysis of the extracted feature values and the class associated with each group of initial or simulated time series signals used to develop that test classification model.
9. The method of claim 8, further comprising extracting (27) feature values from the final time series signals, the final classification model being developed from an analysis of the extracted feature values and the class associated with each group of final time series signals.
10. Method according to one of claims 1 to 9, further comprising an indication (25) of the performance of each test classification model in relation to said at least one acquisition parameter associated with this test classification model.
11. The method of claim 10, wherein said indication of the performance of each classification model comprises a display on a screen of a performance graph comprising said performance of each classification model according to said at least one associated acquisition parameter.
12. The method of claim 1 to 11, wherein said evaluated performance of each test classification model comprises a accuracy, an acquisition time of a time series signal and an amount of data acquired for this time series signal.
13. Method according to one of claims 1 to 12, further comprising a development (29) of a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the final classification model.
14. Computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement a method according to one of claims 1 to 13.
15. Computer system comprising: - a memory (MEM) comprising a computer program product according to claim 14, - a processing unit (UT) configured to execute said computer program product.