Method and apparatus for machine learning-based deviation detection and event time prediction of multivariate time-series data

Abstract

The present invention relates to a machine learning-based method and apparatus for detecting deviations in multivariate time series data and predicting event timings. A method according to one embodiment of the present invention may include the steps of receiving multivariate time series data and static attribute data collected from a plurality of input channels; generating a time series feature vector through a sequence encoder; generating an attention score and an integrated feature vector through a multi-head self-attention layer; calculating a deviation score based on reconstruction error through an autoencoder; determining a prediction time point and a confidence interval through a time-by-time probability distribution prediction model; and determining a priority grade based on the deviation score and the prediction time point.

Description

Technology Field

[0001] The present invention relates to a machine learning-based time series data analysis technology, and more specifically, to a machine learning model inference method and apparatus that analyzes multivariate time series data collected from multiple input channels based on attention to calculate a deviation score and predict the occurrence time and confidence interval of a target event. Background Technology

[0002] Conventional time-series data analysis techniques used methods to determine anomalies or predict future values ​​based on data from a single input channel. However, this approach had limitations in detecting complex pattern changes because it failed to consider the correlation between multiple input channels.

[0003] Furthermore, conventional anomaly detection and event timing prediction technologies were implemented as separate models, making it difficult to make decisions by comprehensively utilizing both results. For example, even if anomaly signs were detected, it was impossible to predict when the anomaly would lead to an actual event, posing a problem that made timely response difficult.

[0004] Meanwhile, conventional event timing prediction models had a limitation in that they output only a single predicted time point, making it impossible to assess the reliability of the prediction. As a result, users could not perceive the uncertainty of the prediction, and it was difficult to perform appropriate risk management when making decisions based on the prediction results.

[0005] Therefore, there is a need for a technology that can learn the correlation between multiple input channels to detect deviations, simultaneously predict the timing of events and confidence intervals, and determine priorities by combining the two results. The problem to be solved

[0006] The present invention has been devised to solve the above-mentioned problems, and aims to provide a method and apparatus for inferring a machine learning model that analyzes multivariate time series data collected from multiple input channels based on attention to learn correlations between input channels, calculates deviation scores through an autoencoder, predicts the timing of event occurrence and confidence intervals through a probability distribution prediction model for each time point, and determines a priority grade by combining the deviation scores and the predicted timing points. means of solving the problem

[0007] A prediction method based on multivariate time series data according to an embodiment of the present invention for achieving the above objective may include: receiving multivariate time series data collected from a plurality of input channels and static attribute data of an analysis target; inputting the multivariate time series data into a sequence encoder to generate a time series feature vector; inputting the time series feature vector into a multihead self-attention layer to calculate an attention score for each of the plurality of input channels and generating a weighted summation integrated feature vector based on the attention score; inputting the integrated feature vector into an autoencoder to generate a reconstruction output and calculating a deviation score based on the reconstruction error between the integrated feature vector and the reconstruction output; inputting the integrated feature vector and the static attribute data into a time-series probability distribution prediction model to calculate the time-series non-occurrence probability of a target event and determining a prediction time point and a confidence interval based on the time-series non-occurrence probability; and determining a priority grade based on the deviation score and the prediction time point.

[0008] In addition, the step of calculating the attention score may be characterized by generating a query vector, a key vector, and a value vector from the time series feature vector, respectively, and calculating the attention score through an inner product operation between the query vector and the key vector.

[0009] Additionally, the autoencoder may include an encoder that compresses the integrated feature vector into a low-dimensional latent vector and a decoder that generates the reconstruction output from the latent vector, and the reconstruction error may be characterized as being calculated as the mean squared error or mean absolute error between the integrated feature vector and the reconstruction output.

[0010] In addition, the above time-specific probability distribution prediction model may be characterized by calculating a reference risk function and a risk ratio from the above-mentioned integrated feature vector and the above-mentioned static attribute data, and the above-mentioned time-specific non-occurrence probability may be calculated based on the cumulative integral of the above-mentioned reference risk function and the above-mentioned risk ratio.

[0011] In addition, the above priority grade may be characterized by being classified as Grade 1 when the deviation score is greater than or equal to a first reference value and the prediction time point is within a first threshold period, as Grade 2 when the deviation score is greater than or equal to a second reference value or the prediction time point is within a second threshold period, and as Grade 3 otherwise.

[0012] In addition, after the step of calculating the deviation score, for each of the plurality of input channels, the method may further include the step of recalculating the reconstruction error while masking the data of the corresponding input channel, and calculating the deviation contribution per input channel based on the amount of change in the reconstruction error before and after masking.

[0013] Additionally, after the step of determining the prediction time point and confidence interval, the method may further include the step of calculating the width of the confidence interval, outputting the prediction time point as is if the width is less than a first threshold, outputting a corrected prediction time point by applying a correction coefficient to the prediction time point if the width is greater than or equal to the first threshold but less than a second threshold, and if the width is greater than or equal to the second threshold, calculating the uncertainty contribution of each of the plurality of input channels and identifying the input channel with the highest uncertainty contribution to request additional data collection.

[0014] Additionally, the method may further include the steps of: receiving the actual occurrence time of a target event after the arrival of the predicted time; calculating an error rate between the predicted time and the actual occurrence time; and, if the error rate exceeds a threshold, analyzing the attention score to identify an input channel with a high error contribution and prioritizing the updating of the weights of a sequence encoder or attention layer corresponding to the identified input channel.

[0015] A multivariate time series data-based prediction device according to another embodiment of the present invention for achieving the above objective includes: a memory in which at least one program is recorded; and a processor that executes said program, wherein the program may include instructions for performing the steps of: receiving multivariate time series data collected from a plurality of input channels and static attribute data of an analysis target; inputting said multivariate time series data into a sequence encoder to generate a time series feature vector; inputting said time series feature vector into a multihead self-attention layer to calculate an attention score for each of the plurality of input channels and generating a weighted summation integrated feature vector based on said attention score; inputting said integrated feature vector into an autoencoder to generate a reconstruction output and calculating a deviation score based on a reconstruction error between said integrated feature vector and said reconstruction output; inputting said integrated feature vector and said static attribute data into a time-series probability distribution prediction model to calculate a time-series non-occurrence probability of a target event and determining a prediction time point and a confidence interval based on said time-series non-occurrence probability; and determining a priority grade based on said deviation score and said prediction time point.

[0016] A multivariate time series data-based prediction device according to another embodiment of the present invention for achieving the above objective may include: an input unit that receives multivariate time series data and static attribute data of an analysis target from a plurality of input channels; a feature extraction unit including a sequence encoder that encodes the multivariate time series data to generate a time series feature vector; an attention operation unit that performs a multi-head self-attention operation on the time series feature vector to calculate an attention score and generates a weighted summed integrated feature vector based on the attention score; an autoencoder unit that receives the integrated feature vector, generates a reconstruction output, and calculates a deviation score based on a reconstruction error; a time point prediction unit that receives the integrated feature vector and the static attribute data, calculates a non-occurrence probability for each time point, and determines a prediction time point and a confidence interval based on the non-occurrence probability for each time point; and a grade determination unit that determines a priority grade based on the deviation score and the prediction time point. Effects of the invention

[0017] According to the present invention as described above, by learning the correlation between multiple input channels based on multi-head self-attention, it is possible to effectively detect complex pattern changes compared to single-channel analysis.

[0018] In addition, by simultaneously calculating an autoencoder-based deviation score and a predicted time point based on a time-specific probability distribution, it is possible to perform anomaly detection and event time point prediction in an integrated manner.

[0019] In addition, by providing a confidence interval along with the prediction time point, users can perceive prediction uncertainty and improve the quality of their decision-making.

[0020] In addition, by determining priority grades based on a combination of deviation scores and prediction time points, it is possible to efficiently allocate resources to multiple analysis targets. Brief explanation of the drawing

[0021] FIG. 1 is a block diagram of a multivariate time series data-based prediction device according to one embodiment of the present invention. FIG. 2 is a diagram showing the detailed configuration of an attention operation unit according to one embodiment of the present invention. FIG. 3 is a diagram showing the detailed configuration of an autoencoder unit according to one embodiment of the present invention. FIG. 4 is a diagram showing the detailed configuration of a time point prediction unit according to one embodiment of the present invention. FIG. 5 is an overall flowchart of an inference method according to one embodiment of the present invention. FIG. 6 is a confidence interval width-based processing flowchart according to one embodiment of the present invention. FIG. 7 is a diagram showing a computer system according to one embodiment of the present invention. Specific details for implementing the invention

[0022] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0023] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0024] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0025] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" may be used to facilitate the description of the relationship between one component and other components as illustrated in the drawings. Spatially relative terms should be understood as encompassing different orientations of components during use or operation, in addition to the orientations depicted in the drawings. For example, if a component depicted in a drawing is inverted, a component described as "below" or "beneath" of another component may be placed "above" of that component. Therefore, the exemplary term "below" may encompass both the lower and upper directions. Components may also be oriented in other directions, and accordingly, spatially relative terms may be interpreted according to the orientation.

[0026] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0027] FIG. 1 is a block diagram of a multivariate time series data-based prediction device according to one embodiment of the present invention.

[0028] Referring to FIG. 1, a multivariate time series data-based prediction device (100) according to one embodiment of the present invention may include an input unit (110), a feature extraction unit (120), an attention operation unit (130), an autoencoder unit (140), a time point prediction unit (150), and a grade determination unit (160).

[0029] The input unit (110) receives multivariate time series data from a plurality of input channels. In addition, the input unit (110) receives static attribute data of the analysis target.

[0030] As an optional embodiment, the multivariate time series data may include measurements from multiple channels collected continuously in chronological order. Each input channel may correspond to a different physical quantity or measurement item, and the data collected from the multiple input channels are aligned on the same time axis to constitute the multivariate time series data.

[0031] As an optional embodiment, static attribute data may include inherent characteristics of the subject of analysis that do not change or change slowly over time. Static attribute data may include numerical attributes and categorical attributes.

[0032] As an optional embodiment, the present invention may be applied to various fields requiring equipment condition monitoring or asset management. For example, multivariate time series data may include measurements collected from multiple sensors, such as temperature, pressure, vibration, current, voltage, speed, and acceleration, and static attribute data may include equipment type, manufacturer, manufacturing date, cumulative operating time, previous maintenance history, etc.

[0033] As an optional embodiment, the input unit (110) may identify intervals in multivariate time series data where missing values ​​exist and perform interpolation or insertion of replacement values ​​for said intervals where missing values ​​exist. For example, linear interpolation, spline interpolation, or a last observation carried forward method may be applied.

[0034] The feature extraction unit (120) generates a time series feature vector by encoding multivariate time series data received from the input unit (110). The feature extraction unit (120) may include a sequence encoder.

[0035] As an optional embodiment, the sequence encoder may be implemented as any one of a Long Short-Term Memory (LSTM), a Gated Recurrent Unit (GRU), or a Transformer encoder. The sequence encoder learns the temporal dependencies of the time series data and outputs a fixed-dimensional time series feature vector.

[0036] As an optional embodiment, if the sequence encoder is implemented as an LSTM, it can effectively learn the long-term dependencies of the time series data. If the sequence encoder is implemented as a transformer encoder, parallel processing is possible, which can improve the speed of learning and inference.

[0037] The attention operation unit (130) performs a multi-head self-attention operation on the time-series feature vector generated by the feature extraction unit (120) to calculate an attention score. The attention operation unit (130) generates a weighted summed integrated feature vector based on the attention score.

[0038] As an optional embodiment, the attention operation unit (130) can generate a query vector, a key vector, and a value vector from a time series feature vector, respectively, and calculate an attention score through an inner product operation between the query vector and the key vector.

[0039] As an optional embodiment, the attention score represents the correlation between multiple input channels and quantifies how much one input channel influences another at a specific point in time. This allows the user to identify interaction patterns between input channels.

[0040] As an optional embodiment, the attention operation unit (130) may identify an input channel whose calculated attention score is less than a preset lower threshold, and may exclude the identified input channel from weighted summation or set the weight of the input channel to 0. This can improve the efficiency of the model by excluding the influence of input channels that have a low contribution to the prediction.

[0041] The autoencoder unit (140) receives the integrated feature vector generated by the attention operation unit (130), generates a reconstruction output, and calculates a deviation score based on the reconstruction error.

[0042] As an optional embodiment, the autoencoder unit (140) may include an encoder that compresses an integrated feature vector into a low-dimensional latent vector and a decoder that generates a reconstruction output from the latent vector. The autoencoder unit (140) is pre-trained with normal pattern data, so that when an input different from the normal pattern is received, the reconstruction error increases.

[0043] As an optional embodiment, the reconstruction error may be calculated as the mean squared error (MSE) or mean absolute error (MAE) between the integrated feature vector and the reconstruction output. The deviation score may be calculated by normalizing the reconstruction error or applying a predetermined transformation function.

[0044] As an optional embodiment, the deviation score may be normalized to a value between 0 and 1. For example, a percentile may be calculated based on the distribution of reconstruction errors derived from the training data, and the position of the current reconstruction error within that distribution may be converted into a deviation score.

[0045] The time point prediction unit (150) receives the integrated feature vector generated by the attention operation unit (130) and static attribute data received from the input unit (110), calculates the non-occurrence probability for each time point, and determines the predicted time point and confidence interval based on the non-occurrence probability for each time point.

[0046] As an optional embodiment, the time prediction unit (150) may concatenate the integrated feature vector and static attribute data and input them into a shared hidden layer, and input the output of the shared hidden layer to the first head and the second head, respectively. The first head outputs a risk ratio, and the second head outputs a variance value.

[0047] As an optional embodiment, the time point prediction unit (150) calculates a reference risk function and a risk ratio from the integrated feature vector and static attribute data, and the probability of non-occurrence at each time point may be calculated based on the cumulative integral of the reference risk function and the risk ratio. The prediction time point may be determined as the first time point in which the probability of non-occurrence at each time point becomes less than or equal to a predetermined threshold (e.g., 0.5).

[0048] As an optional embodiment, the confidence interval may be determined based on the variance value output from the second head. The confidence interval may be determined as an upper and lower limit by adding or subtracting the value obtained by multiplying the square root of the variance value by a coefficient (Z coefficient or t coefficient) corresponding to a preset confidence level around the prediction time point. For example, when applying a 95% confidence level under the assumption of a normal distribution, the coefficient may be 1.96.

[0049] The grade determination unit (160) determines the priority grade based on the deviation score calculated by the autoencoder unit (140) and the predicted time point determined by the time point prediction unit (150).

[0050] As an optional embodiment, the priority grade may be classified as Grade 1 if the deviation score is greater than or equal to a first threshold value and the prediction time is within a first threshold period, as Grade 2 if the deviation score is greater than or equal to a second threshold value or the prediction time is within a second threshold period, and as Grade 3 otherwise.

[0051] As an optional embodiment, when applied to equipment status monitoring, Grade 1 may correspond to a state requiring emergency inspection, Grade 2 to a state requiring planned inspection, and Grade 3 to a normal operating state. This allows for the efficient determination of inspection priorities for multiple analysis targets.

[0052] As an optional embodiment, the multivariate time series data-based prediction device (100) may provide an attention score for each calculated input channel by visualizing it through a user interface. This allows the user to intuitively understand how much each input channel contributed to the deviation score or the prediction time, and improves the explainability of the model.

[0053] FIG. 2 is a diagram showing the detailed configuration of an attention operation unit according to one embodiment of the present invention.

[0054] Referring to FIG. 2, the attention operation unit (130) receives a time series feature vector as input and performs a multi-head self-attention operation.

[0055] First, a query vector, a key vector, and a value vector are generated from the time series feature vector, respectively. The query vector, key vector, and value vector are each generated by linearly transforming the time series feature vector through a learnable weight matrix.

[0056] Next, an inner product operation (indicated by ? in Fig. 2) is performed between the query vector and the key vector to calculate the attention score. The attention score represents the relative importance that each of the multiple input channels has with respect to other input channels.

[0057] As an optional embodiment, the result of the inner product operation can be scaled by dividing it by the square root of the key vector dimension. This stabilizes the variance of the inner product operation result, thereby improving learning efficiency.

[0058] As an optional embodiment, a softmax function may be applied to the result of the inner product operation so that the attention score can be normalized to a probability distribution between 0 and 1. The sum of the normalized attention scores is 1, representing the proportion of each input channel's contribution to the generation of the integrated feature vector.

[0059] Next, a weighted summation between the calculated attention scores and value vectors (indicated by ? in Fig. 2) is performed to generate an integrated feature vector. Weighted summation is an operation that sums value vectors using the attention scores as weights.

[0060] As an optional embodiment, multi-head self-attention may include multiple attention heads. Each attention head generates query, key, and value vectors using different weight matrices and captures correlations between input channels from different perspectives. The outputs of multiple attention heads are concatenated and then linearly transformed to generate a final integrated feature vector.

[0061] As an optional embodiment, the number of attention heads can be set to 4, 8, or 16. As the number of attention heads increases, correlations from various perspectives can be captured, but computational costs increase.

[0062] Through these attention operations, correlations between multiple input channels are dynamically learned, enabling the effective capture of complex pattern changes compared to single-channel analysis. Additionally, by visualizing the attention scores, users can intuitively understand which interactions between input channels influenced the prediction results.

[0063] FIG. 3 is a diagram showing the detailed configuration of an autoencoder unit according to one embodiment of the present invention.

[0064] Referring to FIG. 3, the autoencoder unit (140) receives an integrated feature vector and calculates a deviation score.

[0065] First, the integrated feature vector is input into the encoder. The encoder compresses the integrated feature vector into a low-dimensional latent vector. The latent vector concisely represents the core information of the integrated feature vector.

[0066] As an optional embodiment, the encoder may be composed of a plurality of fully connected layers, each layer may include an activation function (e.g., ReLU, LeakyReLU, or GELU). Each layer of the encoder progressively reduces the input dimension to finally output a latent vector.

[0067] As an optional embodiment, the dimension of the latent vector can be set to 1 / 4 to 1 / 8 of the dimension of the integrated feature vector. The smaller the dimension of the latent vector, the more the autoencoder is forced to learn only the core characteristics of the data, and the reconstruction error increases when data deviating from the normal pattern is input.

[0068] Next, the latent vector is input into the decoder. The decoder generates a reconstructed output from the latent vector. The reconstructed output is the result of restoring the original integrated feature vector.

[0069] As an optional embodiment, the decoder may be configured as the inverse of the encoder. That is, each layer of the decoder progressively expands the input dimension to ultimately generate a reconstructed output of the same dimension as the integrated feature vector.

[0070] Next, in the reconstruction error calculation step, the difference between the integrated feature vector and the reconstruction output is quantified.

[0071] As an optional embodiment, the reconstruction error may be calculated as Mean Squared Error (MSE) or Mean Absolute Error (MAE). Mean Squared Error is more sensitive to large errors, while Mean Absolute Error is more robust to outliers.

[0072] Finally, a deviation score is calculated based on the reconstruction error. Since the autoencoder unit (140) is pre-trained with normal pattern data, when an input different from the normal pattern is received, the reconstruction error increases, and accordingly, the deviation score increases.

[0073] As an optional embodiment, the deviation score can be converted into a value between 0 and 1 by normalizing the reconstruction error. For example, Z-score normalization can be performed based on the mean and standard deviation of the reconstruction error calculated from the training data, and then a sigmoid function can be applied to convert it into a value between 0 and 1.

[0074] As an optional embodiment, if the deviation score exceeds a preset threshold, the data at that point in time may be determined to have deviated from the normal pattern. The threshold may be set based on the upper percentile (e.g., the 95th percentile or the 99th percentile) of the reconstruction error distribution calculated from the training data.

[0075] FIG. 4 is a diagram showing the detailed configuration of a time point prediction unit according to one embodiment of the present invention.

[0076] Referring to FIG. 4, the time point prediction unit (150) receives integrated feature vectors and static attribute data and calculates the predicted time point and confidence interval.

[0077] First, the integrated feature vector and static attribute data are input into the shared hidden layer. The shared hidden layer concatenates the two inputs to generate an integrated representation.

[0078] As an optional embodiment, the shared hidden layer may be composed of a plurality of fully connected layers, and each layer may include an activation function (e.g., ReLU, LeakyReLU, or GELU). The shared hidden layer fuses time series pattern information and static attribute information to extract features necessary for prediction.

[0079] As an optional embodiment, among the static attribute data, numeric attributes can be input into the shared hidden layer after being scaled through a normalization layer, and categorical attributes can be input into the shared hidden layer after being converted into continuous vectors through an embedding layer.

[0080] Next, the output of the shared hidden layer is input to the first head (risk ratio) and the second head (variance), respectively.

[0081] The first head outputs the risk ratio. The risk ratio indicates the relative risk of a target event occurring at a specific point in time.

[0082] As an optional embodiment, the first head may include a fully connected layer and a non-negative activation function (e.g., soft-plus or exponential function) to have a risk ratio of 0 or greater.

[0083] As an optional embodiment, the probability of non-occurrence at each time point can be calculated based on the cumulative integral of a reference risk function and a risk ratio. The reference risk function is estimated from training data, and the risk ratio serves to adjust the reference risk function according to the characteristics of the individual analysis target.

[0084] As an optional embodiment, the prediction time point may be determined as the first time point at which the probability of non-occurrence at each time point becomes less than or equal to a predetermined threshold (e.g., 0.5). Alternatively, the prediction time point may be determined as the time point corresponding to the median of the probability of non-occurrence at each time point curve.

[0085] The second head outputs the variance value. The variance value represents the uncertainty of the prediction time point.

[0086] As an optional embodiment, the second head may include a fully connected layer and a non-negative activation function (e.g., soft-plus or exponential function) to have a variance value greater than or equal to 0.

[0087] As an optional embodiment, the confidence interval may be determined as an upper and lower limit by adding or subtracting the value obtained by multiplying the square root of the variance value around the prediction time by a coefficient (Z coefficient or t coefficient) corresponding to a preset confidence level. For example, when applying a 95% confidence level under the assumption of a normal distribution, the coefficient may be 1.96.

[0088] Through this dual output structure, unlike conventional methods that output only a single prediction time point, a confidence interval is provided along with the prediction time point, allowing users to recognize the uncertainty of the prediction and improve the quality of decision-making.

[0089] As an optional embodiment, when applied to equipment condition monitoring, the prediction time may correspond to the time when consumables need to be replaced or the time when a failure is expected to occur, and the confidence interval may represent the range of uncertainty of the prediction. For example, if the prediction time is 30 days later and the confidence interval is ±5 days, the system may recommend performing an inspection between 25 and 35 days.

[0090] FIG. 5 is an overall flowchart of an inference method according to one embodiment of the present invention.

[0091] Referring to FIG. 5, a multivariate time series data-based prediction method according to one embodiment of the present invention consists of a six-step process.

[0092] First, a multivariate time series data-based prediction device (100) receives multivariate time series data and static attribute data (S510). The multivariate time series data includes measurements collected in chronological order from a plurality of input channels, and the static attribute data includes the unique characteristics of the subject of analysis.

[0093] As an optional embodiment, in step S510, intervals containing missing values ​​among the multivariate time series data are identified, and interpolation or insertion of replacement values ​​may be performed for the intervals containing missing values.

[0094] As an optional embodiment, when applied to equipment condition monitoring, multivariate time series data may include measurements collected from multiple sensors such as temperature, pressure, vibration, and current, and static attribute data may include equipment type, manufacturer, cumulative operating time, previous maintenance history, etc.

[0095] Next, the multivariate time series data-based prediction device (100) generates a time series feature vector (S520). Multivariate time series data is input into a sequence encoder to generate a time series feature vector in which temporal dependency is learned.

[0096] As an optional embodiment, the sequence encoder may be implemented as any one of an LSTM, a GRU, or a transformer encoder.

[0097] Next, the multivariate time series data-based prediction device (100) generates an integrated feature vector (S530). The time series feature vector is input into a multi-head self-attention layer, an attention score for each of the multiple input channels is calculated, and a weighted summed integrated feature vector is generated based on the attention scores.

[0098] As an optional embodiment, input channels whose attention score calculated in step S530 is less than a preset lower threshold are identified, and the identified input channels may be excluded from weighted summation or have their weights set to 0.

[0099] Next, the multivariate time series data-based prediction device (100) calculates a deviation score (S540). An integrated feature vector is input into an autoencoder to generate a reconstruction output, and a deviation score is calculated based on the reconstruction error between the integrated feature vector and the reconstruction output.

[0100] As an optional embodiment, the autoencoder is pre-trained with normal pattern data, and when an input different from the normal pattern is received, the reconstruction error increases, resulting in a higher deviation score.

[0101] As an optional embodiment, after step S540, for each of the plurality of input channels, the reconstruction error is recalculated with the data of the corresponding input channel masked, and the deviation contribution per input channel can be calculated based on the amount of change in the reconstruction error before and after masking. Through this, the user can identify which input channel contributed the most to the increase in the deviation score.

[0102] Next, the multivariate time series data-based prediction device (100) calculates the prediction time point and confidence interval (S550). The integrated feature vector and static attribute data are input into a probability distribution prediction model for each time point to calculate the probability of non-occurrence of the target event for each time point, and the prediction time point and confidence interval are determined based on the probability of non-occurrence for each time point.

[0103] As an optional embodiment, the prediction time point may be determined as the first time point at which the probability of non-occurrence at each time point becomes less than or equal to a predetermined threshold (e.g., 0.5). The confidence interval may be determined as the upper and lower limits of the prediction time point based on the variance value.

[0104] As an optional embodiment, when applied to equipment condition monitoring, the prediction point may correspond to the time when consumables need to be replaced or the time when a failure is expected to occur, and the confidence interval may represent the range of uncertainty of the prediction.

[0105] Finally, the multivariate time series data-based prediction device (100) determines a priority grade (S560). The priority grade of the analysis target is determined based on the deviation score and the prediction time point.

[0106] As an optional embodiment, the priority grade may be classified as Grade 1 (Urgent) if the deviation score is greater than or equal to a first threshold value and the prediction time is within a first threshold period, as Grade 2 (Caution) if the deviation score is greater than or equal to a second threshold value or the prediction time is within a second threshold period, and as Grade 3 (Normal) otherwise.

[0107] As an optional embodiment, the priority grade and prediction time information calculated in step S560 may be provided through a user interface. This allows the user to efficiently determine the inspection priority for multiple analysis targets.

[0108] As an optional embodiment, the attention score for each input channel calculated in step S530 may be visualized and provided through a user interface. This allows the user to intuitively understand the basis of the prediction results and improves the explainability of the model.

[0109] FIG. 6 is a confidence interval width-based processing flowchart according to one embodiment of the present invention.

[0110] Referring to Fig. 6, the confidence interval width-based processing performs different subsequent processing depending on the width of the confidence interval.

[0111] First, the confidence interval width is calculated (S610). The confidence interval width is calculated as the difference between the upper and lower limits of the confidence interval. A larger confidence interval width indicates higher uncertainty at the prediction time.

[0112] Next, it is determined whether the confidence interval width is less than the first threshold (S620). If the confidence interval width is less than the first threshold, the prediction time point is bypassed and output as is (S621). This means that the confidence of the prediction is sufficiently high and no separate correction is required.

[0113] As an optional embodiment, the first threshold value may be set based on the lower percentile (e.g., the 25th percentile) of the confidence interval width distribution calculated from the training data.

[0114] If the confidence interval width is greater than or equal to the first threshold, it is further determined whether the confidence interval width is less than the second threshold (S630). If the confidence interval width is greater than or equal to the first threshold and less than the second threshold, a correction coefficient is applied at the prediction time, and a corrected prediction time is output (S631).

[0115] As an optional embodiment, the correction factor may be determined in proportion to the confidence interval width. For example, a correction factor that adjusts the prediction time point more conservatively (to an earlier time point) as the confidence interval width increases may be applied. This allows for securing a safety margin in situations of high uncertainty.

[0116] As an optional embodiment, the second threshold may be set based on the upper percentile of the confidence interval width distribution calculated from the training data (e.g., the 75th percentile or the 90th percentile).

[0117] When the confidence interval width is greater than or equal to the second threshold, the uncertainty contribution of each of the multiple input channels is calculated, and the input channel with the highest uncertainty contribution is identified (S632).

[0118] As an optional embodiment, the uncertainty contribution may be calculated using a gradient-based method or a masking-based method. In the case of the gradient-based method, each input channel x i The absolute value of the partial derivative of the variance output σ² for |∂σ² / ∂x i | can be defined as the contribution. In the case of the masking-based method, each input channel x i The change in the confidence interval width ΔW when masking is performed (e.g., excluding the channel, replacing it with a replacement value, or setting the weight of the channel to 0). i It can be defined as a contribution.

[0119] The input channel with the highest uncertainty contribution is the input channel that contributes most significantly to the uncertainty of the prediction, and prediction reliability can be improved by securing additional data from that input channel.

[0120] Next, additional data collection for the identified input channels is requested (S633). This can induce the user to collect additional information to resolve the uncertainty of the prediction.

[0121] As an optional embodiment, a request for additional data collection may be provided through a user interface in the form of a message such as "Additional data is needed" along with the corresponding input channel name. Alternatively, a command may be transmitted to a data collection device to temporarily increase the sampling period of the corresponding input channel.

[0122] As an optional embodiment, when applied to equipment condition monitoring, if the confidence interval width is greater than or equal to a second threshold, the collection of additional information, such as vibration sensor data, temperature sensor data, or current sensor data, may be requested. This reduces the uncertainty of the prediction and allows for the determination of a more accurate inspection time.

[0123] Through this confidence interval width-based processing, the reliability of prediction results can be improved by adaptively responding to the level of uncertainty in the prediction.

[0124] FIG. 7 is a diagram showing a computer system according to one embodiment of the present invention.

[0125] A machine learning-based method for detecting deviations in multivariate time series data and predicting event timing according to one embodiment of the present invention can be implemented in a computer system (1000), such as a computer-readable recording medium.

[0126] Referring to FIG. 7, a computer system (1000) may include one or more processors (1010), memory (1030), user interface input device (1040), user interface output device (1050), and storage (1060) that communicate with each other via a bus (1020). Additionally, the computer system (1000) may further include a network interface (1070) connected to a network (1080). The processor (1010) may be a central processing unit or a semiconductor device that executes processing instructions stored in memory (1030) or storage (1060). Memory (1030) and storage (1060) may be various forms of volatile or non-volatile storage media. For example, memory may include ROM (1031) or RAM (1032).

[0127] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the invention.

[0128] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all scopes equivalent to or equivalently modified from the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention.

[0129] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0130] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0131] 100: Multivariate time series data-based forecasting device 110: Input unit 120: Feature extraction unit 130: Attention operation unit 140: Autoencoder unit 150: Time Prediction Unit 160: Grade Determination Unit 1000: Computer System 1010: Processor 1020: Bus 1030: Memory 1031: ROM 1032: RAM 1040: User interface input device 1050: User interface output device 1060: Storage 1070: Network Interface 1080: Network

Claims

Claim 1 A prediction method based on multivariate time series data comprises: (a) receiving multivariate time series data collected from multiple input channels and static attribute data of an analysis target; (b) inputting the multivariate time series data into a sequence encoder to generate a time series feature vector; (c) inputting the time series feature vector into a multi-head self-attention layer to calculate an attention score for each of the multiple input channels and generating a weighted summation integrated feature vector based on the attention scores; (d) inputting the integrated feature vector into an autoencoder to generate a reconstruction output, calculating a reconstruction error based on either the Mean Squared Error (MSE) or the Mean Absolute Error (MAE) between the integrated feature vector and the reconstruction output, and normalizing the reconstruction error to calculate a deviation score; (e) inputting the integrated feature vector and the static attribute data into a shared hidden layer to output a risk ratio from a first head and a variance value from a second head, and based on a preset reference risk function and the cumulative integral of the risk ratio, the time-series non-occurrence probability of a target event (f) a step of calculating, determining the first time point at which the probability of non-occurrence at each time point falls below a preset threshold as the prediction time point, and determining the confidence interval of the prediction time point based on the variance value; and (f) a step of determining a priority grade based on the deviation score and the prediction time point.A method comprising, wherein in step (f), the priority grade is classified as Grade 1 if the deviation score is greater than or equal to a first threshold value and the prediction time point is within a first threshold period, as Grade 2 if the deviation score is greater than or equal to a second threshold value or the prediction time point is within a second threshold period, and as Grade 3 otherwise; further comprising, after step (d), a step of recalculating the reconstruction error for each of the plurality of input channels while masking the data of the corresponding input channel, and calculating the deviation contribution per input channel based on the change in the reconstruction error before and after masking; further comprising, after step (e), a step of calculating the width of the confidence interval, outputting the prediction time point as is if the width is less than a first threshold, outputting a corrected prediction time point by applying a correction coefficient to the prediction time point if the width is greater than or equal to a first threshold but less than a second threshold, and calculating the uncertainty contribution of each of the plurality of input channels if the width is greater than or equal to a second threshold, and identifying the input channel with the highest uncertainty contribution and requesting additional data collection. Claim 2 A method according to claim 1, wherein step (c) is characterized by generating a query vector, a key vector, and a value vector from the time series feature vector, respectively, and calculating an attention score through an inner product operation between the query vector and the key vector. Claim 3 A method according to claim 1, wherein in step (d), the autoencoder comprises an encoder that compresses the integrated feature vector into a low-dimensional latent vector and a decoder that generates the reconstruction output from the latent vector, and wherein the reconstruction error is calculated as the mean squared error or mean absolute error between the integrated feature vector and the reconstruction output. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 In a multivariate time series data-based prediction device, at least one memory on which a program is recorded; The program includes a processor that executes the program, wherein the program comprises: receiving multivariate time series data collected from a plurality of input channels and static attribute data of an analysis target; inputting the multivariate time series data into a sequence encoder to generate a time series feature vector; inputting the time series feature vector into a multi-head self-attention layer to calculate an attention score for each of the plurality of input channels and generating a weighted summated integrated feature vector based on the attention scores; inputting the integrated feature vector into an autoencoder to generate a reconstruction output, calculating a reconstruction error based on either the Mean Squared Error (MSE) or the Mean Absolute Error (MAE) between the integrated feature vector and the reconstruction output, and normalizing the reconstruction error to calculate a deviation score; inputting the integrated feature vector and the static attribute data into a shared hidden layer to output a risk ratio from a first head and a variance value from a second head, calculating a time-series non-occurrence probability of a target event based on a preset reference risk function and the cumulative integral of the risk ratio, and wherein the time-series non-occurrence probability becomes less than or equal to a preset threshold. A step of determining an initial time point as a prediction time point and determining a confidence interval of the prediction time point based on the variance value; and a step of determining a priority grade based on the deviation score and the prediction time point;The method includes instructions for performing, wherein the priority grade is classified as Grade 1 if the deviation score is greater than or equal to a first reference value and the prediction time point is within a first threshold period, as Grade 2 if the deviation score is greater than or equal to a second reference value or the prediction time point is within a second threshold period, and as Grade 3 otherwise, and the program, after the step of calculating the deviation score, for each of the plurality of input channels, recalculates the reconstruction error while the data of the corresponding input channel is masked, and calculates the deviation contribution per input channel based on the amount of change in the reconstruction error before and after masking; A device further comprising a command for performing the following steps: after the step of determining the prediction time point and confidence interval, calculating the width of the confidence interval; if the width is less than a first threshold, outputting the prediction time point as is; if the width is greater than or equal to the first threshold and less than a second threshold, applying a correction coefficient to the prediction time point to output a corrected prediction time point; and if the width is greater than or equal to the second threshold, calculating the uncertainty contribution of each of the plurality of input channels and identifying the input channel with the highest uncertainty contribution to request additional data collection. Claim 10 A prediction device based on multivariate time series data comprises: an input unit receiving multivariate time series data and static attribute data of an analysis target from multiple input channels; a feature extraction unit including a sequence encoder that encodes the multivariate time series data to generate a time series feature vector; an attention operation unit that performs a multi-head self-attention operation on the time series feature vector to calculate an attention score and generates a weighted summed integrated feature vector based on the attention score; an autoencoder unit that receives the integrated feature vector and generates a reconstruction output, calculates a reconstruction error based on either the Mean Squared Error (MSE) or the Mean Absolute Error (MAE) between the integrated feature vector and the reconstruction output, and calculates a deviation score by normalizing the reconstruction error; and inputs the integrated feature vector and the static attribute data into a shared hidden layer to output a risk ratio from a first head and a variance value from a second head, calculates the probability of non-occurrence of a target event at each time point based on a preset reference risk function and the cumulative integral of the risk ratio, and wherein the probability of non-occurrence at each time point becomes less than or equal to a preset threshold. A time point prediction unit that determines an initial time point as a prediction time point and determines a confidence interval of the prediction time point based on the variance value; and a grade determination unit that determines a priority grade based on the deviation score and the prediction time point;A device comprising: a grade determination unit classifying the priority grade into Grade 1 if the deviation score is greater than or equal to a first reference value and the prediction time point is within a first threshold period, Grade 2 if the deviation score is greater than or equal to a second reference value or the prediction time point is within a second threshold period, and Grade 3 otherwise; an autoencoder unit recalculates the reconstruction error for each of the plurality of input channels while masking the data of the corresponding input channel, and calculates the deviation contribution per input channel based on the amount of change in reconstruction error before and after masking; a time point prediction unit calculates the width of the confidence interval, outputs the prediction time point as is if the width is less than a first threshold, outputs a corrected prediction time point by applying a correction coefficient to the prediction time point if the width is greater than or equal to a first threshold but less than a second threshold, and calculates the uncertainty contribution of each of the plurality of input channels if the width is greater than or equal to a second threshold, identifies the input channel with the highest uncertainty contribution, and requests additional data collection.

Images