Predicting cause of abnormal operation of industrial machine

Abstract

The computer identifies a pre-determined future-occurring parametric state transition, where the parametric state transition is a critical transition, as the likelihood that the operational mode of the industrial machine changes in the future is pre-determined to be increased, where the operational mode is the technical state of the machine. The computer processes the run multivariate time series ({{X} op) by processing the run multivariate time series ({{X} op) representing the operation of the particular industrial machine (101) during a particular run time interval (Top) now (tcurrent) is ongoing. The computer provides a future multivariable time sequence ({X} ft) representing predicted operation of a particular industrial machine (101) during arrival of a particular predicted time interval (Tft) in the future. If both of the following conditions are satisfied, the computer predicts (423) a parameter state transition (11a, 31): i) at least one particular parameter is pre-determined to have a value that will differ from the reference value in at least one deviation segment; (ii) the likelihood that the operating mode of the industrial machine changes is pre-judged as increasing.

Description

Technical Field

[0001] Generally speaking, the present disclosure relates to industrial machines, and more particularly, the present disclosure relates to computer systems, methods, and computer program products for identifying parameters that deviate within a future time interval and are critical for the operation of industrial machines. Background Art

[0002] It is inevitable that an individual human will eventually develop a disease at least in the coming years. However, humans try to stay healthy for as long as possible and try to avoid unexpected situations such as changes in their health conditions.

[0003] To achieve these goals, it is crucial to understand the causes of diseases (including so-called "root causes" among these causes), and expertise in biology, medicine, food science, etc. helps to define appropriate activities. Basically, activities can be divided into preventive pro-active activities (aimed at maintaining health) and passive activities (aimed at restoring health when necessary). In both cases, activities involve overall monitoring of the condition, and in particular, monitoring these conditions through subtle signs of the human body. Passive activities require more specialized expertise and more complex diagnoses for monitoring.

[0004] Simply put, similar goals apply to technical devices, such as industrial machines. Even terms such as "health" or "fault prevention" are used metaphorically. A machine should operate within a certain time interval (usually several years or decades), and it should predict inevitable interruptions.

[0005] Experienced machine operators monitor their machines and understand the interactions between machine components. In many cases, the operator knows the cause of the fault so that the operator can act accordingly. To give two examples, the machine operator will proactively keep the bearings of rotating components in sufficient oil preventively, and will prepare some basic tools for some basic repairs.

[0006] However, as the complexity of the machine increases, for at least two reasons, the operator's knowledge may not be available in all cases. First, the operator may not be able to monitor the machine completely. For example, the bearings may be hidden: the bearings may be difficult to reach by the oil tank, and the characteristic noise of a dry or damaged bearing may be attenuated by the overall sound of the machine. Second, in the scenarios envisioned in the context of an automated factory, the operator may not be present at all.

[0007] Complexity exacerbates the lack of operator knowledge, leading to a worse situation: if one of the components of a complex machine fails partially, then the complex machine may fail completely.

[0008] Maintenance (to prevent malfunctions or to repair when they occur) belongs to the measures to keep the machine healthy.

[0009] Therefore, machine operation (including maintenance planning) tends to be data-driven (corresponding to expertise and complex diagnostics in a sense). Information related to the machine may come from various sources: sensors related to the machine, workshop manufacturing systems, equipment for measuring product performance, etc.

[0010] Computer-implemented tools can predict the time range within which a specific failure can be expected. The operator can take measures in advance to prevent the failure from occurring (predictive maintenance).

[0011] Some of these tools provide predictions in more complex environments: for example, the prediction can be accompanied by root cause analysis (RCA), thus allowing the operator to better identify countermeasures, or the prediction can be accompanied by an estimate of the so-called remaining useful life (RUL) of the machine without maintenance.

[0012] Such tools may apply machine learning techniques, and the training of such tools requires the availability of so-called historical data and the identification of failures in the historical data (by expert annotation or other means).

[0013] US2019 / 0147300 A1 relates to a computer system for detecting anomalies in multivariate time series. The computer receives a time series from a monitored device, and then the computer uses a pair of neural networks. Summary of the Invention

[0014] The computer executes a computer-implemented method for identifying a predicted future parameter state transition. The parameter state transition is a critical transition because the likelihood of a change in the operating mode of an industrial machine in the future is predicted to increase. The operating mode is the technical state of the machine. The computer processes a multivariate time series, which is a plurality of univariate time series. Each univariate time series represents a parameter of the industrial machine.

[0015] In the step of predicting the operation of a specific industrial machine, the computer processes an operating multivariate time series, which represents the operation of a specific industrial machine during a specific current operating time interval. In this step, the computer provides a future multivariate time series, which represents the predicted operation of a specific industrial machine during a specific predicted time interval in the future.

[0016] In the step of processing a future multi-variable time series to predict a parameter state transition, the computer evaluates two related AND conditions (i.e., logical AND conditions). If two of the following conditions are met, a parameter state transition is predicted:

[0017] (First condition) The computer processes a parameter represented by a univariate time series that is part of the future multi-variable time series. The first condition is met if the computer determines that at least one specific parameter is predicted to have a value different from the reference value in at least one deviation segment.

[0018] (Second condition) The computer quantifies the deviation of the deviation segment for the at least one specific parameter. The computer thus applies a predetermined rule. The second condition is met if the computer determines that the likelihood of a change in the operating mode of the industrial machine is predicted to increase.

[0019] To predict the operation of a specific industrial machine, the computer can use an operation predictor module. This module can be implemented through a simulator, through a machine learning tool (the machine learning tool has been pre-trained using operating multi-variable time series from past operations), or through a computer function (the computer function applies a mathematical relationship and thus processes data through a formula or a look-up table). Combinations of implementation methods can be used.

[0020] The computer can perform a prediction of the parameter state transition through a criticality predictor module that is adapted to identify deviation segments in the univariate time series by comparing the future multi-variable time series with a reference multi-variable time series separately for each univariate time series. The deviation segments allow the computer to identify parameter deviations, quantify deviation values, or indicate variable-specific errors predicted to occur during a future segment interval.

[0021] The computer can perform a prediction of the parameter state transition through a criticality predictor module that is adapted to identify differences between numerical values obtained by preprocessing image or sound samples.

[0022] The computer can perform a prediction of the parameter state transition through a pre-trained criticality predictor module to determine that the likelihood of a change in the operating mode of the industrial machine has increased. The criticality predictor module can be trained with historical data related to the machine mode, where the machine mode serves as the ground truth at the output and the data related to the parameters and quantified deviations serves as the input data.

[0023] The computer can also perform step simulation in the following way: The computer modifies the representation of the at least one specific parameter to obtain a set of parameter variants, wherein, in at least one univariate time series representing the at least one specific parameter, the computer replaces the deviated segment with a replacement segment obtained from a reference or generated by an autoencoder. For each parameter variant individually, the computer predicts the operation of a specific industrial machine and evaluates whether the at least one specific parameter still has a value different from the reference value and whether there is still a possibility of changing the operation mode of the industrial machine. Then, the computer selects the parameter variant that has the least impact on the operation of the specific industrial machine.

[0024] The computer can execute this method as a method for action recognition. Thus, the computer outputs the selected parameter variant as a recommended action to the operator of the industrial machine.

[0025] The computer can limit the data processing during simulation to parameters that are actuator parameters.

[0026] The parameter state transition can indicate that a specific industrial machine transitions to a specific operation mode of operating as an abnormal machine.

[0027] In addition, when a computer program product is loaded into the memory of a computer system and executed by at least one processor of the computer system, the computer program product causes the computer system to execute the steps of the computer-implemented method.

[0028] The computer system includes multiple modules, and when the multiple modules are executed by the computer system, the multiple modules execute the steps of the computer-implemented method.

[0029] In other words, using a computer system, the parameters of a specific industrial machine can be distinguished to identify a subgroup of parameters that are critical parameters causing abnormal operation of the industrial machine. Description of the Drawings

[0030] Figure 1 A state transition diagram for machine parameters is shown, wherein the state transition diagram includes nodes and edges;

[0031] Figure 2 An industrial machine and a computer are shown.

[0032] Figure 3 A multivariate time series representing multiple machine parameters is shown;

[0033] Figure 4 Another multivariate time series is shown, and the other multivariate time series represents multiple machine parameters in different timing aspects;

[0034] Figure 5shows Figure 3 a multivariate time series, which is divided into: a time series available for the entire time interval between the start time point and the end time point, a time series available for a partial time interval from the start time point to the observation time point applicable to the current operation of the industrial machine, and a time series available for a partial time interval from the predicted time point to the end time point applicable to the future operation of the industrial machine;

[0035] Figure 6 shows the multivariate time series and each univariate time series from the perspective of variable dependence, wherein the univariate time series is shown in a Cartesian coordinate system;

[0036] Figure 7 shows a time graph for the steps performed by running a predictor and a critical predictor through a computer module, and an availability graph for the multivariate time series for different timing aspects;

[0037] Figure 8 shows the continuation of a time graph for additional steps that can be performed by a simulator module, and an additional availability graph;

[0038] Figure 9 shows a symbolic illustration of a bearing;

[0039] Figure 10 relates to Figure 9 the bearing and shows the simulation of temperature;

[0040] Figure 11 repeats Figure 2 the illustration of the industrial machine and the computer 200 in, but adds an example of an alternative use for an autoencoder;

[0041] Figure 12 shows alternative variable preprocessing by showing an adapter;

[0042] Figure 13 shows alternative variable preprocessing by showing image data;

[0043] Figure 14 shows alternative variable preprocessing by data aggregation; and

[0044] Figure 15 shows a general-purpose computer. Detailed Description

[0045] Parameter Status

[0046] Figure 1A state transition diagram for parameter X_i of an industrial machine is shown. For this overview, the specific semantics of parameter X_i (e.g., "temperature", "vibration", etc.) are not important, so the machine parameter is identified by the so-called variable index i. Nodes (shown as rectangles with rounded corners) represent states, and directed edges (shown as arrows) represent state transitions.

[0047] Parameters are associated with values. For simplicity, this specification assumes that these values are numerical. It is conceivable that the values could be more complex data sets, such as images or sound samples, and these data sets are preprocessed into numerical values. Details are described in the section "Multimodal and Variable Preprocessing" at the end of this specification.

[0048] This specification uses the term "deviation" to represent the situation where the value of a parameter is different from a reference value during a certain period in the past, present, or future. Terms and phrases such as "deviation", "deviating", "parameter deviation" refer to states, not transitions. In a time series diagram, the deviation will also be represented by the / / symbol.

[0049] A parameter (e.g., "X_i") can change the state of the parameter from "corresponding to reference" to "deviating from reference" (see arrow 11). For convenience, the specification and the drawings respectively use the abbreviations CORR and DEV to refer to these two states.

[0050] Parameter state transition

[0051] State transitions can be defined in two directions, and as used herein:

[0052] · A transition to state DEV represents a state change from CORR to DEV (arrow 11), and

[0053] · A transition to state CORR represents a state change from DEV back to CORR (arrow 12).

[0054] State transitions can occur in the past, present, or future, and can be referred to as "past transitions", "present transitions", and "future transitions". For ease of explanation, transitions should occur at a point in time such that the duration of the transition can be ignored here.

[0055] For example, machine components can typically rotate, where the value of the parameter "rotation" ranges between 1,000 rpm and 2,000 rpm (state CORR). However, when the machine component decelerates to 800 rpm, the parameter "rotation" will undergo a transition to state DEV (arrow 11, transition to DEV). The component may return to CORR when transitioning back to CORR (arrow 12).

[0056] Typically, the reference values have tolerances (or permitted values) within the so-called tolerance bands. These tolerance bands can be given for values or for time intervals. Thus, exceptions can be defined. For example, if the reference permits a deviation of the time value, at least within a certain predefined interval (e.g., within a few minutes), the parameter will remain in state CORR and the parameter will not show a deviation (see Figure 4 the example in

[0057] From a theoretical point of view, it seems impossible for an industrial machine to operate with all parameters remaining constant over time. When the parameters change, the way the industrial machine operates will be different. However, not all parameter changes are relevant or decisive. Some parameter changes may even go unnoticed.

[0058] It is well known in the art to consider the operation of a machine (and the operation of other technical devices) through the so-called technical state. The machine state can be detected by a human observer, and the human observer can attribute this state to the machine (e.g., by observing that the machine has stopped operating). As a comparative example, a change in the rotational speed of a machine part can be noticed, but a human observer may not attribute a specific state. A state can also be assigned through interaction with a human.

[0059] It is well known that a certain change in quantity leads to a change in quality. The quality of the machine operation may ultimately change. A technician knows that such a change is a change in the operation mode. As used herein, "affecting the operation mode of the machine" is a convenient notation for indicating an increase in the possibility (or probability) that the operation mode has changed (now) or will change (in the future).

[0060] The operation mode of the machine can also be regarded as a kind of machine state, but it can be assumed that the parameter X_i in state CORR does not affect the operation mode of the machine. However, the parameter X_i in state DEV may or may not affect the operation mode of the industrial machine.

[0061] In short, the operation mode (of the machine) can be an additional parameter X_mode for the state of the operation mode (of the machine), and the state for the operation mode (of the machine) is characterized and simplified, for example, as "normal behavior" or "abnormal behavior". Abnormal behavior can be defined among the following criteria and, for example, only according to the following criteria (and detected according to predefined criteria):

[0062] · The machine cannot perform as expected, for example, because the machine delays the processing of physical materials, or because the machine emits substances into the environment and thereby exceeds a threshold (the threshold of permissible emissions).

[0063] ·The machine has a complete failure and stops running.

[0064] ·There are dangers or safety hazards for human operators.

[0065] ·There is a risk that some components of the machine may be damaged.

[0066] For example, the impacts can be summarized as follows:

[0067] If X_i = DEV_CRITICAL

[0068] Then X_mode = abnormal

[0069] Otherwise X_mode = normal.

[0070] The impact of this parameter can be associated with other parameters, for example:

[0071] If X_i = DEV_CRITICAL

[0072] And

[0073] X_k = HIGH

[0074] Then X_mode = abnormal

[0075] Otherwise X_mode = normal.

[0076] The operating mode of the machine can be detected by humans when the machine is running (i.e., now) or even by viewing data (i.e., the machine has run in the past). The human detector is not necessarily the operator of the machine (see, Operator 190). For example, the mentioned processing delay can be detected in subsequent production stages, and emissions can be detected through regular inspections. A complete failure or stop is likely to be detected by anyone nearby. Safety hazards, etc. can also be found during regular inspections. Technicians know other occasions.

[0077] Since the operating mode is easily detectable, data on this mode (as historical data) can also be used for training purposes.

[0078] However, in many cases, this simple causal relationship is unknown. It is also expected that the impacts come from multiple parameters (not only from X_i, but also from X_k and other parameters). However, for simplicity, this description continues to discuss the state and transitions for parameter X_i.

[0079] Figure 1Transitions within a sub - state are shown by arrow 31 (transition to DEV_CRITICAL) and arrow 33 (transition to DEV_NON_CRITICAL). Transitions between sub - states can also occur in the past, present, or future (i.e., can also be past, present, or future transitions). State transitions are concepts introduced here for ease of explanation. Thus, two transitions can occur simultaneously.

[0080] Thus, a transition (here exemplified by parameter X_i) is conceived as follows:

[0081] CORR transitions to DEV_NON_CRITICAL (arrow 11b)

[0082] At a specific point in time, parameter X_i may start to deviate without having a negative impact. Such a transition should be monitored because in some cases, X_i may become DEV_CRITICAL.

[0083] DEV_CRITICAL transitions to DEV_NON_CRITICAL (arrow 32)

[0084] At a specific point in time, parameter X_i can be DEV_CRITICAL, but due to changes in the machine (i.e., changes in other parameters at that time), the deviation is tolerable. The negative impact on the machine's operating mode will stop. This transition to DEV_NON_CRITICAL is desirable but not preferred in many cases because the DEV state will remain unchanged.

[0085] DEV_CRITICAL transitions to CORR, DEV_NON_CRITICAL transitions to CORR (arrow 12)

[0086] At a specific point in time, parameter X_i can return to a value corresponding to the reference, and this transition is highly desirable.

[0087] CORR transitions to DEV_CRITICAL (arrow 11a, drawn in bold)

[0088] At a specific point in time, parameter X_i may start to deviate such that from that time on, the deviation will have a negative impact on the (machine's) operating mode.

[0089] DEV_NON_CRITICAL transitions to DEV_CRITICAL (arrow 31, drawn in bold)

[0090] At a specific point in time, parameter X_i becomes critical, but due to changes in the machine (i.e., changes in other parameters), the deviation is no longer tolerable. This will have a negative impact on the operating mode.

[0091] Due to the impact on the operating mode (X_mode), the two transitions to DEV_CRITICAL (arrows 31 and 11a) are critical transitions.

[0092] Here, the attribute "critical" has the meaning of "decisive" or "relevant" (e.g., a change in the operating mode), "important" (e.g., the machine operator may receive an alarm). Although in daily use the term "critical" has a negative connotation in the sense of needing to avoid something, the impact on the operating mode can be negative or positive. For the sake of illustration only, this description preferably uses examples with negative impacts rather than positive impacts.

[0093] The computer can be trained to determine the impact on the operating mode (such as the negative impact described in the text, but theoretically also a positive impact).

[0094] Event

[0095] The transitions to DEV_CRITICAL can be regarded as events that should be (preventively and proactively) avoided. To highlight this, arrows 11a and 31 are drawn in thick lines in the figure. If they cannot be avoided, at least such events should be anticipated.

[0096] If such an event has indeed occurred, then the transitions to CORR (arrow 12) and to DEV_NON_CRITICAL (arrow 32) (away from DEV_CRITICAL) should be executed. Actions to force or prevent parameter state transitions

[0097] Figure 1 The actions are also shown by dashed lines close to the transition arrows. These actions can be distinguished according to the transitions.

[0098] · Action 611 prevents parameter X_i from changing from the CORR state to the DEV state.

[0099] · Action 611a prevents parameter X_i from changing from the CORR state to the DEV_CRITICAL state.

[0100] · Action 611b prevents parameter X_i from changing from CORR to DEV_NON_CRITICAL.

[0101] · Action 631 prevents parameter X_i from changing from the DEV_NON_CRITICAL state to the DEV_CRITICAL state.

[0102] · Action 612 supports the return of parameter X_i from the DEV state back to CORR.

[0103] · Action 632 supports the conversion of parameter X_i to DEV_NON_CRITICAL.

[0104] Put simply, a "status action" prevents a state transition, while a "transition action" supports a transition. Viewed from a slightly different perspective, a status action is a preventive proactive action, while a transition action is a retroactive proactive action.

[0105] However, Figure 1 The theory is shown, but these actions must be implemented in practice. Generally, these actions can include maintenance (before a failure occurs, either stopping the machine or not stopping the machine), repair (usually stopping the machine because a failure has occurred), calibration of sensors or actuators, replacement of parts, filling of lubricants (such as oil), supply of energy (fuel, charging a battery, etc.), changing a parameter other than X_i (for example, taking an alternative path), and other measures.

[0106] The exact terminology is not important, and it is also not relevant whether, for example, action 632 is called "repair".

[0107] An industrial machine has actuators (such as, for example, a device for changing the oil in a bearing, applying cooling, limiting the load that a robot or vehicle has to carry, keeping the temperature of a component at a predetermined value). An actuator is a machine component that can be controlled by a machine operator or a machine controller (i.e., by a computer belonging to the machine, see the controller 105 in Figure 2 ).

[0108] It is helpful to distinguish parameter X_i into actuator parameters and non-actuator parameters.

[0109] · For X_i as an actuator parameter, the actuator can perform an action and thereby trigger a transition (see arrows 12, 32).

[0110] · For X_i as an actuator parameter, the actuator can perform the action and thereby avoid a transition (see actions 611, 611a). For example, the actuator can be a heater that automatically keeps the temperature X_i within a predetermined range.

[0111] · For X_i as a non-actuator parameter, the actuator can act on a different parameter (the different parameter affects X_i). For example, X_i can be a parameter that describes the quantity and quality of the products produced by the machine, a parameter that represents the substances emitted by the machine into the environment, and X_i can be measurement data such as temperature, vibration, etc.

[0112] The following description will return to the actuator / non-actuator topic, as the distinction can help speed up calculations (or at least save computing resources).

[0113] The distinction between (non)-actuator parameters is basically independent of the above-mentioned distinction between state actions and transition actions. The same action can even be classified as a state action or a transition action. Therefore, this description only gives examples:

[0114] The action 612 that supports the transition to CORR can be implemented in the following way (dashed line) (for the parameter X_i)

[0115] · Modify X_i (by directly modifying the actuator for X_i, and this parameter will be an actuator parameter), or

[0116] · Modify X_delta (by an actuator, where modifying the parameter X_delta causes the parameter X_i to change).

[0117] The action 632 that supports the transition from DEV_CRITICAL to DEV_NON_CRITICAL can be implemented by modifying other parameters (see the example with logical AND).

[0118] Forced conversion requires knowledge of parameter interdependencies. However, this dependency can be learned (e.g., by training a machine learning tool). Modifying X_delta may have a different effect compared to modifying X_beta. This effect can be quantified (e.g., within the time required to return X_i to CORR). In this sense, Figure 1 Different potential contributors to state transitions are also shown. (Different contributors can be identified through simulation, as discussed at the end of the "Causality" section of the specification).

[0119] Criticality

[0120] Figure 1 It is shown that the state transition of a single parameter X_i to DEV_CRITICAL will cause the machine to run abnormally (X_mode = abnormal). However, two or more parameters being in the state DEV simultaneously may also cause this situation:

[0121] If X_i = DEV

[0122] And

[0123] X_k = DEV,

[0124] Then X_mode = abnormal.

[0125] For the sake of illustration, the description assumes that the abnormal operation of the machine can be triggered by a single parameter entering the state DEV.

[0126] The combination of multiple parameters in the state DEV may also be the cause of the machine running abnormally.

[0127] The computer-implemented function of detecting the conversion of a single parameter to DEV_CRITICAL (alternatively, detecting that one or more parameters in DEV cause X_mode to become abnormal) is the function of the criticality detector.

[0128] Since the criteria for criticality are independent of the time point, it is possible to detect criticality for the past, present, and future. Criticality detection includes the logical steps of detecting the deviation (status DEV) and evaluating whether the deviation is critical (DEV_CRITICAL) or non-critical (DEV_NON_CRITICAL).

[0129] As mentioned above, historical data regarding the machine mode is available, and machine learning tools can be trained to detect the impact on the operating mode. The training set (with historical data) will include:

[0130] · The machine mode as the ground truth,

[0131] · Data related to the parameters for quantifying the deviation of at least one specific parameter (e.g., in the deviation segment).

[0132] However, it is not necessary to use machine learning tools to detect the impact.

[0133] Other perspectives

[0134] Incidentally, Figure 1 the principle in will apply to beneficial deviations that have a positive impact on the operating mode, and the status can be named BENEFICIAL instead of CRITICAL. As mentioned above, the term "critical" is used for the overall impact (or, the increased likelihood of a mode change), whether negative or positive. Alternatively, the status CORR does not have to be associated with the normal state, and the status CORR can also be associated with a fault (see, for example Figures 9 to 10 ).

[0135] The simplified notation "critical parameter" or "CP" represents a parameter in the state DEV_CRITICAL. The simplified notation "a parameter becomes a critical parameter" represents the conversion as shown by the two thick arrows 11a and 31 (undesirable events).

[0136] Considering the future, the phrase "a parameter will become critical" represents a future conversion (arrows 11a and 31). More specifically:

[0137] · Currently, the status is CORR, and a conversion from CORR to DEV_CRITICAL will occur in the future (arrow 11a).

[0138] · Currently, the status is DEV_NON_CRITICAL, and a transition to DEV_CRITICAL will occur in the future (arrow 31).

[0139] Other actions can be described accordingly.

[0140] Enhancement

[0141] Figure 1 The state transition diagram for parameter X_i in [] can be enhanced by considering repetitions. Within a predefined time interval, the transition can occur multiple times. Sequences such as CORR to DEV, DEV to CORR, CORR to DEV, etc. can be taken as examples. If such a sequence is observed without a specific interval, the transition back to CORR can be ignored so that parameter X_i will remain in the DEV state of parameter X_i (which may be referred to as "persistent deviation"). As an illustrative example, the transition may be accompanied by microcracks in certain machine components, and the accumulation of such microcracks over time may lead to a failure (X_mode = abnormal).

[0142] Past, present, and future state changes

[0143] This description depicts the progress of time through successive absolute time points "t_" and by relatively differentiating into "past", "present", and "future" times. At "present time", the industrial machine is operating, and at the "current time point" t_current, data is available for the computer such that the computer can start executing the steps of the computer-implemented method. The time required to transfer data from the machine to the computer can be ignored so that t_current marks "now", and time is divided into "past" and "future".

[0144] Terms such as "anticipate" or "predict", which are occasionally used as synonyms, are used to refer to the future in the description.

[0145] As will be explained in more detail below, the computer represents the machine parameters through a time series {{X}}. In such a time series, parameter X_i is one of the i = 1 to N parameters (or "variables").

[0146] For example, when the machine is now decelerating (starting to exhibit abnormal behavior), an experienced operator may immediately identify that bearing damage is the cause of this abnormality (CORR transition to DEV_CRITICAL, arrow 11a). However, modern industrial machines are complex and exhibit relatively high interdependencies between parameters.

[0147] The complexity is even further increased:

[0148] · Now, in order to recognize that a parameter is in state DEV, the computer in principle only needs to compare the parameter value with a reference value, where the parameter and the reference value of the parameter are usually corresponding in type (for example, both are "rotation" values). The computer does not have to recognize the time point at which the transition to DEV has occurred. However, this applies to the recognition of tDEV, see Figure 4 。

[0149] · Now, in order to recognize that a parameter is in state DEV_CRITICAL, the computer may need to consider the common parameters of the machine (X_k, etc.). It does not matter whether the common parameters deviate or not.

[0150] · Now, in order to recognize the reason (or "cause") for a parameter to be DEV (or even DEV_CRITICAL), even more data processing is required.

[0151] These complexity topics are even more severe for predicting the future, to recognize the parameters that will be in state DEV, to recognize that the parameter will also be in state DEV_CRITICAL, and to recognize the reasons that will occur in the future.

[0152] Example

[0153] In an illustrative example for this complexity, it is described that a rotating component (of the machine) is lifted onto a vehicle, and the movement of the vehicle within the factory is studied (see Figure 6 ). For a specific path (i.e., the parameter "path"), if the vehicle continues on this path, the computer recognizes "vibration" as a parameter to become DEV_CRITICAL. In this example, due to the vehicle carrying a load, severe vibration would be critical.

[0154] The presence or absence of a load is a (binary) operating mode parameter. This parameter "load" is potentially affected by severe vibration. When the vehicle is loaded (i.e., carrying a load), the impact may be negative. When the vehicle moves unloaded, the impact may be negligible.

[0155] In other words, a deviation of the parameter "vibration" has a negative impact on other parameters. For example, a deviation of the parameter "vibration" has a negative impact on the parameter "load". The above-mentioned risk of damage applies here. In short, when the vehicle shakes excessively, the load may fall off. The operator usually tries to avoid this situation.

[0156] Letting the vehicle travel unloaded ("vibration" becomes NON_CRITICAL, arrow 32) will tolerate severe vibration because the vehicle cannot lose its load. The parameter "load" no longer has an impact on criticality (see, logical AND statement). However, this method may not be suitable for the industrial environment, at least not always suitable for the industrial environment.

[0157] To avoid the normal transition to DEV (and more particularly to DEV_CRITICAL), the computer recommends specific actions: taking a different path (modifying the parameter "path", action 612). Alternatively, when the loaded vehicle has experienced severe vibrations, performing a specific action will trigger a state transition to CORR (action 612). Or, performing a specific action will maintain the state CORR (in the case where the vibrations are still weak, action 611).

[0158] Then, the description extends the example to the identification of the cause and considers the parameter "surface": the floor may be uneven in some areas, and alternative actions require floor repair.

[0159] However, it should be pointed out again that the computer knows little about semantics. For the computer, it doesn't matter whether the vehicle is carrying a load, whether the vehicle is shaking somewhere in the factory, or whether the floor can be repaired. The computer processes data.

[0160] Possibility

[0161] As explained, the parameter in the state DEV_CRITICAL is related to the abnormal operation of the machine, but the computer needs to distinguish this parameter from many other parameters. As used herein, identifying a specific parameter as DEV_CRITICAL actually determines that this parameter has a higher possibility of becoming (or becoming) DEV_CRITICAL compared to other parameters. In other words, the operating mode of the machine may change or remain unchanged, but the critical transition (transition to DEV_CRITICAL) increases the possibility (or statistical probability) of the change in the operating mode.

[0162] The computer executes this method at a relatively high speed in order to identify critical parameters as early as possible. In other words, there is a trade-off between:

[0163] · Identifying critical parameters as accurately as possible in the case of relatively high resource utilization rates (computer operation and software complexity), and

[0164] · Finding critical parameters as quickly as possible with an accuracy that allows the operator to modify the parameters (or repair / replace machine components).

[0165] Optionally, the computer identifies actions that reduce the possibility of the parameter deviation becoming critical in the future - in the case where this action is actually taken.

[0166] Considering possibilities is more important for predicting future (undesirable) events (as opposed to determining that an event has occurred in the past). For simplicity, it is described that the hypothesized predicted event will actually occur (without taking any action), and (optionally) the identified actions are indeed suitable for the problem.

[0167] Overview of industrial machines and parameters

[0168] Figure 2 Industrial machines 101 and 102 are shown, and computer 200 is shown. Machine 101 (and, according to an embodiment, also machine 102) is controlled by machine controller 105. The figure shows computer 200 separate from controller 105. From a high-level perspective, both controller 105 and computer 200 are computers in the sense of having a processor, a memory, a data storage, etc. Computer 200 performs computer-implemented methods (such as the steps in method 403, Figures 7 to 8 . In an embodiment, computer 200 may be implemented as part of controller 105.

[0169] Machine 101 is an operating industrial machine, and machine 101 is also the machine being observed. Briefly, machine 101 may not be operating as expected, or may even malfunction, and the operator seeks measures to get machine 101 back to normal operation. Looking at the machine at the present time, the following scenarios can be distinguished:

[0170] (Scenario PAST) In machine 101, a particular parameter may have changed to state DEV_CRITICAL in the past. The state may still be DEV_CRITICAL now. Computer 200 can help operator 190 identify the particular parameter and the reason for becoming DEV_CRITICAL. Computer 200 can also identify the parameter to be modified (see the actuator and non-actuator differences above). Such scenarios are related to descriptive and diagnostic analysis.

[0171] (Scenario FUTURE-EVENT) In machine 101, a particular parameter will change to state DEV_CRITICAL (within a predictable time interval in the future), and computer 200 has (now) assisted operator 190 in identifying the particular parameter. Optionally, the computer identifies the reason for becoming DEV_CRITICAL or at least the reason for becoming DEV. Computer 200 can also identify the parameter to be modified (see the actuator, non-actuator differences above). This scenario is related to predictive analysis.

[0172] (Scenario FUTURE-ACTION) In machine 101, certain parameters may transition to the state DEV_CRITICAL (within a predictable time interval in the future), and computer 200 assists operator 190 in identifying the specific parameter (now), but the computer also identifies one or more actions to avoid the specific parameter from transitioning to DEV_CRITICAL. Such a scenario is related to prescriptive analysis. In other words, the action is an event avoidance action.

[0173] Performing an action does not necessarily mean stopping the machine from running. The operator can schedule the action (before the expected event occurs).

[0174] In the vehicle example, the computer anticipates that the parameter "vibration" will become DEV_CRITICAL and suggests actions: taking a different path or fixing the problem by repairing the floor. Note that the parameter "vibration" is a non-actuator parameter. However, taking a different path will act on the actuator that steers the vehicle (the path here is an actuator parameter).

[0175] Experts in the machine maintenance field discuss and analyze in various papers, such as:

[0176] · "Hybrid Data-Driven and Physics-Based Modelling for Prescriptive Maintenance of Gas-Turbine Power Plant" by Sergei Nikolaev et al., DOI: 10.1007 / 978-3-030-42250-9_36.

[0177] · "PriMa-X: A reference model for realizing prescriptive maintenance and assessing its maturity enhanced by machine learning" by Tanja Nemeth et al., Procedia CIRP 72 (2018) 1039–1044.

[0178] This description focuses on prescriptive analysis and prescriptive maintenance, in other words, on scenarios such as FUTURE-EVENT and FUTURE-ACTION. Therefore, this description uses terms that point to the future. For example, the computer "anticipates a state (or transition)" and the computer "predicts behavior".

[0179] However, in order to obtain the results of a FUTURE - EVENT or FUTURE - ACTION, the computer may occasionally process past data. In other words, the computer may perform descriptive and diagnostic analysis as a basis for predictive analysis.

[0180] From a high - level perspective, it is also convenient to view Figure 7 as follows: The computer 200 executes a computer - implemented method (reference numeral 403 in Figure 7 ) to distinguish the parameters of a specific industrial machine 101 and identify the parameters of the parameter CP that will be critical in the future.

[0181] At the present time, as shown in the user interface 290, the computer 200 identifies to the operator 190 the parameter that is (or will become) the CP. The computer may identify other critical parameters, but for simplicity, the description uses the singular (''critical parameter CP''). When a parameter is shown by the interface 290, the parameter does not have to be the CP.

[0182] As already explained, the CP is the parameter that will show a critical deviation. As mentioned above, there are two cascading conditions (DEV and CRITICAL, see Figure 1 ).

[0183] Therefore, it seems appropriate to eliminate the deviation (see arrow 12 in Figure 1 ). The operator can simply readjust the parameter, or have the electronics of the machine controller 105 (see Figure 2 ) readjust the parameter. After the modification, the parameter will again be in the state CORR. As mentioned above, this simple method only applies to actuator parameters.

[0184] This distinction has a practical application for the performance of the computer 200. In two FUTURE scenarios (predicting an undesired event or suggesting an action to be taken), before the event occurs (see arrows 11a and 31 in Figure 1 ) and / or before the action is taken, the computer is faced with ''time pressure'' to provide results (i.e., identify the CP, propose an action for parameter modification, etc.).

[0185] Therefore, optionally, the computer can limit the computer's calculations (or more generally, data processing) to parameters that are actuator parameters. For such actuator parameters, the action is expected to be promising (higher likelihood of success). (This does not exclude the possibility that the computer also processes non - actuator parameters.)

[0186] Industrial machine 101 and computer 200 operate substantially simultaneously (at least for present and future scenarios). In other words, the operating time of machine 101 may substantially correspond to the operating time of computer 200 such that the critical parameter CP is seen early enough for operator 190 to take action in time (before an undesired event occurs).

[0187] Reference machine

[0188] Figure 2 Machines 101 / 102 performing different functions are shown:

[0189] · Industrial machine 101 is a machine that must identify a specific parameter as the critical parameter CP (in the future) in order to be able to take actions etc.

[0190] · Industrial machine 102 (reference machine, reference function) has provided (or continues to provide) reference data related to the operation of machine 101. Providing the reference is a condition for predicting the states CORR, DEV and predicting the state transitions to DEV and CORR.

[0191] Having the reference is also useful for detecting DEV_CRITICAL (or generally distinguishing between NON_CRITICAL and CRITICAL). The reference may have caused machine failures in the past. This is convenient but not necessary, and there are cases where machine failures (in the past) may not be available, at least for specific parameters.

[0192] In the vehicle example, the failure of the load dropping may not be available yet. The vehicle may have just been launched such that the failure data is not available yet (at least for such failures). However, the computer can detect the parameter "vibration" as a parameter that can be converted into CP based on the prediction of the future.

[0193] Computer

[0194] In short, computer 200 processes data representing parameters (at least some of the parameters). Conveniently, the data can be used as a multivariate time series {{X}} ( Figure 3 reference numeral 500 in the time series diagram). A person skilled in the art obtains such data, for example, by communicatively coupling machine 101 and computer 200, and thus this description will not elaborate further.

[0195] Computer 200 predicts whether (and when) a specific parameter becomes the critical parameter CP, and the computer can identify actions that can prevent the parameter from becoming critical.

[0196] Data source from a time perspective

[0197] As used herein, a multivariate time series {{X}} can be conveniently differentiated according to the time when it becomes available (to a computer) during which the time series is relevant (to be processed). Describing uses the terms "temporal aspect" and abbreviations,

[0198] · {{X}}_rf (reference),

[0199] · {{X}}_op (operation),

[0200] · {{X}}_ft (future),

[0201] · {{X}}_ft_CP (future with critical parameters), and

[0202] · {{X}}_ft_action (future with actions).

[0203] Reference time series

[0204] The reference time series {{X}}_rf represents a multivariate time series with reference data. As Figure 2 shown, {{X}}_rf can be obtained from machine 102, and the reference data can have different uses:

[0205] · The reference data can provide data representing the operation of machine 101 in a normal operating mode so that the reference data can be used to determine that a parameter is (or will be) in state DEV. In other words, the reference data can be data representing the ideal operation of the machine.

[0206] · The reference data can include annotations by human experts, for example to identify situations (including state DEV_CRITICAL, state DEV_NON_CRITICAL).

[0207] · The reference data can provide replacement segments (for the time series to be processed) to be used in alternative steps of the method.

[0208] A person skilled in the art can obtain reference data in various implementations, such as {{X}}_rf. Machine 102 can be considered a physical machine or an equivalent implemented in some other way, such as an equivalent implemented by a computer (e.g., by computer 200 or by a different computer).

[0209] In a first implementation scenario, machines 101 and 102 are physically the same machine but play different roles at different times. The machine (acting as 102) has provided {{X}}_rf as historical data. There will typically be a collection of such data.

[0210] In a second implementation scenario, machines 101 and 102 are physically independent machines belonging to a group of similar machines.

[0211] In a third implementation scenario, machine 102 is a machine virtualized by a computer. A person skilled in the art is familiar with the concept of digital twin (machine 101 is the original machine and machine 102 is the twin machine). The digital twin will provide {{X}}_rf.

[0212] In a fourth implementation scenario, computer 200 employs an autoencoding module, and the details are described in combination with Figure 11 below.

[0213] In a fifth implementation scenario, computer 200 implements a predefined rule. For example, when a variable value (e.g., temperature) exceeds a predefined threshold, the computer can detect that a univariate time series starts to deviate (i.e., has a deviation segment), and if the variable value is below the threshold, the computer can identify the end of the deviation. A person skilled in the art can apply such and other rules in other ways (e.g., different thresholds to reflect hysteresis).

[0214] Although the reference time series {{X}}_rf may have annotations, for example, the machine had a fault due to some parameter values in the past, such annotated reference data may not be available. In the vehicle example, the reference data may point to non-standard vibrations but not to vehicle faults.

[0215] Running time series

[0216] The running time series {{X}}_op represents a multivariate time series that has data representing the operation of machine 101 during a specific running time interval (from t_start to t_current) that is currently ongoing. The time point t_start should be selected such that the numerical values in {{X}}_op still affect the operation of the machine.

[0217] {{X}}_op will be obtained as measurement data from sensors by collecting metadata (from shop floor applications, etc.). As described above, {{X}}_rf may have annotations, but for {{X}}_op, experts usually do not have time to annotate {{X}}_op. Over time (e.g., a new T_WINDOW), {{X}}_op can become {{X}}_rf.

[0218] Future time series

[0219] The future time series {{X}}_ft represents a multivariate time series having data representing future operations (from t_predict to t_end, during T_ft) of the machine 101. {{X}}_ft cannot come from the machine 101 / 102, but rather from the computer 200 (from the operation predictor 210).

[0220] The future time series {{X}}_ft_CP represents a multivariate time series having data representing future operations of the machine 101 but with the additional identification of CP. The time series is applicable from t_predict to t_end, but becomes available at t_result. t_result follows t_predict, but with a delay due to the execution of the method. It is convenient to describe CP as part of the multivariate time series, as this notation identifies the point in time when parameter deviations become critical (occurring within the interval T_event, see Figure 1 In the vehicle example, the parameter "vibration" (ie, {X}3) would be CP.

[0221] {{X}}_ft_CP may include additional information, such as the duration during which the parameter is expected to remain critical (see Figure 7 The T_event in the DEV_CRITICAL knows what happened, but some prediction can even estimate the duration of DEV_CRITICAL).

[0222] The action time series {{X}}_ft_action represents a multivariate time series with indications of possible actions that mitigate the impact of (future) critical parameter deviations or prevent critical deviations from occurring. Figure 1 Actions on state and transition differences taking into account (non-)actuator parameters etc. have been discussed in . The indication may take these differences into account.

[0223] In many use cases, the action consists of modifying a parameter (if it is an actuator parameter, modifying CP; if CP is a non-actuator parameter, modifying some other parameter). In other words, the parameter is no longer critical if it is modified (see Figure 1 Again, the time series notation is convenient, since an action identifier can be suggested for a specific point in time (t_action). Given the human health metaphor introduced above, such an action corresponds to a preventive proactive activity.

[0224] As illustrated below in connection with the simulation, {{X}}_ft_action may be available in different variables. Different variables can identify different actions, and the simulator can select an appropriate action to take (from among multiple action candidates).

[0225] Modules in a computer

[0226] Figure 2 Only computer modules 210 / 220 / 230 that are involved in performing a computer-implemented method are mentioned.

[0227] · The run predictor 210 receives {{X}}_op and provides {{X}}_ft).

[0228] · The criticality predictor 220 obtains {{X}}_ft and {{X}}_rf and provides {{X}}_ft_CP). This module performs future criticality detection.

[0229] · The simulator 230 is an optional module that obtains {{X}}_ft_CP and provides {{X}}_ft_action). Optionally, the simulator 230 can also receive {{X}}_rf.

[0230] The user interface 290 is a module that presents {{X}}_ft_CP (or just CP) and / or {{X}}_ft_action to the operator 190. In an embodiment, the module 290 can be associated via the controller 105 such that the action can become a control signal for the operation of the machine 100.

[0231] The receive / obtain and provide activities of modules 210 and 220 respectively apply to the so-called run times of these modules, i.e., steps 413 and 423 of the computer-implemented method (see Figure 7 ). Details of the simulator 230 will be described in connection with Figure 8 .

[0232] As used herein, the term "receive" has the meaning of data entering the computer, while "obtain" has the meaning of the computer generating data through processing. For example, the criticality predictor 220 obtains {{X}}_ft from the run predictor 210.

[0233] Although the notation of the multivariate time series {{X}} suggests that many univariate time series {X}i are to be processed, the computer 200 does not have to process all the data that can be provided by the machines 101 / 102. A person skilled in the art can filter out some data.

[0234] Module implementation

[0235] Each module in the module can be implemented in a variety of ways. In principle, these methods are independent of each other. This specification will briefly introduce some methods, but some of these methods will be described in more detail below.

[0236] The run predictor 210 can be implemented in a variety of ways.

[0237] · The run predictor 210 can be implemented by a simulator (but not simulator 230). Such a simulator can be used in a machine controller, such as controller 105 ( Figure 2 ).

[0238] · The run predictor 210 can be implemented by a machine learning (ML) tool pre-trained, for example, by processing the reference time series {{X}}_rf. For training, {{X}}_rf can be obtained from multiple {{X}}_op acquired from past runs. In other words, the training can be based on processing historical data. A prominent example of an ML tool is a neural network. The run predictor 210 can use a so-called autoencoder, the details of which will be described below.

[0239] · The run predictor 210 can also be implemented by a computer function that applies mathematical relationships and thus processes data through formulas, look-up tables, etc. For example, an industrial machine can have a water heater as a component. Future temperature values can be calculated based on the current value {temperature}_op and the power of the heater element. Or, the look-up table references historical observations that still apply.

[0240] The run predictor 210 can be implemented by a combination of these implementation methods, and those skilled in the art can also apply other techniques.

[0241] The criticality predictor 220 can be implemented in a variety of ways, and two logical steps for detecting criticality have been mentioned.

[0242] To detect the state DEV, the criticality predictor 220 can be implemented to perform, for example, the following:

[0243] · The criticality predictor 220 can identify a deviated segment in the time series by comparing {{X}}_ft with {{X}_rf, see Figure 4 .

[0244] · The criticality predictor 220 can compare the data obtained through preprocessing. For example, {X}i can include an image (or a sound sequence), and {X}i_ft and {{X}_rf can be image classifications to be compared. (The example should be the "uneven floor" mentioned).

[0245] The critical predictor 220 may replace the deviation segments (in the time series) with the corresponding segments from the reference time series. This replacement (details below) may approximate the actual time series to its reference time series. In short, the difference between the unmodified time series (e.g., the original_op) and the modified time series (i.e., with replacements) may serve as an indication of criticality.

[0246] Figure 2 The simulator 230 is shown as a single box, but in an implementation, the simulator 230 may be a module that calls the functions of the run predictor 210 or the critical predictor 220.

[0247] For example and as will be described in detail below, the simulator 230 may obtain information that one or more parameters will become DEV in {{X}}_ft (e.g., the parameter value exceeds a threshold). The simulator 230 will also obtain information that one of the DEV parameters will become DEV_CRITICAL in {{X}}_ft_CP. Then, the simulator 230 may replace the deviation (e.g., replace with reference data, and in an implementation, only replace some of the deviation segments), and may have the run predictor 210 and the critical predictor 220 provide {{X}}_ft_CP and {{X}}_ft_CP again. The simulator 230 may repeat the replacement for different variants. Some of the variants will result in the parameter transitioning to the state CORR (or DEV_NON_CRITICAL). Replacing the deviation with reference data is equivalent to modifying the parameter. After multiple simulations, the simulator may output an action recommendation.

[0248] In the vehicle example, the simulator 230 will obtain {{X}}_ft_CP, indicating that {X}3 will become DEV, and that the DEV in {X}3 will even be DEV_CRITICAL. The variables {X}1 and {X}2 are in the DEV state (since the reference has other values). The variant results in the following scenario: {{X}}_ft avoids the DEV_CRITICAL state for {X}3.

[0249] The simulation model may be local because the simulation model is only related to a part of the machine. Such a simulation model may support identifying the origin of an event (such that adjustments are needed to avoid the event from occurring).

[0250] Figure 3 A multivariate time series 500 representing multiple parameters is shown. Each variable represents a single parameter. {{X}} represents the multivariate time series, where the i = 1 to N variables {X}i represent N parameters. The index i is the variable index (see Figure 1) where N is the variable number. {{X}} includes multiple univariate time series: {X}1, {X}2, ..., {X}i, ..., {X}N.

[0251] The variable {X}i is given as a univariate time series. In an alternative notation, the univariate time series can be given as a sample sequence {X1...XM}i.

[0252] M is the number of samples in the observation interval T_WINDOW (i.e., the time length of the time series), and each sample is identified by an index m. The interval has a duration of T_WINDOW = Δt * M. Δt represents the sampling interval. T_WINDOW will be shown again in Figure 5 the example of.

[0253] As used herein, Δt is the same for all i. This is for convenience of explanation, but in fact it is not necessary. Different univariate time series can use different sampling intervals. For example, temperature will be measured once a minute, Δt = 60 seconds, but current will be measured once a second, Δt = 1 second.

[0254] The symbol Xmi represents the value of the parameter sample 505 in the variable {X}i at the time point tm. For example, Xmi can be a temperature value of 30°C. Since the semantics are irrelevant, the values can be normalized. For example, for a temperature range from -30°C to 70°C (from "0" to "1"), 30°C can be treated as "0.5". The figure shows the variable range by an arrow 510 (i.e., the y-axis for each variable). A person skilled in the art can apply preprocessing to filter out infeasible variable values. For example, a defective sensor may occasionally output a value of 1.000°C, so such overly high values can be ignored.

[0255] The time point tm is given at the end of each sampling interval Δt. This is just a convenient convention. In other words, tm represents the sampling interval that ends at tm.

[0256] As used herein, the time series completely represents the observation interval from t1 to tM. The division in time is called a "segment". For example, Figure 3 shows the segment {X}i(ts,te), and the segment {X}i(ts,te) includes the values from Xsi to Xei. The indices "s" and "e" represent the "start" and "end" time points of the segment. Figure 3 Arbitrarily shows a segment, and Figure 4 will show different segments with specific meanings in.

[0257] As described above, a time series represents a parameter (of an industrial machine). Accordingly, a segment of the time series represents the parameter during the time interval from the start time point to the end time point (i.e., during the segment interval).

[0258] Since this specification distinguishes multivariate time series and univariate time series by writing {{}} or {}, this specification will occasionally omit "multivariate / univariate" or put multivariate / univariate in parentheses respectively.

[0259] This specification refers to a time series in which the parameter sample 505 has a numerical value. Whether or not normalization is performed, a person skilled in the art can encode Xmi in an appropriate data format (e.g., a real number in a floating point number; an integer, etc.). Since some parameters are usually represented by binary data (TRUE / FALSE, ON / OFF, etc.) or even by text or a string, a person skilled in the art can apply this method to such variables.

[0260] A person skilled in the art can preprocess data in other formats (e.g., an image) so that the data in other formats is assigned a numerical value. Examples will be given at the end of the specification.

[0261] Figure 3 There is no distinction made between the time aspects {{X}}_op, _rf, etc. (see Figure 2 ), but the differences between the time aspects {{X}}_op, _rf, etc. will be described next.

[0262] Figure 4 The multivariate time series 500 is shown by distinguishing {{X}}_op and {{X}}_ft (uniformly with thick line 501) from {{X}}_rf (dashed line 502). Figure 3 of.

[0263] The multivariate time series 501 and 502 are corresponding because the reference series 502 has samples corresponding to the samples in the running time series 501:

[0264] · ({{X}}_op corresponds to {{X}}_rf)

[0265] · ({{X}}_ft corresponds to {{X}}_rf).

[0266] This correspondence has two aspects:

[0267] · Variable correspondence means that the univariate time series {X}i_op has an equivalent univariate time series {X}i_rf because the two series refer to the same type of parameter.

[0268] · Temporal correspondence means that for two univariate time series {X}i_op (or {Xi}_ft) and {X}i_rf, there are samples taken at the same relative times. The figure illustrates the temporal correspondence with respect to {X}1 with the vertical line 514.

[0269] A person skilled in the art can pre-identify the corresponding time series, and the same index i is used in this specification.

[0270] For simplicity, it can also be assumed that in {{X}}_op (or {{X}}_ft) and {{X}}_rf, the number of variables N is the same. This is convenient but not necessary.

[0271] As mentioned before, reference data is usually available in the case of tolerance bands:

[0272] · {X}1_rf should have upper and lower bounds that remain constant throughout T_WINDOW. The bounds have different line styles.

[0273] · {X}2_rf should be similar to {X}1_rf but with a larger tolerance (here: distance between bounds)

[0274] · {X}3_rf should increase during T_WINDOW.

[0275] · {X}N_rf should remain constant for a relatively long period of time but should decrease near the end of T_WINDOW.

[0276] The tolerance should not be confused with the range 510 (see Figure 3 ).

[0277] Segments can also be identified, not as arbitrarily shown in Figure 3 but based on the relationship between the univariate time series {X}i_op (or _ft) and {X}i_rf.

[0278] As explained above through the Figure 1 state transition diagram in, a parameter can change the state of a parameter, and the states and transitions can also be related to the timing diagram (as shown in Figure 4 ).

[0279] According to a predetermined rule, a computer can distinguish the segments into:

[0280] · Deviation segments (parameter state DEV, here in the example, from tDEV to tM, given as / X / i for {X}i and as / X / N for the shorter interval of {X}N, or

[0281] · Non-deviation segments (parameter state CORR).

[0282] As described above, the segment representation of a time series represents a parameter during a segment interval over a time interval. Thus, the attributes "deviating" and "non-deviating" also apply to the parameters during the segment duration, where the parameter may deviate, or the parameter may not deviate. Thus, a deviating segment identifies a parameter deviation, quantifies the deviation value, or indicates a specific error of a variable. A deviating segment may occur in the past, present, or future (if predicted) during the time interval - the segment interval.

[0283] For example, if the running (or future) value Xmi_op (or Xmi_ft) starts to differ from its corresponding reference value Xmi_rf, the first rule may define the start of a deviating segment / / . The first rule may correspondingly define the end of the deviating segment / / .

[0284] According to this first rule, the time series {X}i_op ( Figure 4 within) has a deviation in the segment / X / i_op because the value (Xmi) of the time series {X}i_op is different from the corresponding value in {X}_rf. The same principle also applies to the segment / X / N_op.

[0285] For this first rule, it does not matter whether the reference value is higher than the running value (such as for variable i) or lower than the running value (such as in variable N). The second rule may take into account "higher than" or "lower than".

[0286] The third rule may be specific to a time series with binary values. For example, for the binary time series {0,1,1,1,0} with a reference value of {0,0,0,0,0}, there will be a deviating segment / 1,1,1 / because this segment is not zero as in the reference.

[0287] The number of segments of each time series {X}i is limited by the number M of values within T_WINDOW.

[0288] The fourth rule may be based on other relationships. For example, if the reference time series produces oscillations (e.g., between +1 at tm and -1 at t(m + 1), and so on), while the running (or future) time series remains at zero (with minimal variation), then the running (or future) time series will be considered deviating as long as there are no oscillations.

[0289] The fifth rule may take into account different reference values. It is convenient to take a vehicle as an example (see Figure 6 ): The route taken by the vehicle corresponds to a specific path category (non-deviating, state CORR), but deviates from (state DEV) other route categories.

[0290] Similarly, a deviation segment can also be determined for {{X}}_ft. Instead of {}, the symbol / / only indicates that this segment is a deviation segment. The computer can track the timing of each deviation segment separately. For example, the computer can track the start time point and the end time point (see Figure 3 The ts and te introduced in it are applied to the deviation segment / / ) to track the timing of each deviation segment separately.

[0291] A person skilled in the art can select appropriate rules and can set additional rules.

[0292] Qualitative analysis of segments

[0293] The basic knowledge for segment identification (and distinguishing these segments into deviation segments and non - deviation segments) has been described. This specification will now discuss some more advanced observations that can be derived from the segments and applied to the univariate time series {X}i_op (or equivalently, {X}i_ft). The advanced observations can be related to criticality and can be performed by a criticality detection function (e.g., by the criticality predictor 220).

[0294] Again, these segments are merely convenient illustrations in the time series diagram to illustrate state transitions (see Figure 1 ).

[0295] In the first category, the univariate time series can be distinguished into a deviation time series (requiring at least one deviation segment / / ) and a non - deviation time series. Figure 4 It is shown that {X}1 and {X}2 are non - deviation time series, while {X}i and {X}N are deviation time series.

[0296] In the same first category, the deviation time series can be further distinguished according to the number of deviations: a multi - deviation time series has at least two / / , and a single - deviation time series has exactly one / / , as in {X}i and in {X}N. The number of deviations corresponds to the number of state transitions (multiple CORR to DEV transitions, or only a single transition).

[0297] In the second category, univariate time series (with deviations) can be compared with each other by criteria such as the following:

[0298] · For each (deviation) time series, the integral of the change over time of the difference between the value for the (deviation) segment and a reference value can be calculated. Assuming that the two time series use the same range (see Figure 3 The range 510 in it), the integrals can be compared. The two sequences are distinguished by the integrals of the two sequences.

[0299] · The number of deviation segments can be used to classify time series, which is simplified to distinguish between sequences with frequent deviations and those with occasional deviations. This criterion can be regarded as a cross-variable criterion as it takes into account multiple variables (i = 1 to N).

[0300] · The duration of the deviation can also be used as a criterion. In a simplified example, for {0, 1, 1, 1, 0} relative to the reference value {0, 0, 0, 0, 0}, the deviation has a relative duration of 60%.

[0301] This second type of discrimination may help (subsequently) determine whether the deviation (DEV status, following the CORR to DEV transition) is also a critical deviation (thus leading to abnormal operation of the machine).

[0302] Detection of criticality

[0303] Not every deviation (whether a segment deviation or otherwise) will result in DEV_CRITICAL. If multiple parameters are in the DEV state, the computer can introduce a ranking. The ranking can be advantageous as it can enhance the computer's robustness to prevent it from alerting the operator by showing potentially non-critical parameters.

[0304] For example, this specification now describes a (optional) method for detecting criticality.

[0305] The computer (Predictor 220) can evaluate the deviation segments / / to calculate the impact of the deviation (or the impacts of multiple deviations).

[0306] Taking {X}i as an example, within the time interval from tDEV to tM (the duration of the deviation segment, the transition from CORR to DEV), the graphical area between the dashed line and the thick line can quantify the deviation. The calculation is a simple integral calculation. The deviation should have a deviation value DEV(i).

[0307] Similarly, for {X}N, the graphical area between the dashed line and the thick line results in the value DEV(N). The deviation segment is shorter.

[0308] Rules can be defined to identify criticality, and these rules can be similar to the rules for identifying deviations. The following are given by way of example:

[0309] · If the deviation value is high (optionally, the magnitude of the value), then the first univariate time series is more critical than the second univariate time series. For example, DEV(i) > DEV(N), such that the deviation in parameter i is critical. This is an example of criticality identification that compares deviation segments in different variables (here: i and N).

[0310] · If the value is higher than the threshold, the univariate time series deviates critically. This is an example of criticality identification that can be maintained within the variable without the need to compare with other variables.

[0311] · The univariate time series may be partially critical. For example, the time series may have a non-critical deviation followed by a critical deviation.

[0312] The deviation value DEV(i) can be regarded as a variable-specific error. The overall error ERROR can also be defined, for example, as the sum of all DEV(i) from i = 1 to i = N. In a modification, the overall error can be calculated as the sum of the squares of all DEV(i). There are other options.

[0313] Replacement segment

[0314] When the computer attempts to identify an action (see Figure 1 ), the computer (the combination of the criticality predictor 220 and the simulator 230) can—for simulation purposes only—replace the deviating segment with a non-deviating segment.

[0315] This will be illustrated with an example for Figure 4 . Through the dashed line and the symbol ~~, Figure 4 the replacement segment corresponding to the deviating segment / / in time is shown. The value of the replacement segment can correspond to a reference (in the example, the intermediate value between the upper tolerance value and the lower tolerance value). The replacement segment can also be calculated by an autoencoder. This will be explained in detail below.

[0316] For example, the replacement segment ~X~i corresponds in time to the deviating segment / X / i (the segments start and stop simultaneously), but corresponds in value (the deviation “drops” but the reference rises and the replacement replaces the reference).

[0317] For example, the replacement segment ~X~N corresponds in time to the deviating segment / X / N, but does not correspond in value.

[0318] The computer (the predictor 220) can use the replacement segment ~~ to calculate the impact of the deviation.

[0319] The computer can calculate:

[0320] · The deviation value DEV(i) as described above, which only applies to the deviating segment;

[0321] · The “reference difference” REF_DIFFERENCE(i) (the difference between _op or _ft and _rf (to the upper or lower boundary or the midline)).

[0322] For simulations, the computer can apply variations (v1...v4), for example, the computer can apply variations (v1...v4) by selectively replacing deviating segments, or not replacing deviating segments. The computer calculates REF_DIFFERENCE(1) and REF_DIFFERENCE(2) for the unchanged time series, but for (i) and (N), the computer generates differences.

[0323] ·Simulation (1) with variant v1, in which / X / i and / X / N are replaced by ~X~i, ~X~N.

[0324] • Simulation (2) with variant v2, where only / X / i is replaced.

[0325] • Simulation (3) with variant v3, where only / X / N is replaced.

[0326] Simulation (4) with variant v4, no replacement.

[0327] In general, there are K variations vk (k=1 to K), for example, K=4 variations are just simplified examples.

[0328] The computer can calculate the ERROR for all variables {{X}} from i=1 to i=N, and the error will be specific to the case simulation. Since substitution (theoretically) will reduce the deviation value (in the simulation), one of simulations (2) or (3) will point to the variable with the greatest impact. (Simulations (1) and (4) can be ignored).

[0329] Other options are also available.

[0330] Action Revisited

[0331] In short, the most critical deviations may lead to the highest level of action. This will be illustrated by example. In view of the (non)actuator theme introduced above, it is important to note that the greatest impact produced by simulation may not (yet) lead to the described action.

[0332] Simplified example of a time series

[0333] Figure 5 The univariate time series of parameter X_1 is shown in Figures (i), (ii) and (iii).

[0334] Diagram (i) shows the time series of the complete window time interval T_WINDOW between the time point t_start and the time point t_end.

[0335] ·Chart (ii) shows a time series of a partial time interval from the starting time point t_start to the observation time point t_current applicable to the current operation of the industrial machine.

[0336] ·Chart (iii) shows a time series from the predicted time point t_predict to the time point t_end applicable to the future operation of the industrial machine.

[0337] In the simplified example, the univariate time series {X}1 in the chart should belong to Figure 3 the multivariate time series {{X}} introduced in. Using the value-time notation, {X}1 is shown vertically with the numerical value of {X}1 and horizontally with time progression.

[0338] The T_WINDOW interval is conveniently divided into a first sub-interval from the starting time point t_start to the time point t_intermediate, and a second sub-interval up to t_end. These two sub-intervals can have approximately equal durations, such that t_intermediate occurs approximately at T_WINDOW / 2 after t_start.

[0339] As shown in Chart (i) of the figure, during the T_WINDOW interval, the numerical value of {X}1 should rise from the minimum value ("min") to the maximum value ("max") during the first sub-interval and should fall back to the minimum value again during the second sub-interval. Since the figure is given by a straight line, it can be assumed that the increase (or decrease) rate can be approximately constant. This is just a convenient simplification.

[0340] It can also be assumed that this increase / decrease behavior can be derived from the reference data. For example, this increase / decrease behavior can be derived from {{X}}_rf (more specifically, from {X}1_rf that has occurred multiple times in the past). In other words, training the ML tool can establish {X}1_rf, which will not be further elaborated here.

[0341] As shown in Chart (ii), the numerical value of {X}1 should be a measured value belonging to {{X}}_op, see Figure 2 . The numerical value of {X}1 should rise from the minimum value (min) at t_start, at least to t_current (which will be the last time point when the operating value is available). The time point t_current corresponds to the current time. The line does not look as straight as in Chart (i), just to indicate that {X}1_op will be a measured value.

[0342] Since the future cannot be observed, {{X}}_op is not available at any time in the future. However, the operation predictor 210 (seeFigure 2 ) It is possible to predict what will happen next.

[0343] As shown in graph (iii) in the figure, the value of {X}1 should continue to rise (until t_intermediate) and then should fall, as described in graph (i). Therefore, this value belongs to {{X}}_ft (“future”).

[0344] In principle, almost all variables in {{X}} can be discussed, and all variables can be classified into reference (_rf), operation (_op), and future (_ft).

[0345] For this single-variable example, it is possible to detect the deviation in {X}1 (and ultimately identify the criticality of the deviation), but the following example with a vehicle requires looking at additional parameters.

[0346] Time series example

[0347] Figure 6 The multi-variable time series {{X}} and the individual single-variable time series {X}1, {X}2, and X{3} are shown from a slightly different perspective below: not with respect to the progression of time (as Figure 4 shown), but from the perspective of variable dependencies.

[0348] As shown at the top of the figure, the value of {X}1 corresponds to the vertical axis (between “min” and “max”), the value of {X}2 corresponds to the horizontal axis (also between “min” and “max”), and the value of {X}3 is represented by the line thickness. In the example, the value of {X}3 is a binary value represented by thin and thick lines.

[0349] Vehicle example

[0350] It doesn't matter what physical parameters the single-variable time series {X}1, {X}2, and {X}3 actually represent. The computer 200 knows nothing about these physical parameters, at least for most of the calculations. In the example, it is convenient to imagine the industrial machines 101 / 102 as vehicles moving within the factory 800. Such in-factory vehicles can move more or less autonomously. The vehicles can be powered by batteries. The vehicles can carry (intermediate) products, and deviations in product delivery may have different impacts on the factory.

[0351] Although the vehicle battery can be easily replaced or recharged (if needed) and the arrival delay can be negligible, the damage to the load will be difficult to handle. (Given the above rankings, the load damage will be the highest-ranked deviation.) The overall complexity does matter here. Incidentally, the factory itself can be regarded as an industrial system, and the vehicle is just one of the components in this industrial system. The vehicle can also be regarded as an example of a machine that only requires a minimal amount of interaction from a human operator to organize maintenance.

[0352] For this example, the figure also shows a top view of the factory 800. In a normal process step (or more precisely, a transportation step), the vehicle should move within the factory 800 from the first corner 801 (at the minimum / minimum position) to the second corner 802 (at the maximum / maximum position), and then the vehicle should return to the first corner 801. The vehicle can transport intermediate products throughout the factory, for example, from corner 801 to corner 802. In other words, the vehicle will return from corner 802 without a load.

[0353] {X}1 and {X}2 can represent two coordinates of the vehicle's position. The transportation step can have an average duration of T_WINDOW. Not all transports require the same amount of time.

[0354] In this scenario, the combination of {X}1 and X{2} will represent the vehicle's path 805. {X}1 will generally change over time, as Figure 4 shown, not necessarily at a constant speed, but the position {X}1 will change from "min" to "max" and then back to "min". The same applies to {X}2.

[0355] The path 805 can pass through specific areas within the factory 800, such as areas 811 and 812. The figures represent areas 811 and 812 with rounded rectangles, but the shape of the areas is not important. The computer can detect that the vehicle is located in areas 811 and 812 at any time based on the (X1,X2) coordinates within {{X}_op.

[0356] For the sake of illustration, {X}3 can also be given a meaning. For example, {X}3 can also be given the mechanical vibration of the vehicle. To maintain binary values, there should be "weak" and "strong" amplitudes of vibration, also with binary distinctions to keep the example as simple as possible. Of course, a technician can identify thresholds, etc., and can even establish the difference between the two binaries through machine learning. It should be noted that the computer treats these values as numerical values, and the use of "weak" or "strong" or other semantics here is only for the sake of illustration.

[0357] The reference data {{X}}_rf should be used for {X}1 and {X2} (i.e., increasing and decreasing), as Figure 4 shown, and the reference data {{X}}_rf should also be available for {X}3_rf. For example, for the (X1,X2) coordinates located within region 812, {X}3 should be "strong". This applies to both {{X}}_rf and {{X}}_op.

[0358] It is also possible to combine {X}1 and {X}2 and classify the reference paths. For example, there are the following:

[0359] · A "wall path" class for vehicles moving close to a wall, as in case (A),

[0360] · A "diagonal path" class for vehicles moving along the shortest path, as in case (B), or

[0361] · A "wall - diagonal path" class, as in (C).

[0362] The class names in quotes "" are given arbitrarily here, and the computer does not have to use these class names.

[0363] It can be assumed that {{X}}_rf is a set of historical data, which in the case of vehicles records past movements and may have a distribution of 50 / 40 / 10% for cases (A), (B), and (C).

[0364] Figure 5 Another modal difference is also shown: the operating value {{X}}_op is given by white circles, while the future value {{X}}_ft is given by black circles. The figure is again simplified to show approximately 10 values per trip. A more realistic in - factory vehicle takes several minutes, and a data sampling interval ΔT of one second is appropriate. Thus, a single transport step (i.e., a round trip) will provide hundreds of values.

[0365] The prediction accuracy may decrease as the time distance increases. The black circles at the end of the path may be larger than those at earlier time points.

[0366] Operating prediction

[0367] The running predictor 210 processes {X}_op and thus uses the prediction for the values of each univariate time series {X}1, {X2} (e.g., the parameter "position" with two coordinates) and {X3} (e.g., the parameter "vibration"). The last white circles (a) in (A), (b) in (B), and (c) in (C) represent the position (X1, X2) at time point t_current. (For example, if the running predictor 210 is implemented as a machine learning tool trained using historical reference data, the reference time series {X}_rf also has its role).

[0368] For example, based on the first position coordinates (X1, X2) received by the running predictor 210, the running predictor 210 can predict whether the vehicle will take the "wall path" in (A) or the "diagonal path" in (B).

[0369] The running predictor 210 also uses the path prediction for the value of {X}3. For example, if the vehicle moves through area 811 (at t_current), the vehicle will experience some "strong" vibrations in area 812.

[0370] Graph (A) shows the position (a) in area 811 at t_current with a white circle, and graph (A) shows black circles in area 811 (thin line, "weak vibration") and area 812 (thick line, strong vibration). For graphs (B) and (C), the running predictor 210 predicts movement along the diagonal path, but the vibration is predicted to be "weak".

[0371] In other words, the running predictor 210 at least partially predicts or anticipates the remaining path. The accompanying drawings show black circles for predicting the position. The running predictor 210 may not have yet distinguished the return paths starting from corner 802 in (B) and (C), but the running predictor 210 predicts {X}3 at least together with the expected position.

[0372] Deviation

[0373] It should be assumed that vibrations are unavoidable. For {X}3_ft corresponding to area 812, the criticality predictor 220 can identify the vibration value as a deviation segment (see the two black circles on the thick line). In other words, the criticality predictor 220 can anticipate the transition from CORR to DEV in {X}3 for future time points (see T_event in Figure 7 . (In other words, "weak" corresponds to CORR and "strong" corresponds to DEV).

[0374] Critical detection

[0375] It can also be assumed that at a relatively early time point t_predict (see Figure 7 ), the critical predictor 220 also predicts the new state as DEV_CRITICAL (not only DEV, but also DEV_CRITICAL). (There may be some other parameters or past observations, such as DEV_CRITICAL, due to additional parameters of the vehicle carrying the load (from corner 801 to corner 802). In other words, the event (see, thick arrow 11a) will occur during T_event, and this event will change the state of X3.

[0376] The critical predictor 220 can output {{X}}_ft_CP (to the operator 190 via the user interface 290), and identify that CP = X3 ("vibration") and the estimated time interval T_event. For example, the computer can output the identified future event and a warning (to the operator 190). (In the example, {{X}}_ft_CP can also be accompanied by an indication of the vehicle's position).

[0377] Simulation

[0378] Outputting a warning may not be sufficient. In a factory, the operator may be busy with other tasks, making the vehicle their second concern.

[0379] The simulator can simulate with the position (X1, X2) as the parameter to be changed (actuator parameter) (here, two variants along the wall or the diagonal). ERROR will be calculated, and one simulation will show that "vibration" can be minimized.

[0380] Of course, the operation of the computer is independent of semantics, and the computer may calculate DEV(1) and DEV(2) as potential contributions to the error, but taking different paths will not contribute to the overall performance of the vehicle.

[0381] Figure 7 A timing diagram showing the steps performed by the computer 200 for the method 403 implemented by the executing computer is shown.

[0382] Method 403 includes:

[0383] · In step 413, run the predictor 210 to predict the operation of the machine 101, where {X}_ft is provided, and

[0384] · In step 423, the critical predictor 220 predicts the following event: in this event, the parameter shows a deviation that becomes critical (DEV_CRITICAL, critical transition, represented by Figure 1 the arrow 11a or 31 in).

[0385] Figure 7 The illustrations and descriptions in

[0386] Figure 7 are simplified by assuming that a critical transition will occur. However, the critical predictor 220 may also perform step 423, and the result will be that no critical transition occurs. Figure 2 ) For training, the reference time series {{X}} is used as historical data.

[0387] To place the step execution in the context of data availability, Figure 7 the availability of the multivariate time series {{X}} in {{X}}_rf, {{X}}_op, {{X}}_ft, and {{X}}_ft_CP is shown. The progression of time is represented by a time axis from left to right, but the axis is not drawn to scale.

[0388] The steps are represented by boxes, where the left side of the box is projected to the earliest time point at which the step starts to execute, and the right side of the box is projected to the earliest time point at which the result of the step execution can be used (for subsequent activities).

[0389] The availability of {{X}} is represented by a horizontal line drawn parallel to the time axis. The horizontal line may be dashed to indicate that the time series is available but not necessarily used by modules 210 / 220.

[0390] Vertical lines (with arrows) (usually starting from the horizontal line) extending to the box indicate that the time series {{X}} of a specific modality is (at least partially) supplied to modules 210 / 220.

[0391] Time points and time intervals

[0392] This specification now describes (re)introducing time points and time intervals in chronological order.

[0393] The time point t_collect represents the collection of {{X}}_rf that started in the past. It is not necessary to know the exact time point t_collect. The collection may have started several years ago.

[0394] The time point t_train_1 indicates the start of training steps 412 and 422. If implemented by a machine learning tool, step 412 represents the process of training the running predictor 210, and step 422 represents the process of training the criticality predictor 220. Training is typically feasible when the appropriate time series {{X}}_rf is available. This is also indicated by the vertical arrows 1 and 2 from {{X}}_rf to the left sides of boxes 412 and 422, respectively.

[0395] The time point t_train_2 indicates that the training has been completed, enabling the running predictor 210 to predict the operation of the machine 101 (see Figure 2 ), and the criticality predictor 220 to identify the state DEV_CRITICAL.

[0396] In the drawings, the training of the two modules 210 / 220 is shown as starting and stopping simultaneously at t_train_1 and at t_train_2, respectively. This is only a simplification of the drawings and is not actually necessary. Training can continue for new available {{X}}_rf.

[0397] Starting from the time point t_train_2, the running predictor 210 and the criticality predictor 220 are ready to be used, but data supply by these modules is not yet required.

[0398] The time point t_start indicates that the industrial machine 101 starts a new operating cycle, such that {{X}}_op becomes gradually available starting from t_start (see the example of the vehicle leaving the corner 801 in Figure 6 ).

[0399] The time point t_current indicates the time when {{X}}_op is available with the latest data. T_op is the currently ongoing operating time interval (i.e., from t_start to t_present). Starting from the time point t_current, the running predictor 210 starts to execute step 413 ("prediction"), and step 413 ("prediction") includes receiving {{X}}_op. The vertical arrow 3 indicates that the running predictor 210 processes {{X}}_op (from t_start to t_current as long as available). In the example, the vehicle will be at the position of the last white circle (a), (b), or (c), see Figure 6 .

[0400] After t_current, the operation of the machine continues, and the time required for the running predictor 210 to execute step 413 is relatively short (compared to T_op, the figure is not drawn to scale).

[0401] The time point t_predict represents the earliest time point at which the running predictor 210 provides {{X}}_ft as a result (see the right side of the box). The figure is again simplified here because step 413 can continue to be executed. As time goes on, more and more data in {{X}}_op becomes available, making {{X}}_ft likely to become more accurate. In Figure 5 the illustration of, the black circles will become smaller.

[0402] The time interval from t_current to t_predict can be regarded as the running time of the running predictor 210. In an ideal situation, {{X}}_op will also continue to be available at t_predict so that the running predictor 210 can repeat this step. Therefore, from Figure 6 view, the black circles will become smaller.

[0403] The vertical arrow 4 indicates the data handover of {{X}}_ft to the critical predictor 220.

[0404] Therefore, the time point t_predict represents the earliest time point at which the critical predictor 220 can start to execute step 423. As shown by the vertical arrows 5 and 6, the critical predictor 220 does not have to rely on {{X}}_ft, but optionally can process {{X}}_rf and {{X}}_op.

[0405] At the time point t_result, the critical predictor 220 has identified a future critical state DEV_CRITICAL (i.e., in the future after t_result) for at least one parameter.

[0406] The figure shows the result of {{X}}_ft_CP by an additional horizontal line, that is, a simplified multivariate time series for future {{X}}_ft with computer-generated annotations. For example, the annotations can indicate specific univariate time series {X}i where a deviation is expected to occur, the start and end time points of the variation, numerical estimates, etc., which are represented here by the key parameter "CP". The vertical arrow 7 only shows that {{X}}_ft_CP comes from the critical predictor 220 that executes step 423.

[0407] The time interval from t_predict to t_result can be regarded as the running time of the critical predictor 220. (Since the critical predictor 220 processes {{X}}_ft but not {{X}}_op, the amount of data to be processed is relatively small compared to the amount of data available for processing.) Since the running time is relatively short, t_result will occur before the deviation will occur.

[0408] The time interval T_event represents the time interval during which the criticality predictor 220 predicts a transition to DEV_CRITICAL, i.e., the event to be avoided. T_event is given as an interval.

[0409] The time point t_end represents the latest time point at which the running predictor 210 can predict {{X}}_ft. In other words, there is a prediction time interval (T_ft) from t_future to t_end. This time point is shown in the figure to indicate that the prediction is time-limited.

[0410] In summary, at the time point t_result, the computer 200 notifies the operator 190 that a specific event (i.e., a parameter becoming critical) is expected to occur during T_event (in the future, relative to t_result). Then, the operator 190 can take appropriate actions, usually aimed at avoiding this event.

[0411] This specification will describe how the computer 200 helps the operator 190 find appropriate measures (or "actions"). This specification will refer to Figure 8 , but a brief look at the vehicle example should be helpful. Consider the timing aspects of the vehicle example

[0412] The computer 200 (with the running predictor 210 and the criticality predictor 220) does not have to deal with semantics. However, for the sake of illustration, take a brief look again at Figures 6 to 7 .

[0413] Starting from the time point t_trained, the running predictor 210 will be able to predict the vehicle path and will also be able to predict some strong vibrations in the area 812. Assume that at t_current the vehicle has approached a "wall" path, then the running predictor 210 will provide {X}3_ft because strong vibrations are predicted (see the black dots on the thick line in Figure 6 , case (A)). If at t_current the vehicle enters a diagonal path, then {X}3_ft will show "weak" vibrations (black circles on the thin line, Figure 6 , case (B)).

[0414] Since strong vibrations are not necessarily recognized as critical events (events that transition to DEV_CRITICAL, see the thick arrows in Figure 1 ), the deviation recognizer may not recognize this change in vibration as a deviation. The deviation recognizer may recognize other deviations.

[0415] Given the predicted path (through region 812), the critical predictor 220 (optionally receiving {{X}}_rf) may identify {X}3 as deviating during T_event (i.e., when the vehicle will cross region 812).

[0416] Simulation

[0417] Figure 8 Shows a continuation of the timing diagram for additional step 433 to be performed by simulator 230, as well as an additional data availability chart.

[0418] Figure 8 Repeat steps 413 and 423, and introduce step 433 as a simulation step (performed by simulator 230).

[0419] As previously described, the run predictor 210 performs step 413 (at least initially) from t_current to t_predict; and the critical predictor 220 performs step 423 (at least initially) from t_predict to t_result.

[0420] Starting from time point t_result, {{X}}_ft_CP is available (horizontal line), and the simulator 230 can start the simulation (step 433), where the simulator 230 identifies an action and the time point (t_action) for taking this action. The time point t_action should be before T_event. (The computer is again in a "time pressure" state.)

[0421] The simulator 230 performs the simulation in step 433 through the following sub-steps:

[0422] · (1) Modify the critical parameters (or other parameters) to a set of variants,

[0423] · (2) For each variant, anticipate the machine operation for the modified parameters (e.g., by calling the run predictor 210 to update {{X}}_ft),

[0424] · (3) Determine whether these variants still cause the parameter to become DEV_CRITICAL by evaluating {{X}}_ft,

[0425] · (4) By selecting the variant with the least impact on the operation (i.e., the least significant), e.g., avoiding the event (during T_event), or postponing the occurrence of the event to a more distant future (to the right in the figure) - the impact can be selected according to a predefined rule or a pre-trained module - and

[0426] ·(5) Optionally, by taking the selected output as the recommended action (or by outputting an identifier that deviates from the parameter replaced by ~~ so that referring to Figure 2 , the operator 190 understands what action should be taken).

[0427] This specification has introduced the simulator 230 as an optional module above (refer to Figure 2 ), and continues to explain the details. In other words, outputting the selection (optionally, in the context of {{X}}_ft_CP) allows the operator 190 to modify the parameter (provided that the parameter can be modified, the "actuator"). Given CP, the selection may correspond to CP, but it is not necessary.

[0428] A more detailed study of the sub-steps:

[0429] (1) The simulator 230 does not actually modify the parameter; what the simulator 230 modifies is the data representing the parameter. For example, the simulator 230 does not change the temperature or make the machine vibrate less. The simulator 230 only changes the data. Modifying a single parameter to have at least two different values results in two variants in a group, but more realistically, modifying multiple parameters (including parameters predicted to become critical (become DEV_CRITICAL). The specification will illustrate examples of vehicles with modified parameters (such as vibration, path, etc.), and an example of a bearing with four parameters having 12 variants.

[0430] (2) The simulator 230 calls the operation predictor 210, and it should be noted that for the variant, the operation predictor 210 provides a "would-be" operation. Since the parameter does not change in reality (i.e., not at the machine 101, but only in the simulation), the "would-be" operation is not necessarily a "to-be" operation.

[0431] (3) In the following paragraphs, the specification refers to vehicle examples: A specific variant (the selection of a specific path) will prevent the parameter "vibration" from becoming critical. The description here gives a counterexample because the operator 190 is interested in preventing the parameter from becoming critical (refer to the actions in Figure 1 , such as action 611a).

[0432] (4) and (5) Selecting the lowest (or in other words, "least influential") variant does not force the operator 190 to apply the variant. Outputting the selection should be understood as a recommendation. Although given in the singular form, the selection may include multiple variants so that the operator can make a final choice.

[0433] Vehicle examples

[0434] As described above, industrial machines can be very complex, and having approximately N = 3 variables is a simplification for illustrative purposes only.

[0435] Computer 200 can use simulator 230 to simulate alternative scenarios. In the example, X3 "vibration" cannot be modified, but the parameter pair (X1, X2) represents the position of the vehicle as a modifiable parameter. (In other words, the parameter pair is an actuator parameter.)

[0436] The simulator identifies two paths and a re - prediction of X3:

[0437] · "Wall path", where X3 is expected to become critical (CP = X3)

[0438] · "Diagonal path", where X3 is expected to remain CORR.

[0439] Then, simulator 230 will output the "diagonal path" as the recommended action (to be executed as t_action). In other words, the position can be modified by simply not allowing the vehicle to enter area 812 and by having the vehicle take an alternative path. Figure 6 This alternative path is shown in (D).

[0440] It is worth noting that simulator 230 provides alternative scenarios by changing actuator parameters (the vehicle has actuators to change its path), but does not provide alternative scenarios by changing non - actuator parameters (the vibration cannot be avoided, so X3 cannot be simulated as becoming "weaker").

[0441] Therefore, operator 190 can interact with the vehicle (not for all cases, but only when the vehicle is approaching area 812) to have the vehicle take an alternative path, as shown in case (D). Initially, predictor 210 may have provided {{X}}_ft, and due to the path change, {{X}}_ft will become obsolete, but predictor 210 can recalculate {{X}}_ft for the alternative path.

[0442] More specifically, simulator 230 has modules 210 and 220 repeat some calculations. Simulator 230 modifies (simulated, not real) the position (X1, X2) for the following first variant: where the remaining path predicted in {{X}}_ft, i.e., the position (a1) in case (A), the black circle. Similarly, simulator 230 modifies (X1, X2) for positions that the vehicle can also (quickly) reach.

[0443] The simulator 230 calls the updated prediction. The first variant will indicate that X3 is "heavy" (since the predicted vehicle is within region 812), while the second variant will result in X3 being "weak" (outside region 812).

[0444] The simulator 230 will re-evaluate (by calling the critical predictor 220) these two variants, but the second variant will not show deviation (X3 will still remain "weak"). There will be an impact (the vehicle will take an alternative path), but this impact is acceptable (for example, the impact is just a negligible delay for the vehicle to return to reach the corner 801).

[0445] The simulator 230 will select the second variant because X3 is expected to be "weak".

[0446] The simulator 230 will select the output as the recommended action, such as using the modified position instruction. Figure 6 This is illustrated for case (D), where the vehicle avoids region 812.

[0447] Root cause

[0448] Industrial machines provide a wide variety of data that can be used here. Continuing with the vehicle example: The vehicle can also be regarded as a machine, and the specification illustrates more aspects by referring to this example. The positions X1, X2, and the vibration X3 are parameters, but the presence (or absence) of a load at the vehicle can be another vehicle parameter X4.

[0449] It can be assumed that more data about the factory is available, such as a set of images for each position, like X5 (possibly not for every position, but in sufficient quantity to obtain images along the path (including the predicted path)). The images do not have to be used in a time series, but as the vehicle moves along the path, the set of images can be regarded as a de facto time series.

[0450] The images can be classified (image processing is an existing technology), and the classifier (the result of classification) can be used as input data (for the predictor 210, for the detector 220, for the simulator 230). The images should be classified because the images divide the floor into two categories. For example, a single image is classified as "uneven floor" or "smooth floor". The uneven floor is the (root) cause of the vibration.

[0451] The computer can ultimately relate the floor to the vibration so that the action (similar to an actuator) can be: take an alternative path unless the floor is repaired (as Figure 6 (D) shows).

[0452] Bearing example

[0453] It doesn't matter whether the action (see the output of the simulator 230) involves the operation of the machine (taking a specific path to avoid a specific area) or a special mode (in many cases, the operation may stop, for example, during maintenance). This specification continues to discuss the action (to avoid abnormal operation) by referring to the example of a bearing.

[0454] Figure 9 A symbolic diagram of a bearing is shown (two concentric circles represent the inner and outer rings), and the bearing has data type parameters "temperature", "number of rotations", "torque", "(oil) pressure", and "oil type". The figure also shows that an increase in temperature causes the bearing to fail. This typical time series can be regarded as {temperature}_rf representing a "fault". The fault may be related to a decrease in the rotational speed of the rotor supported by the bearing (for example, dropping to 600 rpm). Given the deviation introduced above: a sudden increase in temperature can be regarded as a transition from CORR to DEV_CRITICAL (see Figure 1 ).

[0455] Adding or replacing oil is a common method to avoid or delay faults (see Action 611 in Figure 1 ). However, again, the computer doesn't necessarily implement the rules considering the oil type. However, this example is for illustration purposes.

[0456] Figure 10 involves Figure 9 the bearing, and shows the simulation of the temperature parameter {temperature} of variants v1...v12 of four actuator parameters ("load", "oil type", "pressure", and "torque"). The simulator 230 (cooperating with the operation predictor 210) can perform simulations using three different ranges #1, #2, and #3 for each input parameter, resulting in 4x3 = 12 temperature values changing within a time interval (such as during T_WINDOW or during a shorter interval). In this example, there are K = 12 variants and K = 12 simulations. For simplicity, Figure 10 only four graphs of temperature changing over time are shown (instead of 12 graphs of 12 simulations of K = 12).

[0457] In other words, the time series can go from {temperature}|#1#1#1#1_ft (variant v1, where "load", "oil type", "pressure", and "torque" all have the value #1, dashed line) to {temperature}|#3#3#3#3_ft (variant v2, with the value #3, also dashed line).

[0458] Values # are specific to the parameter. For example, for the parameter "load", the values can be #1 = "unloaded", #2 = "50% loaded", and #3 = "fully loaded". There can be three different oil types, type #1, type #2, and type #3. The abbreviation _ft points to the future. The simulated data does not have to be equivalent to the operating data_op.

[0459] One time series in the time series corresponds to Figure 9 a fault scenario in, and the computer can recognize a specific temperature pattern as corresponding to a fault reference. Other time series will show scenarios (as actions) using different oil types.

[0460] Regarding Figure 1 the state and transitions in, {temperature}|#1#1#1#1_ft corresponds to {temperature}_rf( Figure 9 ).

[0461] Selected parameter values

[0462] When performing method step 433 (simulation), the simulator 230 can select the parameter variant that has the least impact on the operation of a specific industrial machine (101). There can be more than one variant. In this example, the variant of oil type #2 has the least impact. Figure 10 The variant v2 = #1, #2, #1 is illustrated. (#, #2, # can be more general: the oil type is important, other parameters are not). Since the operator 190 (see Figure 2 ) can understand from the temperature curve (not substantially heated) that selecting oil type #2 is potentially beneficial, this curve is also given as "{temperature}_ft_action", which is a specific example of {{X}}_ft_action (see Figure 2 ).

[0463] In other words, the action of using oil of type #2 to make the bearing operate can be regarded as action 611 (see Figure 1 ), because this action can keep the temperature at CORR.

[0464] Causality

[0465] As mentioned above with Figure 1 in, the parameters are interdependent in their effects.

[0466] Although Figure 1 actions 611, 612, etc. are distinguished according to the transitions (to prevent or support the transitions), the computer does not have to consider causality completely precisely.

[0467] Simulation allows for the comparison of parameter changes while taking into account the effects of parameter changes (in general on the operation of the machine, and / or particularly on the state changes of the parameters). For the vehicle example, the simulator 230 can modify additional (actuator) parameters, such as the vehicle speed {X}7, and driving at a low speed can also reduce vibrations ( Figure 1 actions 611, 612 in). The computer may not consider semantics, so only the additional parameter {X}7 needs to be changed ( Figure 1 action 612 in). Then, the computer can rank different actuator parameters.

[0468] The computer can also rank different actions (equivalent to ranking parameters). Actions with a higher rank must be executed prior to actions with a lower rank.

[0469] The differences between the actions introduced above (see Figure 1 ) can be taken into account in the ranking. The rank of action 632 (removing criticality) may be higher than that of action 611 (maintaining CORR).

[0470] Both the actions of "decelerating" and "taking an alternative path" reduce the likelihood of encountering vibrations (and the risk of the load falling off the vehicle), but these two actions may have different efficiencies. The computer may give a higher rank to "taking an alternative path" (also because the delay impact can be ignored for this action and the vehicle will basically arrive on time). Both of these actions can be regarded as Figure 1 actions 611 or 612 in.

[0471] In the example of a bearing, changing the oil type (e.g., changing to #3) may have a higher rank compared to changing the oil pressure.

[0472] Discussion on supervision

[0473] This method provides a solution for prescriptive maintenance, but historical records of past actions (such as maintenance) are not necessarily available. This method is an unsupervised method.

[0474] In the vehicle example, processing the reference data {{X}}_rf allows for predicting that X3 will become DEV (or even DEV_CRITICAL) in the future, but there is no historical record of available actions. There is no historical data available for the vehicle to take an alternative path (see Figure 6 (d)).

[0475] In the simulation, the "optimal" actuator parameters are those that produce different criticalities in the sharpest contrast: the simulated "wall path" results in (simulated) "strong" vibrations, while the simulated "diagonal path" results in (simulated) "weak" vibrations. The simulation may produce other parameters, such as different transport durations (i.e., T_WINDOW as a parameter expected to be different for the paths). However, the most significant difference is in the vibrations (from "strong" to "weak"), rather than the duration (which may be a small early arrival time). This difference in parameter evaluation does not have to be communicated to the computer through supervision; the computer automatically selects the parameters with the sharpest contrast without supervision.

[0476] Similarly, the historical data (a part of {{X}}_rf) includes data on maintenance (e.g., recharging the vehicle battery) and on failures (e.g., battery failures that lead to recharging). But the historical data does not have to contain data on all types of failures. A fully loaded vehicle may not have fallen (in the past), but potential dangerous situations can be prevented (by the vehicle taking an alternative path).

[0477] In all cases, it is impossible to avoid supervision (by human experts), but computer-implemented methods as described above can be used to identify the actions of an increasingly autonomously operating factory.

[0478] Timing

[0479] As previously mentioned, the computer 200 (through the simulator 230) proposes actions, and there should be no substantial delay. The time intervals (t_current to t_predict, running the predictor; t_predict to t_result, criticality predictor) need to be short enough for the computer to transmit the results before the event occurs ({{X}}_ft_OP) and before an action can be taken ({{X}}_ft_OP_action). The operation of the computer module can be accelerated by simplifying the following:

[0480] · The identification of the DEV state allows ignoring non-deviating parameters, so the criticality predictor 220 does not check the CORR parameters.

[0481] · Actions can be identified only for actuator parameters. This method reduces the number of candidates for identifying actions. The method can include restricting the variations to actuator parameters. In other words, actuators that cannot be changed in reality will remain "inactive" during the simulation. For example, the parameter "vibration" with two values "strong" and "weak" is caused by changing other parameters (such as position, path, etc.), and changing the vibration to a specific value may not correspond to reality and can be avoided.

[0482] Autoencoder

[0483] Deep autoencoders can be used to identify features that can be used to train the architectures explained in the paper, such as: Karl-Philipp Kortmann, Moritz Fehsenfeld, Mark Wielitzka, "Autoencoder-based Representation Learning from Heterogeneous Multivariate Time Series Data of Mechatronic Systems", arXiv:2104.02784.

[0484] Autoencoders are explained in scientific papers, such as "Cyriana MARoelofs, Marc-Alexander Lutz, Stefan Faulstich, Stephan Vogt: Autoencoder-based anomaly root cause analysis for wind turbines", Energy and Artificial Intelligence 4 (2021) 100065.

[0485] The autoencoder can be a convolutional autoencoder, or an LSTM autoencoder, or any other structure that can process sequential data, where examples are illustrated in "Roy Assaf, Ioana Giurgiu, Jonas Pfefferle, Serge Monney, Haris Pozidis and Anika Schumann, 'An Anomaly Detection and Explainability Framework using Convolutional Autoencoders for Data Storage Systems'", Proceedings of the Demonstration Session of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-2000).

[0486] Figure 11 Repeated Figure 2 illustrations of industrial machines 101 and 102 and computer 200, but with examples of the optional use of autoencoders 201, 202, and 203 added:

[0487] · In the example, the autoencoder 201 receives {{X}}_op and provides {{X}}_rf (as a supplement to or an alternative for {{X}}_rf from machine 102).

[0488] · In the example, the autoencoder 202 receives {{X}}_ft and provides a reference, not as {{X}}_rf from machine 102, but a reference based on anticipation.

[0489] · In the example, the simulator 203 uses the autoencoder for various purposes, such as to identify a replacement segment ~X~ (replacing the deviated segment in the above variations before simulation). In other words, the autoencoder 203 can also be used to calculate the replacement segment ~~ (see Figure 4 the examples of ~X~1 and ~X~N in

[0490] The illustrations and descriptions are simplified. For example, the autoencoder does not have to encode all variables of a multivariate time series, but can apply autoencoding to only some variables.

[0491] For the described method, using the autoencoder can conveniently establish {{X}}_rf (or other data), even in cases where historical data is sparse. Additionally, using the autoencoder may be beneficial because it can reduce the workload of human experts annotating relevant features (in {{X}}_rf).

[0492] For example, as Figure 4 shown by the deviation, / X / i is represented by a value different from the value in the reference band, but it may require human experts to define these reference bands (with upper and lower boundaries, etc.).

[0493] In different examples, the autoencoder can distinguish the future path of a vehicle as driving along a wall or along a diagonal. In this sense, the run data {{X}}_op (representing the position, i.e., the white circles in 6) serves as a code for the computer (here: the run predictor 210) to deduce the future path.

[0494] Multimodal and variable preprocessing

[0495] Figures 12 to 14 Illustrates variable preprocessing. More specifically, Figure 12 illustrates that the adapter 280 has image data to be adapted, Figure 13 illustrates the image data being adapted, while Figure 14 illustrates the data aggregation of the adapter 280.

[0496] The adapter 280 can be implemented as part of a module that performs a prediction step (see step 413). For example, Figure 12Shows an adapter 280 for preprocessing data from an industrial machine 101 (or from a machine 102). The adapter 280 receives one or more multivariate measurement time series from the industrial machine 101 or from a reference machine 102. The figure is simplified by passing all N variables through the adapter, but in an implementation, not all variables need to be adapted.

[0497] As described above, the symbol Xmi represents the numerical value of the parameter sample 505 at the time point tm in the variable {X}i, see Figures 3 to 4 , and the parameter sample is shown as a scalar. A person skilled in the art can receive (or obtain) samples by well-known methods. For example, a person skilled in the art can receive (or obtain) samples by collecting measurement data, by collecting metadata (from shop floor applications, etc.).

[0498] However, data capture is not limited to scalars. Data preprocessing can be applied to adapt data modalities, such as from image data to scalar data, from sound data to scalar data, from vector data to scalar data, etc.

[0499] In other words, the adapter 280 can optionally implement modality adaptation of data that was initially available as non-scalar data. For example, this adaptation is shown for the variable {X}p.

[0500] The data of the variable {X}p can be a set of digital images 281 (i.e., a matrix with pixels representing colors). The images can be acquired by a camera or a scanner. The images 281 can be used as an image sequence (e.g., images every Δt, from t1 to t8 in the figure). Given the above vehicle example, the images can show, for example, the floor of a factory.

[0501] The adapter 280 can process the images and assign scalar values to these images, which are shown here as the time series {X}p'. The label'only indicates that the assignment has occurred. A person skilled in the art can apply image processing techniques, such as classification, through a pre-trained neural network or other means.

[0502] For example, the images can show movable machine parts (or immovable parts, such as the floor). The adapter 280 can classify the parts as "uneven" or "even" (binary classification of the floor), classify as moving at speeds "0", "1", "2", "3", etc. (for the moving parts).

[0503] Since the floor may be uneven, the images can be classified accordingly. In short, an image showing unevenness can be encoded as 1 ("presence of unevenness"), while an image without unevenness can be encoded as 0 ("absence"). Figure 16 very symbolically shows the unevenness with triangles. The resulting univariate time series can be {X}p' as "op" with parameter sample {0,0,0,0,0,1,1,1}', see 505, and the computer will execute the method using {X}p'. Assuming the reference can be {X}p'_rf with parameter sample {1,1,1,1,0,0,0,0}, the deviation can be determined as the undesired "flatness" at the start of WINDOW and the undesired "unevenness" at the end of WINDOW.

[0504] A person skilled in the art can adjust the sampling rate as needed. For example, the image data may arrive in the form of a video stream (25 image frames per second), and some frames can be deleted and arrive at Δt (possibly much longer than 1 / 25 second).

[0505] In another scenario (not shown in the figure but easily imaginable), the non-scalar element can be a sound sequence. For example, {X}p can be a collection of audio recordings from a microphone sensor (or "sound recordings", e.g., each sound recording having a duration of Δt or shorter). A person skilled in the art can apply appropriate sound processing, e.g., by sampling the sound at a frequency of 20 kHz, resulting in 60 * 20,000 audio samples per minute. For the vehicle example, it is worth noting that a vehicle passing over an uneven ground may emit some characteristic sounds.

[0506] In the context of providing a time series with Δt intervals, the adapter 280 then processes these millions of samples into a single scalar. For example, the scalar can indicate: rising tone during Δt, falling tone during Δt, constant tone during Δt, rising and falling tone during Δt, and so on. These patterns can be assigned to integers, or the changing tone itself can be assigned to the parameter sample.

[0507] Figure 13 is shown as Figure 12The adapted image data shown, with modifications. Here, the adapter 280 also receives an image sequence (shown for times tm = t6, t7, and t8), but assigns scalar values in two or more (univariate) time series, given here as {X}p' and {X}q' (the prime again represents assignment). The adapter 280 identifies regions 282, 283 within the image 281 and processes the data for the regions separately. In the example, region 282 shows a machine part (the circular symbol in the figure) that rotates in the direction "+1", or rotates in the direction "-1", or does not rotate at all "0", as shown for the assignment in {X}p'. Region 283 shows "fire" (triangle symbol) corresponding to "1" or "no fire" shown as "0". In this example, the image 281 at time t6 (rotating along "+1", with fire) is shown in the figure.

[0508] Processing data requires computing resources (e.g., CPU, memory, etc.), and the overall time to identify critical parameters (with state transitions) can be crucial (see the real-time requirements mentioned above). Figures 12 to 14 The example shows that preprocessing (assigning non-scalar data to scalar data) can save resource consumption.

[0509] It is worth noting that preprocessing sound samples or images by the adapter simplifies the implementation. By being specific to sound samples or images, the adapter eliminates the complexity of the predictor. The predictor has to deal with scalars instead of sound or images.

[0510] Figure 14 shows data aggregation, and applying such aggregation may also contribute to resource savings. As mentioned above ( Figures 4 to 5 ), the multivariate time series {{X}} is a group of univariate time series {X}i. Subgroups (of univariate time series) can be identified and aggregated before performing the method. To perform the method, the computer can use the aggregated time series at least in part.

[0511] The figure shows an example where three univariate time series {X}1, {X2}, {X3} belong to {{X}}. The three univariate time series {X}1, {X2}, {X3} are aggregated into a univariate time series {X}'. Then, {X}' will become part of the multivariate time series for the method execution.

[0512] In the example, {X}1 and {X}2 represent the position (of a vehicle), and {X}3 represents vibration.

[0513] To illustrate this by way of example only, a motor (or vehicle) (or vibrating vehicle) that is forced to stop makes some characteristic noises and may heat up. A domain expert can define appropriate rules such that {X}' changes (e.g., starting from t6, from 0 to 1).

[0514] In addition, data aggregation (i.e., the reduction in complexity from N to 1, from N to 2, from N to "small numbers") is described in the following papers: "Latent Variable Models for Dimensionality Reduction" by Zhihua Zhang and Michael I. Jordan, Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, PMLR 5:655 - 662, 2009; and "Generalized Autoencoders: A Neural Network Framework for Dimensionality Reduction" by W. Wang, Y. Huang, Y. Wang, and L. Wang, Workshop of the IEEE Conference on Computer Vision and Pattern Recognition 2014, 2014, pp. 496 - 503.

[0515] For Figures 12 to 14 the scenario described in, the adapter 280 can be implemented by a neural network that has been previously trained, possibly under supervision (by a human expert). For example, the adapter 280 can be trained with annotated images showing unevenness, annotated sound sequences, etc. The adapter 280 can also be trained in an unsupervised manner to recognize an abstract representation of an image (or sound). This ensures the maximization of the information content of the image. Then, the image (or sound) sequence can be reduced to a scalar sequence through this process.

[0516] To identify regions within an image (see Figure 13 ), a technician can apply autonomous driving techniques (e.g., an automotive computer differentiating traffic signs and pedestrians). Statistical data may play a role in training the network, applying rules, etc.

[0517] Similar expert involvement can be applied to the selection of subgroups to be aggregated (see Figure 14 ). Selecting subgroups (time series) effectively divides industrial machines into groups, and the selection can take into account components (see 110, 120, 130).

[0518] The univariate time series provided by the adapter (e.g., {X}', {X}p', {X}p' in FIGS. 16 to 18) can optionally be semantically related. This semantics is typically invoked using metaphorical terms such as "health index" or "operating status". In the example of a motor, {X}' is obtained by aggregation and is more precisely the "motor problem index", which represents the potential failure of a machine component (i.e., a specific motor). In this sense, {X}' has not shown the key parameters, but can enable the computer to find the (root) cause faster (see the scenarios of vehicles and bearings). In other words, {X}' may display the CR with relatively low precision, but the precision is sufficient for the computer to further investigate the machine without having to investigate the machine components. {X}' allows for prioritization, thus saving computing resources.

[0519] General-purpose computer

[0520] Figure 15 An example of a general-purpose computer device that can be used with the technology described herein is shown. Figure 15 is a diagram showing examples of a general-purpose computer device 900 and a general-purpose mobile computer device 950 that can be used with the technology described herein. The computing device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The general-purpose computer device 900 can correspond to Figure 2 the computer system 200. The computing device 950 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, in-vehicle computers of a driving assistance system or a vehicle, and other similar computing devices. For example, the computing device 950 can be used by a user (e.g., an operator of a blast furnace) as a front end to interact with the computing device 900. The components, connections and relationships of the components, and the functions of the components shown herein are only intended to be exemplary and are not intended to limit the implementation of the inventions described and / or claimed in this document.

[0521] The computing device 900 includes a processor 902, a memory 904, a storage device 906, a high-speed interface 908 connected to the memory 904 and a high-speed expansion port 910, and a low-speed interface 912 connected to a low-speed bus 914 and the storage device 906. Each of the components 902, 904, 906, 908, 910, and 912 is interconnected using various buses and can be mounted on a common motherboard or in other suitable manners. The processor 902 can process instructions for execution within the computing device 900, including instructions stored in the memory 904 or on the storage device 906, to display graphical information of a GUI on an external input / output device, such as a display 916 connected to the high-speed interface 908. In other implementations, multiple processors and / or multiple buses, as well as multiple memories and multiple types of memories, may be used as appropriate. Additionally, multiple computing devices 900 can be connected, with each device providing a necessary part of the operation (e.g., as a server group, a blade server group, or a multi-processor system).

[0522] The memory 904 stores information within the computing device 900. In one implementation, the memory 904 is one or more volatile storage units. In another implementation, the memory 904 is one or more non-volatile storage units. The memory 904 can also be other forms of computer-readable media, such as magnetic disks or optical discs.

[0523] The storage device 906 is capable of providing large-capacity storage for the computing device 900. In one implementation, the storage device 906 can be or include a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disc device, or a tape device, a flash memory, or other similar solid-state memory devices, or an array of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly embodied in an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods, such as the above methods. The information carrier is a computer or machine-readable medium, such as the memory 904, the storage device 906, or the memory on the processor 902.

[0524] The high-speed controller 908 manages the bandwidth-intensive operations of the computing device 900, while the low-speed controller 912 manages the less bandwidth-intensive operations. This allocation of functionality is merely exemplary. In one implementation, the high-speed controller 908 is coupled to the memory 904, the display 916 (e.g., via a graphics processor or accelerator), and is coupled to the high-speed expansion port 910, which can accept various expansion cards (not shown). In this implementation, the low-speed controller 912 is coupled to the storage device 906 and the low-speed expansion port 914. The low-speed expansion port can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), and the low-speed expansion port can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a network device such as a switch or a router, e.g., via a network adapter.

[0525] The computing device 900 can be implemented in many different forms, as shown. For example, the computing device 900 can be implemented as a standard server 920, or implemented multiple times in a group of such servers. The computing device 900 can also be implemented as part of a rack server system 924. Additionally, the computing device 900 can be implemented in a personal computer such as a laptop computer 922. Alternatively, components from the computing device 900 can be combined with other components in a mobile device (not shown) such as the device 950. Each of such devices can include one or more computing devices among the computing devices 900, 950, and the entire system can be composed of multiple computing devices 900, 950 communicating with each other.

[0526] The computing device 950 includes components such as a processor 952, a memory 964, an input / output device such as a display 954, a communication interface 966, and a transceiver 968. The device 950 can also be provided with a storage device, such as a microdrive or other device, to provide additional storage space. Each of the components 950, 952, 964, 954, 966, and 968 is interconnected using various buses, and multiple of these components can be mounted on a common motherboard or in other suitable manners.

[0527] The processor 952 can execute instructions within the computing device 950, including instructions stored in the memory 964. The processor can be implemented as a chipset including separate and multiple analog and digital processors. The processor can provide, for example, the coordination of other components of the device 950, such as the control of the user interface, the applications running on the device 950, and the wireless communication of the device 950.

[0528] The processor 952 can communicate with a user through a control interface 958 and a display interface 956 connected to a display 954. The display 954 can be, for example, a TFT LCD (Thin Film Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, or other suitable display technologies. The display interface 956 can include appropriate circuitry for driving the display 954 to present graphics and other information to the user. The control interface 958 can receive commands from the user and translate the commands for submission to the processor 952. Additionally, an external interface 962 can be provided to communicate with the processor 952 so that the device 950 can communicate closely with other devices. The external interface 962 can provide, for example, wired communication in some implementations, or can provide wireless communication in other implementations, and can also use multiple interfaces.

[0529] The memory 964 stores information within the computing device 950. The memory 964 can be implemented as a computer-readable medium or media, one or more volatile memory units, or one or more non-volatile memory units, or a combination thereof. An expansion memory 984 can also be provided, and the expansion memory 984 is connected to the device 950 through an expansion interface 982, which can include, for example, a SIMM (Single In-line Memory Module) card interface. Such expansion memory 984 can provide additional storage space for the device 950, or can also store application programs or other information for the device 950. Specifically, the expansion memory 984 can include instructions for executing or supplementing the above processes, and can also include security information. Thus, for example, the expansion memory 984 can act as a security module for the device 950 and can be programmed with instructions that allow for the secure use of the device 950. Additionally, security applications and additional information can be provided via the SIMM card, such as placing identification information on the SIMM card in an unbreakable manner.

[0530] The memory can include, for example, flash memory and / or NVRAM memory, as described below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as the above methods. The information carrier is a computer or machine-readable medium such as the memory 964, the expansion memory 984, or the memory on the processor 952, which can be received, for example, through the transceiver 968 or the external interface 962.

[0531] Device 950 may communicate wirelessly via communication interface 966, which may include digital signal processing circuitry when necessary. Communication interface 966 may provide communication in various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messages, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, etc. For example, such communication may be performed via radio frequency transceiver 968. Additionally, short-range communication may also be performed, such as using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, a GPS (Global Positioning System) receiver module 980 may provide additional navigation and location-related wireless data to device 950, which may be used by applications running on device 950 as needed.

[0532] Device 950 may also perform audio communication using audio codec 960, which may receive voice information from a user and convert the voice information into usable digital information. Audio codec 960 may also generate audible sounds for the user. For example, audio codec 960 may generate audible sounds for the user via a speaker, such as a speaker located in the earpiece of device 950. Such sounds may include sounds from a voice telephone call, may include recorded sounds (e.g., voice messages, music files, etc.), and may also include sounds generated by applications running on device 950.

[0533] Computing device 950 may be implemented in many different forms, as shown in the figure. For example, computing device 950 may be implemented as cellular phone 980. Computing device 950 may also be implemented as part of a smartphone 982, a personal digital assistant, or other similar mobile devices.

[0534] Various implementations of the systems and techniques described herein may be implemented in digital electronic circuitry, integrated circuits, specially designed ASICs (Application Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include being implemented in one or more computer programs capable of being executed and / or interpreted on a programmable system including at least one programmable processor, which may be special purpose or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0535] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus, and / or device (e.g., a disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives the machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0536] For interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other types of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0537] The systems and techniques described herein can be implemented in a computing device that includes a backend component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a frontend component (e.g., a client computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), and the Internet.

[0538] A computing device can include clients and servers. Clients and servers are typically remote from each other and typically interact through a communication network. The relationship of client and server arises through computer programs running on respective computers and has a client-server relationship with each other.

[0539] Numerous embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0540] In addition, the logical flows depicted in the figures do not require the particular order or sequential order shown to achieve the desired result. Moreover, additional steps may be provided, or steps may be eliminated from the flow, and other components may be added to or removed from the described system. Accordingly, other embodiments fall within the scope of the appended claims.

[0541] List of reference numerals

[0542] CORR, DEV status

[0543] DEV_CRITICAL

[0544] DEV_NON_CRITICAL

[0545] 11, 11a, 11b, 12, 31, 32 state transitions

[0546] 101 Industrial machine being observed

[0547] 102 One or more industrial machines for reference

[0548] 105 Machine controller

[0549] 190 Operator

[0550] 200 Computer

[0551] 210 Run - predictor

[0552] 220 Criticality - predictor

[0553] 230 Simulator

[0554] 280 Adapter

[0555] 281, 282, 283 Images, regions within the images

[0556] 402 Training method

[0557] 403 Method for identifying parameter state transitions

[0558] 412, 422 Method steps (training)

[0559] 413, 423, 433 Method steps (identification)

[0560] 500, 501, 502 Multivariate time series

[0561] 505 Parameter samples

[0562] 510 Range

[0563] Actions of 611, 612, 631, and 632

[0564] Factory 800

[0565] Corners 801 and 802

[0566] Regions 811 and 812

[0567] General-purpose computer 9xx with components.

Claims

1. A computer-implemented method (403) for identifying a predicted parameter state transition (11a, 31) that will occur in the future, wherein, since the likelihood of a change in the operating mode (X_mode) of an industrial machine (101) in the future is predicted to increase, the parameter state transition is a critical transition, wherein the operating mode is the technical state of the machine; wherein the computer (200) processes a multivariate time series ({{X}}), the multivariate time series ({{X}}) being a plurality of univariate time series ({X}i), wherein each univariate time series ({X}i) represents a parameter (X_i) of the industrial machine (101), and the method (403) comprises the following steps: Predicting (413) the operation of a specific industrial machine (101), by processing an operating multivariate time series ({{X}}_op), the operating multivariate time series ({{X}}_op) representing the operation of a specific industrial machine (101) during a specific operating time interval (T_op) that is currently ongoing (t_current), and by providing a future multivariate time series ({X}_ft), the future multivariate time series ({X}_ft) representing the predicted operation of a specific industrial machine (101) during a specific predicted time interval (T_ft) in the future; and Processing the future multivariate time series ({X}_ft) under the following relevant conditions to predict (423) the parameter state transition (11a, 31), (a) For a parameter represented by a univariate time series that is part of the future multivariate time series ({X}_ft), the computer determines that at least one specific parameter (X3) is predicted (DEV) to have a value that will be different from a reference value in at least one deviation segment ( / X / i), and (b) By quantifying the deviation (DEV(i)) in the deviation segment for the at least one specific parameter and by applying a predetermined rule, the computer determines that the likelihood of a change in the operating mode of the industrial machine (101) is predicted to increase.

2. The method (403) according to claim 1, wherein, the computer (200) performs the prediction (413) of the operation of a specific industrial machine by running a predictor module (210), the predictor module (210) being implemented by an item selected from the following: a simulator; a machine learning tool that has been pre-trained using an operating multivariate time series from past operations; and a computer function that applies a mathematical relationship and thereby processes data by a formula or a look-up table.

3. The method (403) according to any one of claims 1 to 2, wherein, The computer (200) performs a prediction (423) on the parameter state transition (11a, 31) through a criticality predictor module (220), and the criticality predictor module (220) is adapted to identify a deviation segment in the univariate time series by comparing the future multivariate time series ({X}_ft) with a reference multivariate time series ({X}_rf) for each univariate time series individually, such that the deviation segment thereby identifies a parameter deviation, a quantization deviation value, or a variable-specific error predicted to occur during a future segment interval.

4. The method (403) according to any one of claims 1 to 3, wherein, the computer (200) performs a prediction (423) on the parameter state transition (11a, 31) through a criticality predictor module (220), and the criticality predictor module (220) is adapted to identify the difference between the numerical values obtained by preprocessing an image or a sound sample.

5. The method (403) according to any one of claims 1 to 4, wherein, the computer (200) performs a prediction (423) on the parameter state transition (11a, 31) through a pre-trained criticality predictor module (220) to determine an increased likelihood of a change in the operating mode (X_mode) of the industrial machine, wherein the criticality predictor module (220) has been trained using historical data related to the machine mode, wherein the machine mode serves as the ground truth at the output, and the data related to the parameters and quantization deviations serves as the input data.

6. The method (403) according to any one of claims 1 to 5, wherein, the computer (200) also performs step simulation (433) through the following: Modify the representation of the at least one specific parameter (X3) to obtain a set of parameter variants (v1...v4, v1...v12), wherein, in the at least one univariate time series representing the at least one specific parameter (X3), the computer replaces the deviation segments ( / X / 1, / X / N) with replacement segments obtained from a reference or generated by an autoencoder (203) ～ X ～ 1, ～ X ～ N) For each parameter variant (vk) individually, predicting the operation of a specific industrial machine (101), and evaluating whether the at least one specific parameter (X3) still has a value different from the reference value and whether there is still a possibility of changing the operating mode (X_mode) of the industrial machine; and Selecting the parameter variant that has the least impact on the operation of a specific industrial machine (101).

7. The method (403) according to claim 6, the method (403) is also a method for action recognition, wherein, Outputting the selected parameter variant as a recommended action to the operator (190) of the industrial machine (101).

8. The method (403) according to any one of claims 6 to 7, wherein, The computer (200) limits the data processing during simulation to the parameters that are actuator parameters.

9. The method (403) according to any one of claims 1 to 8, wherein, The parameter state transition indicates that a specific industrial machine transitions to a specific operating mode of operating as an abnormal machine.

10. A computer program product which, when loaded into the memory of a computer system and executed by at least one processor of the computer system, causes the computer system to perform the steps of the computer-implemented method (403) according to any one of claims 1 to 19.

11. A computer system (200) comprising a plurality of modules (210, 220, 230) which, when executed by the computer system, perform the steps of the computer-implemented method (403) according to any one of claims 1 to 9.

12. Use of a computer system (200) according to claim 11 for differentiating parameters of a particular industrial machine (101) in order to identify a subgroup of parameters as critical parameters (CP) causing abnormal operation of the industrial machine (101).

Images