Predicting causes of abnormal behavior in industrial machinery

Abstract

The computer identifies parameter state transitions predicted to occur in the future, where the parameter state transitions are critical transitions resulting from a predicted increase in the likelihood that the operating mode of the industrial machine will change in the future, and the operating mode is the technical state of the machine. The computer processes the operational multivariate time series ({{X}}_op) by processing an operational multivariate time series ({{X}}_op) representing the operation of the specific industrial machine (101) during a specific operating time interval (T_op) currently in progress (t_current). The computer provides a future multivariate time series ({X}_ft) representing the predicted operation of the specific industrial machine (101) during a specific prediction time interval (T_ft) reaching the future. The computer predicts (423) the parameter state transitions (11a, 31) if both of the following conditions are met: (i) at least one specific parameter is predicted to have a value different from the reference value in at least one deviation segment; and (ii) the likelihood of a change in the operating mode of the industrial machine is predicted to be high.

Description

[Technical Field]

[0001] Generally, the present disclosure relates to industrial machinery, and more particularly, the present disclosure relates to a computer system, method, and computer program product for identifying parameters that exhibit deviations for a future time interval and become critical to the operation of the industrial machinery. [Background technology]

[0002] It is inevitable that individuals will eventually develop disease, at least for the next few years, but humans try to stay healthy for as long as possible and avoid surprises such as changes in their health.

[0003] To meet these objectives, understanding the causes of disease (among them the so-called "root causes") is of paramount importance, and expertise in fields such as biology, medicine, and food science helps define appropriate actions. Essentially, actions can be distinguished between proactive actions (aimed at remaining healthy) and reactive actions (aimed at re-establishing health if necessary). In both cases, actions involve monitoring the general condition and, in particular, the body's subtle signs. Reactive actions require monitoring with more specialized expertise and more advanced diagnostics.

[0004] Similar, very simplified objectives apply to technical equipment such as industrial machines. Even terms like "health" or "failure prevention" are used metaphorically: the machine should operate within a certain time interval (usually years or decades) and inevitable interruptions should be predicted.

[0005] An experienced machinist monitors machinery and knows the interactions between machine components. In many situations, the machinist knows the cause of a failure so that they can act accordingly. To give two examples, the machinist proactively keeps bearings on rotating parts well oiled and has some basic tools available for any basic repairs.

[0006] However, with the increasing complexity of machinery, the operator's knowledge may not be available in all situations for at least two reasons: First, the operator may not be able to fully monitor the machine; for example, bearings may be hidden and difficult to access with an oiler, and the characteristic noise of a dry or broken bearing may be dampened by the overall sound of the machine. Second, in scenarios contemplated in the context of an autonomous factory, the operator may not be able to respond at all.

[0007] Complexity magnifies the operator's lack of knowledge into even worse scenarios: a complex machine may fail completely if one of its components fails partially.

[0008] Maintenance (preventing breakdowns or repairing them in the event of a breakdown) is a measure to keep machines in good condition.

[0009] Thus, machine operation (including maintenance planning) tends to be data-driven (which in a sense corresponds to specialized expertise and advanced diagnostics). Machine-related information can come from a variety of sources, such as from sensors associated with the machine, from shop floor systems, from devices that measure product characteristics, etc.

[0010] Computer-implemented tools can predict the time horizon within which a particular failure can be expected, allowing operators to take steps in advance to prevent the failure (predictive maintenance).

[0011] Some of these tools offer predictions in more advanced settings: for example, predictions can involve root cause analysis (RCA) that allows operators to better identify measurements, or estimation of the so-called remaining useful life (RUL) of a machine if no maintenance is performed.

[0012] Such tools may apply machine learning techniques, and training such tools requires the availability of so-called historical data and the identification of faults in that historical data (through expert annotation or otherwise).

[0013] Patent Document 1 describes a computer system for detecting anomalies in multivariate time series, in which the computer receives the time series from a monitored device and uses a pair of neural networks. Summary of the Invention [Means for solving the problem]

[0014] The computer executes a computer-implemented method for identifying parameter state transitions predicted to occur in the future. The parameter state transitions are critical transitions resulting from a predicted increase in the likelihood that an industrial machine's operating mode will change in the future. The operating mode is a technical state of the machine. The computer processes a multivariate time series, which is a plurality of univariate time series, each representing a parameter of the industrial machine.

[0015] In a step of predicting the operation of a particular industrial machine, the computer processes an operational multivariate time series representing the operation of the particular industrial machine during a particular operational time interval currently in progress, and provides a future multivariate time series representing the predicted operation of the particular industrial machine during a particular prediction time interval extending into the future.

[0016] In processing the future multivariate time series to predict parameter state transitions, the computer evaluates two related conditions (i.e., a logical AND condition). A parameter state transition is predicted if both of the following conditions are met:

[0017] (First Condition) The computer processes the parameters represented by the univariate time series as part of a future multivariate time series. The first condition is met if the computer determines that at least one particular parameter is predicted to have a value different from the reference value in at least one deviation segment.

[0018] (Second condition) The computer quantifies the deviation in the deviation segment for at least one specific parameter. This allows the computer to apply a predetermined rule. The second condition is met when the computer determines that a change in the operating mode of the industrial machine is predicted to be highly likely.

[0019] To predict the behavior of a particular industrial machine, a computer can use a behavior predictor module. The module can be implemented by a simulator; by a machine learning tool (pre-trained using multivariate time series of behavior from past operations); or by a computer function (applying mathematical relationships, thereby processing data via formulas or lookup tables). Implementation approaches can be combined.

[0020] The computer may perform the step of predicting parameter state transitions by comparing future multivariate time series with a reference multivariate time series, separately for each univariate time series, with a significance predictor module adapted to identify deviation segments within the univariate time series, the deviation segments enabling the computer to identify parameter deviations, quantify deviation values, or indicate variate-specific errors predicted to occur during future segment intervals.

[0021] The computer may perform the step of predicting parameter state transitions by means of a significance predictor module adapted to identify differences between numerical values ​​obtained by preprocessing image or audio samples.

[0022] The computer may perform the step of predicting parameter state transitions with a significance predictor module that has been pre-trained to determine an increasing likelihood of a change in an operating mode of the industrial machine. The significance predictor module may be trained with historical data related to the machine modes, with the machine modes serving as ground truth for the output and data related to the parameters and quantified deviations serving as input data.

[0023] The computer may further perform the simulating step by: modifying a representation of at least one specific parameter to obtain a set of parameter variations; and in at least one univariate time series representing the at least one specific parameter, replacing a deviating segment with a replacement segment obtained from a reference or generated by an autoencoder. Separately for each parameter variation, the computer predicts the operation of the specific industrial machine and evaluates whether the at least one specific parameter still has a value different from the reference value and still changes the likelihood of a change in the operating mode of the industrial machine. The computer then selects the parameter variation that has the least impact on the operation of the specific industrial machine.

[0024] A computer can execute the method as a method for identifying an action, whereby the computer outputs the selected parameter variation as a recommended action to an operator of the industrial machine.

[0025] The computer can limit the data processing during the simulation to parameters that are actuator parameters.

[0026] The parameter state transition may indicate that a particular industrial machine is switching into a particular operating mode that is abnormal machine operation.

[0027] Also, a computer program product that, when loaded into the memory of a computer system and executed by at least one processor of the computer system, causes the computer system to perform the steps of the computer-implemented method.

[0028] The computer system comprises a number of modules that, when executed by the computer system, perform the steps of the computer-implemented method.

[0029] In other words, the computer system can be used to distinguish between parameters of a particular industrial machine and identify a subset of parameters that are critical parameters that cause abnormal operation of the industrial machine. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 2 illustrates a state transition graph of machine parameters with nodes and edges.

[0031] [Figure 2] FIG. 1 illustrates an industrial machine and a computer.

[0032] [Figure 3] FIG. 1 illustrates a multivariate time series representing multiple machine parameters.

[0033] [Figure 4] FIG. 10 shows further multivariate time series representing multiple machine parameters under different timing aspects.

[0034] [Figure 5] FIG. 4 illustrates the multivariate time series of FIG. 3 in a separation into a time series available for the complete time interval between the start time point and the end time point, for a sub-time interval from the start time point to the observation time point applicable to the current operation of the industrial machine, and for a sub-time interval from the prediction time point to the end time point applicable to the future operation of the industrial machine.

[0035] [Figure 6] FIG. 1 shows a multivariate time series with individual univariate time series taking into account variate dependencies, where the univariate time series are shown in Cartesian coordinates.

[0036] [Figure 7] FIG. 10 is a time diagram of the steps performed by the computer modules Performance Predictor and Importance Predictor, together with availability graphs of multivariate time series in different timing aspects.

[0037] [Figure 8] FIG. 10 shows a continuation of the timeline of further steps that may be performed by the simulator module, along with a further availability graph.

[0038] [Figure 9] This is a symbol diagram of bearings.

[0039] [Figure 10] FIG. 10 is a diagram showing a simulation of temperature with reference to the bearing of FIG. 9.

[0040] [Figure 11] FIG. 3 repeats the diagram of industrial machines and computers 200 of FIG. 2, but adds an example of the optional use of an autoencoder.

[0041] [Figure 12] FIG. 10 illustrates optional variable preprocessing by showing adapters.

[0042] [Figure 13] FIG. 10 illustrates optional variable pre-processing by representing image data.

[0043] [Figure 14] FIG. 1 illustrates optional variate preprocessing with data aggregation.

[0044] [Figure 15] FIG. 1 illustrates a general-purpose computer. DETAILED DESCRIPTION OF THE INVENTION

[0045] Parameter Status Figure 1 shows a state transition graph for parameter X_i of an industrial machine. In this overview, the specific semantic of parameter X_i (e.g., "temperature", "vibration", etc.) is not important, and therefore machine parameters are identified by the so-called variable index i. Nodes (denoted by rectangles with rounded corners) represent states, and directed edges (denoted by arrows) represent state transitions.

[0046] A parameter is associated with a value. For simplicity, the values ​​are assumed to be numeric in this specification. The values ​​can be more complex datasets such as images or audio samples, and it is contemplated that such datasets will be preprocessed into numeric values. Further details are provided in the "Multimodality and Variant Preprocessing" section at the end of this specification.

[0047] The term "deviation" is used herein for the situation where the value of a parameter differs from a reference value at a particular time period in the past, present, or future. Terms and phrases such as "deviation," "deviant," and "parameter deviates" refer to states rather than transitions. In time diagrams, deviations are also represented by the / / symbol.

[0048] A parameter (such as "X_i") may change its state from "corresponding to norm" to "deviant from norm" (see arrow 11). For convenience, the description and drawings refer to these two states by the acronyms CORR and DEV, respectively.

[0049] Parameter state transition State transitions can be defined in both directions and as used herein: The transition to state DEV indicates a change of state from CORR to DEV (arrow 11). The transition to state CORR indicates that the state changes (returns) from DEV to CORR (arrow 12).

[0050] State transitions may occur in the past, present, or future and may be referred to as "past transitions," "present transitions," and "future transitions." For simplicity of explanation, transitions should occur at a point in time, so transition durations can be ignored here.

[0051] For example, a machine part may normally rotate (state CORR) with a parameter "revs" within a range of values ​​between 1,000 and 2,000 rpm. However, if the machine part slows down to 800 rpm, the parameter "revs" transitions to state DEV (arrow 11, to DEV). The part can then transition back to CORR (arrow 12) and back again.

[0052] Typically, a criterion has a tolerance (or tolerance) for values ​​within a so-called tolerance band. The band can be given for a value or a time interval. Exceptions can therefore be defined. For example, if a criterion allows for temporary value deviations, then for at least some predetermined interval (e.g., a few minutes), the parameter remains in state CORR and does not exhibit any deviations (see an example in Figure 4).

[0053] Substates CRITICAL and NON_CRITICAL and machine modes From a theoretical point of view, it seems impossible to operate an industrial machine with all parameters remaining constant over time. If parameters change, the industrial machine will behave differently. However, not all parameter changes are relevant or decisive. Some parameter changes may even go unnoticed.

[0054] It is well known in the art to consider the operation of machines (and other technical equipment) in terms of so-called technical states. Machine states can be detected by a human observer, who can attribute a state to the machine (e.g., by observing that the machine has stopped working). In contrast, a change in the rotational speed of a machine component can be noticed, but a human observer cannot attribute a specific state. States can also be assigned through human interaction.

[0055] It is also well known that certain changes in quantity result in changes in quality. The quality of machine operation may ultimately change. Those skilled in the art know such changes as changes in operational mode. As used herein, "affecting the operational mode of a machine" is a convenient notation for increasing the likelihood (or probability) that the operational mode will change (presently) or will change (in the future).

[0056] The operating mode of the machine can also be considered as a machine state, but it can be assumed that the parameters X_i in state CORR do not affect the operating mode of the machine. However, the parameters X_i in state DEV may or may not affect the operating mode of the industrial machine.

[0057] To simplify, an operating mode (of a machine) can be characterized and simplified for example by "normal behavior" or by "abnormal behavior", with a further parameter X_mode for that state. Abnormal behavior can be defined (and detected according to predetermined criteria), among others, for example only by the following criteria: The machine does not operate as expected, for example, because it delays the processing of physical materials or because it releases substances into the environment, thereby exceeding a threshold (of acceptable emissions). The machine breaks down completely and stops working. · There is a danger or security risk to human workers. There is a risk that some components of the machine may be destroyed.

[0058] The impact can be summarized, for example, as follows: If X_i=DEV_CRITICAL, X_mode=abnormal, Otherwise, X_mode = normal.

[0059] The influence of a parameter may also be related to other parameters, e.g. If X_i=DEV_CRITICAL, and If X_k=high, X_mode=abnormal, Otherwise, X_mode = normal.

[0060] The machine's operating mode can be detected by a human while the machine is operating (i.e., in the present) or by looking at the data (i.e., the machine has operated in the past). The human detector does not necessarily have to be the machine's operator (see operator 190). For example, the processing delays mentioned above can be detected in a follow-up production stage, and emissions can be detected by periodic inspections. Complete failures or steps are likely to be detectable by a nearby human. Security risks, etc. may also become apparent during periodic inspections. Those skilled in the art are aware of further opportunities.

[0061] Since the operating modes are easily detectable, data about the modes (as historical data) is also available for training purposes.

[0062] However, in many situations, such simple causal relationships are not known. Furthermore, influences can be expected to arise from multiple parameters (not only from X_i, but also from von X_k and other parameters). Nevertheless, for simplicity, this specification continues to discuss the states and transitions of parameter X_i.

[0063] Figure 1 shows transitions within substates by arrows 31 (to DEV_CRITICAL) and 33 (to DEV_NON_CRITICAL). Transitions between substates can occur in the past, present, or future (i.e., as past, present, or future transitions). State transitions are a concept introduced here for ease of explanation. Thus, two transitions can occur simultaneously.

[0064] Therefore, the transitions (here for example parameters X_i) are contemplated below.

[0065] CORR to DEV_NON_CRITICAL (arrow 11b) At certain points, parameters X_i may begin to deviate, but without adverse effects. X_i may change DEV_CRITICAL in some circumstances, so such transitions should be monitored.

[0066] DEV_CRITICAL to DEV_NON_CRITICAL (arrow 32) At a particular point in time, parameter X_i may be DEV_CRITICAL, but deviations are permitted due to machine changes (i.e., changes in other parameters at that time) until the adverse effects on the machine's operating mode cease. While such a transition to DEV_NON_CRITICAL is desired, in many situations it is undesirable since state DEV remains.

[0067] DEV_CRITICAL to CORR, DEV_NON_CRITICAL to CORR (arrow 12) At a certain point in time, the parameters X_i can return to values ​​corresponding to the reference, and such a transition is highly desirable.

[0068] CORR to DEV_CRITICAL (arrow 11a, drawn in bold) At a certain point in time, the parameters X_i may start to deviate, which will have a negative effect on the operating mode (of the machine) from that point onwards.

[0069] DEV_NON_CRITICAL to DEV_CRITICAL (arrow 31, drawn in bold) At a certain point, a parameter X_i may become critical, but due to changes in the machine (i.e., changes in other parameters), the deviation is no longer acceptable. There will be a negative impact on the operating mode.

[0070] The two transitions to DEV_CRITICAL (arrows 31 and 11a) are critical transitions due to their impact on the operating mode (X_mode).

[0071] The attribute "critical" here has the notation "determinative" or "relevant" (e.g., the operating mode changes) and "important" (e.g., the machine operator may be alerted). In everyday usage, the word "critical" has a negative connotation, meaning that something needs to be avoided, but the impact on the operating mode may be negative or positive. Solely for ease of explanation, examples with a negative impact are preferred herein over examples with a positive impact.

[0072] The computer can be trained to determine the effects on the operating mode (negative effects as described herein, but also theoretically positive effects).

[0073] phenomenon A transition to DEV_CRITICAL can be considered an event that should be (proactively) avoided. To emphasize this aspect, the diagram depicts arrows 11a and 31 in bold. If avoidance is not possible, such an event should at least be predicted.

[0074] If such an event does indeed occur, transitions t to CORR (arrow 12) and to DEV_NON_CRITICAL (arrow 32) should be executed (away from DEV_CRITICAL). Actions to enforce or prevent parameter state transitions

[0075] Figure 1 also indicates actions by dashed lines near the transition arrows. Actions can be distinguished according to the transition. Action 611 prevents the transition of parameter X_i from state CORR to state DEV. Action 611a prevents the transition of parameter X_i from state CORR to state DEV_CRITICAL. Action 611b prevents the transition of parameter X_i from CORR to DEV_NON_CRITICAL. Action 631 prevents the transition of parameter X_i from DEV_NON_CRITICAL to DEV_CRITICAL. Action 612 supports the transition of parameter X_i from DEV back to CORR. Action 632 supports the transition where parameter X_i becomes DEV_NON_CRITICAL.

[0076] To simplify, "state actions" prevent state transitions, and "transition actions" support transitions. From a slightly different perspective, state actions are proactive actions, while transition actions are retroactive actions.

[0077] However, Figure 1 shows the theory, and actions must be implemented in practice. In general, actions may include maintenance (with or without stopping the machine before the failure), repair (usually with a shutdown because the failure has already occurred), calibration of sensors or actuators, replacement of parts, filling with lubricant (e.g., oil), supplying energy (e.g., fuel, rechargeable battery), changing non-X_i parameters (e.g., taking an alternative route), and other measures.

[0078] The exact terminology does not matter, for example, whether action 632 is called "repair."

[0079] Industrial machines have actuators (e.g., devices that change oil in bearings, apply cooling, limit the load that a robot or vehicle must carry, maintain the temperature of a component at a predetermined value, etc.) Actuators are machine components that can be controlled by a machine operator or by a machine controller (i.e., by a computer belonging to the machine, see controller 105 in FIG. 2).

[0080] It is useful to distinguish between parameters X_i as actuator parameters and non-actuator parameters. If X_i is an actuator parameter, the actuator can perform an action, thereby triggering a transition (see arrows 12, 32). If X_i is an actuator parameter, the actuator can perform an action, thereby avoiding the transition (see actions 611, 611a). For example, the actuator can be a heater that automatically maintains the temperature X_i within a predetermined range. If X_i is a non-actuator parameter, the actuator can act on a different parameter (which influences X_i). For example, X_i can be parameters describing the quantity and quality of the product that the machine produces, representing the substances that the machine releases into the environment, X_i can be measurement data such as temperature, vibration, etc.

[0081] We return to the actuator / non-actuator topic below in this document, as the distinction can help speed up calculations (or at least save computational resources).

[0082] The distinction into (non-)actuator parameters is substantially independent from the above distinction into state actions and transition actions. One and the same action can even be distinguished to be a state action or a transition action. Therefore, this specification describes only examples.

[0083] The action 612 supporting the transition to CORR can be implemented (for parameters X_i) by the following (dashed line): Modifying Xi (the parameters become actuator parameters by actuators that directly modify Xi), or Modifying X_delta (by means of the actuators the correcting parameter X_delta changes the parameters X_i).

[0084] Actions 632 supporting the transition from DEV_CRITICAL to DEV_NON_CRITICAL can be implemented by modifying other parameters (see example with logical AND).

[0085] Enforcing transitions requires knowledge of parameter interdependencies. However, it is possible to learn such dependencies (e.g., by training a machine learning tool). Modifying X_delta may have a different effect than modifying X_beta. Such effects can be quantified (e.g., in the time it takes to return X_i to CORR). In that sense, Figure 1 also illustrates various potential triggers for state transitions. (Various triggers can be identified through simulation, as described at the end of this specification under "Causality").

[0086] Importance Figure 1 shows that the state transition of a single parameter X_i to DEV_CRITICAL will cause the machine to operate abnormally (X_mode=abnormal). However, two or more parameters simultaneously in state DEV can cause this. If X_i=DEV, and If X_k=DEV X_mode=abnormal.

[0087] For simplicity of explanation, it is assumed herein that abnormal operation of the machine can be triggered by a single parameter transitioning to state DEV.

[0088] The combination of multiple parameters in state DEV can also be the reason for the machine to behave abnormally.

[0089] A computer-implemented function for detecting the transition of a single parameter to DEV_CRITICAL (or alternatively detecting one or more parameters in DEV that cause X_mode to become abnormal) is the function of a severity detector.

[0090] The severity criteria are time independent, so severity can be detected for the past, present, and future. Severity detection involves the logical steps of detecting a deviation (state DEV) and evaluating whether the deviation is critical (DEV_CRITICAL) or not (DEV_NON_CRITICAL).

[0091] As mentioned above, historical data on machine modes is available and machine learning tools can be trained to detect the effects on operational modes. The training set (with historical data) includes: Machine modes as ground truth, and · Data on parameters, quantified deviations of at least one specific parameter (e.g., within deviation segments).

[0092] However, it is not required to use machine learning tools to detect the effects.

[0093] Other perspectives As a side note, the principles of Figure 1 may be applied to beneficial deviations that have a positive impact on the operating mode, and the condition may be named BENEFICIAL rather than CRITICAL. As noted above, the term "critical" is generally used for any impact (or increased likelihood of a mode change), whether negative or positive. Alternatively, condition CORR need not be associated with normality; condition CORR may also be associated with failure (see Figures 9-10 for examples).

[0094] The shorthand notation "critical parameter" or "CP" represents a parameter that is in the state DEV_CRITICAL. The shorthand notation "parameter becomes critical parameter" represents the transition as indicated by the two bold arrows 11a and 31 (an undesirable event).

[0095] Considering the future, the phrase "parameter becomes critical" refers to transitions (arrows 11a and 31) that occur in the future. · At present, the state is CORR, and the transition from CORR to DEV_CRITICAL (arrow 11a) occurs in the future. · At present, the state is DEV_NON_CRITICAL, and a transition to DEV_CRITICAL (arrow 31) occurs in the future.

[0096] Other actions can be explained accordingly.

[0097] Improvements The state transition graph for parameter X_i in Figure 1 can be improved by taking repetition into account. Within a given time interval, a transition can occur multiple times. A sequence such as CORR to DEV, DEV to CORR, CORR to DEV, etc. can serve as an example. If such a sequence is observed without a specific interval, the transition back to CORR can be ignored so that parameter X_i maintains its state DEV (potentially called "consistently deviating"). To take an illustrative example, a transition may involve microcracks in some machine parts, and the accumulation of such microcracks over time can lead to failure (X_mode = abnormal).

[0098] Past, present, and future state changes This specification describes the progression of time in terms of successive absolute points in time "t_" as well as in terms of a relative distinction into "past," "present," and "future" times. At "current time," the industrial machine is operating, and at "current time," t_current, data is available to the computer so that the computer can begin executing the steps of a computer-implemented method. The time it takes to communicate the data from the machine to the computer can be ignored, as t_current marks the "present" and divides the time between the "past" and the "future."

[0099] This specification may refer to the future by terms such as "projection" or "prediction," which are synonyms.

[0100] As explained in more detail below, the computer represents the machine parameters by a time series {{X}}, where parameter X_i is one of the parameters (or "variates"), i=1 to N.

[0101] For example, today, when a machine slows down (begins to exhibit abnormal behavior), an experienced operator can immediately see a bearing failure as the reason for such an abnormality (CORR to DEV_CRITICAL, arrow 11a). However, modern industrial machines are complex and exhibit relatively high inter-parameter dependencies.

[0102] The complexity rises further. Currently, to identify a parameter as being in state DEV, the computer only needs to compare the parameter value with a reference value, where the parameter and the reference usually correspond in type (e.g., both "rotation" values). The computer does not need to identify when the transition to DEV occurred. (However, this is suitable for identifying tDEV, see Figure 4.) Currently, to identify a parameter as being in the DEV_CRITICAL state, the computer may need to take into account the co-parameters of the machine (such as X_k), whether the co-parameters deviate or not. Currently, more data processing is required to identify the reason (or "cause") why a parameter is DEV (or even DEV_CRITICAL).

[0103] These complexity topics are even more acute for forecasting into the future, to identify a parameter that will be in state DEV, identify that parameter will also be in state DEV_CRITICAL, and identify the reasons for that future occurrence. [Example]

[0104] In an illustrative example of this complexity, the present specification explores the movement of a vehicle within a factory, urging the vehicle over rotating parts (of a machine) (see FIG. 6). For a particular path (i.e., parameter "path"), the computer identifies "vibration" as a parameter that will become DEV_CRITICAL if the vehicle continues on that path. In this example, the vehicle is carrying a load, so severe vibration becomes critical.

[0105] The presence or absence of a load is a (binary) operating mode parameter. This parameter "load" can be affected by severe vibrations. When the vehicle is loaded (i.e. carrying a load), the effect can be negative. When the vehicle is moving without a load, the effect can be negligible.

[0106] In other words, the deviation parameter "vibration" has a negative impact on other parameters, such as the parameter "load". The destruction risk mentioned above applies here. In plain English, the load may simply fall off when the vehicle swings too much. Operators usually try to avoid that.

[0107] If we let the vehicle proceed without a load ("Vibration" becomes NON_CRITICAL, arrow 32), the vehicle will endure severe vibrations since it cannot lose its load. The parameter "Load" no longer contributes to the criticality (see statement with logical AND). However, such an approach may not be suitable for industrial environments, or at least not always.

[0108] To avoid a transition to DEV in general (and more specifically to DEV_CRITICAL), the computer proposes certain actions, namely: taking a different route (modifying the parameter "route", action 612); or performing certain actions triggers a state transition to CORR, when the loaded vehicle is already experiencing severe vibrations, action 612); or performing certain actions maintains the state CORR (if the vibrations are still weak, action 611).

[0109] Next, the present specification extends the example to cause identification, taking into account the parameter "surface": the floor may be uneven in certain areas, and the alternative action requires repairing the floor.

[0110] But again, note that computers are largely semantics agnostic: they don't care if a vehicle is carrying a load, if a vehicle is shaking somewhere in a factory, or if the floor can be repaired. Computers process data.

[0111] Likelihood As explained, a parameter in state DEV_CRITICAL is associated with abnormal machine operation, but the computer needs to distinguish this parameter from many other parameters. As used herein, identifying a particular parameter as DEV_CRITICAL is rather a determination that this parameter is more likely to be (or become) DEV_CRITICAL than other parameters. In other words, the machine's operational mode may change or remain the same, but a critical transition (to DEV_CRITICAL) increases the likelihood (or statistical probability) that the operational mode will change.

[0112] The computer executes the method relatively quickly to identify critical parameters etc. as early as possible. Identifying critical parameters as accurately as possible with relatively high resource usage (computer activity and software complexity); Finding critical parameters as quickly as possible with an accuracy that allows the operator to correct the parameters (or repair / replace machine components) There is a compromise between.

[0113] Optionally, the computer identifies actions that, if taken, will reduce the likelihood that the parameter deviation will become critical in the future.

[0114] Taking likelihood into account is even more important for predicting future (undesirable) events (as opposed to establishing that an event has already occurred in the past). For simplicity, this specification assumes that the predicted events actually occur (with no action taken) and, optionally, that the identified actions actually fit the problem.

[0115] Overview of industrial machinery and parameters FIG. 2 illustrates industrial machines 101 and 102 and shows computer 200. Machine 101 (and machine 102, depending on the implementation) is controlled by machine controller 105. The figure shows computer 200 separate from controller 105. From a high-level perspective, both controller 105 and computer 200 are computers in the sense that they have processors, memory, data storage, etc. Computer 200 performs computer-implemented methods (e.g., method 403, steps of FIGS. 7-8). In implementations, computer 200 can be implemented as part of controller 105.

[0116] Machine 101 is an industrial machine in operation, machine 101 is also a machine under observation. In simplified terms, machine 101 does not operate or even function as expected, and an operator seeks a way to make machine 101 resume normal operation. Looking at the machine in the current time, the following scenarios can be distinguished:

[0117] (Scenario PAST) In machine 101, a particular parameter may have been in the DEV_CRITICAL state in the past. The state may currently remain DEV_CRITICAL. Computer 200 can assist operator 190 in identifying this particular parameter and the reason it is DEV_CRITICAL. Computer 200 can also identify parameters to modify (see actuator vs. non-actuator distinction above). Such scenarios are relevant for descriptive and diagnostic analysis.

[0118] (Scenario FUTURE-EVENT) In machine 101, a specific parameter will go into state DEV_CRITICAL (within a foreseeable time interval in the future) and computer 200 already (presently) helps operator 190 identify this specific parameter. Optionally, computer identifies the reasons why it is DEV_CRITICAL, or at least the reasons why it is DEV. Computer 200 can also identify parameters to modify (see actuator / non-actuator distinction above). Such a scenario relates to predictive analysis.

[0119] (Scenario FUTURE-ACTION) In machine 101, a specific parameter may enter the state DEV_CRITICAL (within a predictable time interval in the future), and computer 200 assists operator 190 in identifying this specific parameter (in the present), but the computer also identifies one or more actions that will prevent this specific parameter from transitioning to DEV_CRITICAL. Such a scenario relates to prescriptive analysis. In other words, the actions are event avoidance actions.

[0120] Taking an action does not necessarily mean stopping the machine from operating: the operator can schedule the action (before the event is expected to occur).

[0121] In the vehicle example, the computer predicts that the parameter "vibration" will become DEV_CRITICAL and suggests actions to address the cause by taking a different path or repairing the floor. Note that the parameter "vibration" is a non-actuator parameter; however, taking a different path affects the actuators that steer the vehicle (path here is an actuator parameter).

[0122] Experts in the mechanical maintenance community have explored the analysis in various papers, for example: ·Sergei Nikolaev et al., “Hybrid Data-Driven and Physics-Based Modelling for Prescriptive Maintenance of Gas-Turbine Power Plant” DOI:10.1007 / 978-3-030-42250-9_36. ·Tanja Nemeth et al., "PriMa-X: A reference model for realizing prescriptive maintenance and assessing its maturity enhanced by machine learning" Procedia CIRP 72 (2018) 1039-1044.

[0123] This document focuses on scenarios such as prescriptive analytics and prescriptive maintenance, in other words, FUTURE-EVENT and FUTURE-ACTION. Therefore, this document uses terms that refer to the future. For example, a computer "predicts a state (or transition)" and a computer "anticipates a behavior."

[0124] However, the computer may process data from the past to obtain results regarding a FUTURE-EVENT or FUTURE-ACTION. In other words, the computer may perform descriptive and diagnostic analysis as a basis for predictive analysis.

[0125] From a high-level perspective, it is also useful to view FIG. 7: a computer 200 executes a computer-implemented method (reference numeral 403 in FIG. 7) for distinguishing parameters of a particular industrial machine 101 while identifying parameters that will become critical parameters CP in the future.

[0126] At the current time, computer 200 identifies a parameter that is (or will become) a CP for operator 190, as indicated by user interface 290. The computer may identify additional critical parameters, but for simplicity, the singular form ("critical parameter CP") is used herein. When a parameter is indicated by interface 290, it does not yet have to be a CP.

[0127] As already explained, CP is a parameter that indicates a deviation that is critical. As explained above, there are two cascade conditions (DEV and CRITICAL, see Figure 1).

[0128] It therefore seems appropriate to remove the deviation (see proceeding along arrow 12 in FIG. 1). The operator may simply readjust the parameters or let the electronics of the machine controller 105 (see FIG. 2) do this. After correction, the parameters are again in state CORR. As explained above, such a straightforward approach applies only to actuator parameters.

[0129] This distinction has practical applications for the performance of computer 200. In both FUTURE scenarios (predicting an undesirable event or proposing an action), there is a "time constraint" for the computer to provide results (i.e., identify CPs and propose actions for parameter modification, etc.) before the event occurs (see arrows 11a and 31 in FIG. 1) and / or before the action can be taken.

[0130] Therefore, optionally, the computer limits its calculations (or more generally data processing) to parameters that are actuator parameters, and actions are expected to be more promising (more likely to succeed) for such actuator parameters (this does not exclude the possibility that the computer also processes non-actuator parameters).

[0131] The industrial machine 101 and the computer 200 operate substantially simultaneously (at least in current and future scenarios). In other words, the operating time of the machine 101 may substantially correspond to the runtime of the computer 200, so that the critical parameters CP become visible early enough for the operator 190 to apply measures in a timely manner (before an undesirable event occurs).

[0132] Reference Machine FIG. 2 shows machines 101 / 102 performing different roles. The industrial machine 101 is a machine for which certain parameters must be identified to become (in the future) critical parameters CP, actions can be applied, etc. The industrial machine 102 (reference machine, reference role) provides (or continues to provide) reference data related to the operation of the machine 101. Providing criteria are conditions for predicting states CORR, DEV and predicting state transitions to DEV and CORR.

[0133] It may also be advantageous to have a reference to detect DEV_CRITICAL (or to distinguish between NON_CRITICAL and CRITICAL in general). The reference may be linked to past machine failures. This is convenient but not required, and there are situations where (past) machine failures may not be available, or at least not for certain parameters.

[0134] In the vehicle example, the load-dropping fault may simply not be available yet. The vehicle may have been introduced very recently, so fault data is not available (at least for that type of fault). Nevertheless, the computer can detect the parameter "vibration" as a possible CP based on a prediction into the future.

[0135] computer Briefly, the computer 200 processes data representing the parameters (at least some of them). Conveniently, the data is available as a multivariate time series {{X}} (reference numeral 500 in the time diagram of FIG. 3). A person skilled in the art obtains such data, for example, by communicatively coupling the machine 101 and the computer 200, and so this specification will not go into such details.

[0136] The computer 200 predicts whether (and when) a particular parameter will become a critical parameter CP, and the computer can identify actions to prevent the parameter from becoming critical.

[0137] Time-sensitive data provenance As used herein, it is convenient to distinguish between multivariate time series {{X}} according to the time they become available (to a computer) and for how long they are relevant (to be processed). In this specification, we use the term "timing aspects" and the following acronyms: ·{{X}}_rf(reference), ·{{X}}_op(action), ·{{X}}_ft(future), {{X}}_ft_CP (future with critical parameters), and ·{{X}}_ft_action (future with action).

[0138] Reference time series The reference time series {{X}}_rf represents a multivariate time series with reference data. As in Figure 2, {{X}}_rf can be obtained from a machine 102, and the reference data can have different purposes. The reference data may provide data representative of the operation of the machine 101 in a normal operating mode, so that the reference data can be used to determine which parameters are (or will be) in state DEV. In other words, the reference data may be data representative of the ideal operation of the machine. · The reference data may include annotations by human experts, such as to identify situations (including states DEV_CRITICAL, DEV_NON_CRITICAL). The reference data may provide a replacement segment (of the time series to be processed) that is used in optional steps of the method.

[0139] Those skilled in the art can obtain reference data such as {{X}}_rf in various implementations. Machine 102 can be considered a physical machine or an equivalent implemented in other ways, such as by a computer (e.g., by computer 200 or by a different computer).

[0140] In a first implementation scenario, machines 101 and 102 are the same physical machine operating in different roles at different times. The machine (acting as 102) provides {{X}}_rf as historical data. Typically, there is a collection of such data.

[0141] In a second implementation scenario, machines 101 and 102 are physically separate machines that belong to a fleet of similar machines.

[0142] In a third implementation scenario, machine 102 is a computer-virtualized machine. Those skilled in the art are familiar with the concept of having a digital twin (machine 101, the original, machine 102, the twin). The digital twin provides {{X}}_rf.

[0143] In a fourth implementation scenario, the computer 200 uses an autoencoder module, the details of which are described in connection with FIG.

[0144] In a fifth implementation scenario, computer 200 implements predetermined rules. For example, the computer can detect that a univariate time series has begun to deviate (i.e., has a deviating segment) when a variable value (such as temperature) exceeds a predetermined threshold, and the computer can identify the end of the deviance when the variable value falls below the threshold. Those skilled in the art can otherwise apply such and other rules (e.g., different thresholds to reflect hysteresis).

[0145] The reference time series {{X}}_rf may, for example, have annotations that some parameter values ​​caused a machine to fail in the past, but such annotated reference data may not be available. In the vehicle example, the reference data may point to non-standard vibrations, but not to a vehicle failure.

[0146] Operation timeline The operation time series {{X}}_op represents a multivariate time series with data representing the operation of the machine 101 during a particular operation time interval (t_start to t_current) that is currently in progress. The time point t_start should be chosen so that the values ​​in {{X}}_op still affect the operation of the machine.

[0147] {{X}}_op is obtained as measurement data from sensors, by collecting metadata, etc. (e.g., from worksite applications). As mentioned above, {{X}}_rf can have annotations, but for {{X}}_op, there is usually no time for an expert to annotate them. Over time (e.g., in a new T_WINDOW), {{X}}_op can become {{X}}_rf.

[0148] Future Timeline The future time series {{X}}_ft represents a multivariate time series with data representing the future behavior of the machine 101 (during T_ft, from t_predict to t_end). {{X}}_ft cannot come from the machine 101 / 102, but from the computer 200 (from the behavior predictor 210).

[0149] The future time series {{X}}_ft_CP represents a multivariate time series having data representing the future operation of the machine 101, but with the additional identification of CP. This time series is applicable from t_predict to t_end, but becomes available at t_result, which follows t_predict with a delay caused by executing the method. Describing CP as part of a multivariate time series is advantageous because the notation identifies the point in time at which a parameter deviation becomes critical (occurring within the interval T_event (see arrows 11a and 31 in FIG. 1)). In the vehicle example, the parameter "vibration" (i.e., {X}3) would be the CP.

[0150] {{X}}_ft_CP may contain further information such as the duration the parameter is expected to remain critical (see T_event in Figure 7 for occurrence, but some predictions may even estimate the duration of DEV_CRITICAL).

[0151] The action time series {{X}}_ft_action represents a multivariate time series with instructions for possible actions to mitigate the impact of (future) critical parameter deviations or prevent critical deviations from occurring. The actions are already considered in Figure 1 for state and transition differences, taking into account (non-)actuator parameters, etc. The instructions can take such differences into account.

[0152] In many use cases, the action involves modifying a parameter (CP if it is an actuator parameter, or another parameter to modify if the CP is a non-actuator parameter). In other words, a parameter is no longer critical when it is modified (see arrows 12, 32 in Figure 1). Again, a time series notation is advantageous as it allows action identification to be advised relative to a specific point in time (t_action). Considering the human health metaphor introduced above, such actions correspond to proactive activity.

[0153] As described below in connection with simulation, {{X}}_ft_action may be available in various variations, with different variations capable of identifying different actions, allowing the simulator to select the appropriate action to take (from multiple possible actions).

[0154] Modules in a computer FIG. 2 simply refers to computer modules 210 / 220 / 230 involved in performing the computer-implemented method. The motion predictor 210 receives {{X}}_op and provides {{X}}_ft. Importance predictor 220 takes {{X}}_ft and {{X}}_rf and provides {{X}}_ft_CP). This module performs importance detection for the future. Simulator 230 is an optional module that obtains {{X}}_ft_CP and provides {{X}}_ft_action). Optionally, simulator 230 can also receive {{X}}_rf.

[0155] User interface 290 is a module for presenting {{X}}_ft_CP (or simply presenting CP) and / or {{X}}_ft_action to operator 190. In an embodiment, module 290 may be associated with controller 105 such that the action may become a control signal for operation of machine 100.

[0156] The receiving / getting and providing activities of modules 210 and 220 are applicable at the so-called runtime of these modules, respectively, in steps 413 and 423 of the computer-implemented method (see FIG. 7). The simulator 230 is described in more detail with reference to FIG.

[0157] As used herein, the term "receive" means that data enters a computer, and "obtain" means that data is generated by a computer through processing. For example, the importance predictor 220 obtains {{X}}_ft from the behavior predictor 210.

[0158] Although the notation of multivariate time series {{X}} suggests processing many univariate time series {X}i, computer 200 need not process all the data available from machine 101 / 102. One skilled in the art can filter out some of the data.

[0159] Module Implementation Each of the modules can be implemented using various approaches, which in principle are independent of each other. This document will briefly introduce some approaches, some of which will be described in more detail below.

[0160] The motion predictor 210 can be implemented by a variety of approaches. The motion predictor 210 may be implemented by a simulator (but not the simulator 230). Such a simulator may be available in a machine controller, such as the controller 105 (FIG. 2). The motion predictor 210 can be implemented, for example, by a machine learning (ML) tool that has been pre-trained by processing with a reference time series {{X}}_rf. In case of training, {{X}}_rf can be obtained from a number of {{X}}_op obtained from previous operations, in other words, the training can be based on processing of historical data. A prominent example of an ML tool is a neural network. The motion predictor 210 can use a so-called autoencoder, which will be explained in more detail below. The operation predictor 210 can also be implemented by a computer function that applies mathematical relationships, thereby processing data by means of formulas, look-up tables, etc. For example, an industrial machine may have a water boiler as a component. A future temperature value can be calculated based on the current value {temperature}_op and the power of the heater element. Alternatively, a look-up table citing past observed values ​​remains applicable.

[0161] The motion predictor 210 may be implemented by a combination of these implementations, and those skilled in the art may also apply other techniques.

[0162] The importance predictor 220 can be implemented by various approaches, and two logical steps for detecting importance have already been described.

[0163] To detect the state DEV, the importance predictor 220 can be implemented, for example, to do the following: The significance predictor 220 can identify deviant segments in the time series by comparing {{X}}_ft with {{X}_rf, see FIG. 4. The importance predictor 220 can compare the data obtained by pre-processing. For example, {X}i can contain images (or audio sequences), and {X}i_ft and {{X}_rf can be image classifications to be compared (an example would be the mentioned "bumpy floor").

[0164] The importance predictor 220 can replace deviant segments (in a time series) with corresponding segments from a reference time series. Such replacement (detailed below) can approximate the actual time series to its reference. In simple terms, the difference between an unmodified time series (e.g., original _op) and a modified time series (i.e., with replacement) can be an indication of importance.

[0165] Although FIG. 2 illustrates simulator 230 with a single box, in implementation it may be a module that invokes the functionality of behavior predictor 210 or importance predictor 220 .

[0166] For example, as described in more detail below, simulator 230 may obtain information that one or more parameters will transition to DEV in {{X}}_ft (e.g., the parameter value exceeds a threshold). Simulator 230 also obtains information that one of the DEV parameters will transition to DEV_CRITICAL in {{X}}_ft_CP. Simulator 230 may then replace the deviations (e.g., in an implementation, replace only some deviation segments with reference data) and have behavior predictor 210 and severity predictor 220 provide {{X}}_ft_CP and {{X}}_ft_CP again. Simulator 230 may repeat the replacement for different variations. Some of the variations may result in the parameters entering the CORR state (or DEV_NON_CRITICAL). Replacing the deviations with reference data corresponds to modifying the parameters. After multiple simulations, simulator 230 may output action recommendations.

[0167] In the vehicle example, the simulator 230 obtains {{X}}_ft_CP, which indicates that {X}3 changes to DEV, and information that DEV at {X}3 is even DEV_CRITICAL. The variables {X}1 and {X}2 are in DEV (because the criteria have other values). This variation results in a scenario in which {{X}}_ft avoids the DEV_CRITICAL state of {X}3.

[0168] A simulation model can be local, as it may relate to only a part of the machine. Such a simulation model can support the identification of the source of an event (and therefore need to be adapted to avoid the event).

[0169] Figure 3 shows a multivariate time series 500 representing multiple parameters. Each variate represents a single parameter. {{X}} represents the multivariate time series, with variates {X}i, i = 1 to N, representing N parameters. Index i is the variate index (see Figure 1), and N is the number of variates. {{X}} contains multiple univariate time series: {X}i, {X}2, ..., {X}i, ..., {X}N.

[0170] The variables {X}i are given by a univariate time series. In alternative notation, a univariate time series can be given as a sequence of samples {X1...XM}i.

[0171] M is the number of samples in the observation interval T_WINDOW (i.e., the time length of the time series), and individual samples are identified by index m. The interval has a duration of T_WINDOW=Δt*M, where Δt represents the sampling interval. T_WINDOW is again shown in the example of Figure 5.

[0172] As used herein, Δt is the same for all i. This is convenient for illustration purposes, but is not required in practice. Different univariate time series may use different sampling intervals. For example, temperature may be measured every minute, Δt = 60 seconds, while current may be measured every second, Δt = 1 second.

[0173] The notation Xmi represents the numerical value of the parameter sample 505 at time tm in the variable {X}i. For example, Xmi may be a temperature value of 30°C. Since semantics do not matter, values ​​can be normalized. For example, in a temperature range of -30°C to 70°C ("0" to "1"), 30°C can be treated as "0.5". The diagram indicates the variable range by arrow 510 (i.e., the y-axis of each variable). One skilled in the art can apply preprocessing to filter out infeasible variable values. For example, a faulty sensor may output a value of 1,000°C, and therefore such excess values ​​can be ignored.

[0174] A time instant tm is given for the end of each sampling interval Δt. This is simply a convenient convention, in other words tm identifies the sampling interval that ends at tm.

[0175] As used herein, a time series represents the complete observation interval from t1 to tM. A division of time is called a "segment." As an example, FIG. 3 shows a segment {X}i(ts,te) that includes values ​​from Xsi to Xei. The indices "s" and "e" represent the "start" and "end" times of the segment. FIG. 3 shows the segments arbitrarily, and different segments with specific meanings are shown in FIG. 4.

[0176] As explained above, a time series represents a parameter (of an industrial machine), and thus a segment of the time series represents the parameter during a time interval from a start time point to an end time point (i.e., during the segment interval).

[0177] In this specification, we distinguish between multivariate and univariate time series by notating them as {{}} or {}, so we may abbreviate "multivariate / univariate" herein or write them in parentheses, respectively.

[0178] This specification refers to time series in that the parameter samples 505 have numerical values. With or without normalization, one skilled in the art can encode Xmi in an appropriate data format (floating point real numbers; integers, etc.). Some parameters are typically represented by binary data (TRUE / FALSE, ON / OFF, etc.) or by text or strings, so one skilled in the art can apply the methods to such variables.

[0179] Those skilled in the art will appreciate that other types of data (such as images) can be preprocessed so that they are assigned numerical values. An example is provided at the end of this document.

[0180] FIG. 3 does not distinguish between the timing aspects {{X}}_op, _rf, etc. (see FIG. 2), but the differences between them are explained next.

[0181] FIG. 4 illustrates the multivariate time series 500 of FIG. 3 by distinguishing {{X}}_op and {{X}}_ft (collectively bold lines 501) from {{X}}_rf (dashed line 502).

[0182] Multivariate time series 501 and 502 correspond in that reference series 502 has samples that correspond to samples in operational time series 501 . ({{X}}_op corresponds to {{X}}_rf) ({{X}}_ft corresponds to {{X}}_rf)

[0183] This correspondence has two aspects. · Univariate correspondence means that a univariate time series {X}i_op has an equivalent univariate time series {X}i_rf, since both series refer to the same type of parameters. Time correspondence means that for two univariate time series {X}i_op (or {Xi}_ft) and {X}i_rf, there are samples taken at the same relative time. The figure shows the time correspondence with a vertical line 514 for {X}i.

[0184] Those skilled in the art will be able to identify the corresponding time series in advance, and the same index i will be used herein.

[0185] For simplicity, we can also assume that the number of variables N is the same in {{X}}_op (or {{X}}_ft) and {{X}}_rf. This is convenient, but not required.

[0186] As already mentioned above, reference data is usually available in the tolerance band. {X}1_rf should have upper and lower borders that remain constant across the entire T_WINDOW. The borders have different line styles. {X}2_rf should be similar to {X}1_rf, but with a larger tolerance (here, the boundary-to-boundary distance). ·{X}3_rf should rise during T_WINDOW. ·{X}N_rf should be constant for a relatively long time, but should decrease towards the end T_WINDOW.

[0187] Tolerance should not be confused with range 510 (see FIG. 3).

[0188] Segments can also be identified in terms of the relationship between the univariate time series {X}i_op (or _ft) and {X}i_rf, rather than as arbitrarily shown in FIG.

[0189] As explained above with the state transition graph of FIG. 1, parameters can change their states, and states and transitions can also be associated with a time diagram (as in FIG. 4).

[0190] According to certain rules, the computer can distinguish between the following segments: a deviation segment given as / X / N (here the parameter state DEV in the example from tDEV to tM for {X}i given as / X / i and for a shorter interval for {X}N), or Non-deviant segments (parameter state CORR).

[0191] As mentioned above, a segment of a time series represents a parameter during a time interval during a segment interval. Therefore, the attributes "deviant" and "non-deviant" are applicable to a parameter during a segment duration, and the parameter may deviate or may not deviate. Thus, a deviant segment identifies a deviation of a parameter, quantifies the deviation value, or indicates a variable-specific error. A deviant segment may occur during a time interval—a segment interval—in the past, present, or, if predicted, in the future.

[0192] For example, a first rule may define the start of a deviation segment / / when the operational (or future) value Xmi_op (or Xmi_ft) begins to differ from its corresponding reference value Xmi_rf. The first rule may define the end of the deviation segment / / accordingly.

[0193] According to this first rule, the time series {X}i_op (Fig. 4) has a deviation in the segment / X / i_op because its value (Xmi) differs from the corresponding value in {X}_rf. The same principle applies to the segment / X / N_op.

[0194] In this first rule, it does not matter whether the reference value is higher (for variable i) or lower (as in variable N) than the operating value. The second rule can take "higher" or "lower" into account.

[0195] A third rule can be specific to time series with binary values: for example, for a binary time series {0,1,1,1,0), in the reference {0,0,0,0,0}, there is one deviating segment / 1,1,1 / , because the segment is not 0 as in the reference.

[0196] The number of segments for each time series {X}i is limited by the number of values ​​M in the T_WINDOW.

[0197] The fourth rule can be based on other relationships, for example, if the reference time series oscillates (e.g., between +1 at tm and -1 at t(m+1)), and the operational (or future) time series remains 0 (with minimal change), then the operational (or future) time series is of good quality as a deviation unless there is an oscillation.

[0198] The fifth rule can take into account different criteria. It is convenient to take the example of a vehicle (see Figure 6): the route taken by the vehicle corresponds to a certain route class (non-deviation, state CORR), but deviates from other route classes (state DEV).

[0199] Similarly, deviation segments can be determined for {{X}}_ft. The notation / / instead of {} simply indicates that the segment is a deviation segment. The computer can track the timing separately for each deviation segment, such as by tracking the start and end times (see ts, te introduced in Figure 3, which apply to the deviation segment / / ).

[0200] Those skilled in the art can select appropriate rules and can also set up additional rules.

[0201] Verify segment eligibility Having described the basics of identifying segments (and distinguishing between deviant and non-deviant segments), we now consider some more advanced observations that can be derived from the segments and applied to the univariate time series {X}i_op (or equivalently {X}i_ft). Advanced observations can be related to importance and can be performed by an importance detection function (e.g., by importance predictor 220).

[0202] Again, the segments are merely a convenient illustration in a time diagram to explain state transitions (see FIG. 1).

[0203] In the first category, univariate time series can be distinguished into deviant time series (at least one deviant segment / / is required) and non-deviant time series. Figure 4 shows {X}1 and {X}2 as non-deviant, and {X}i and {X}N as deviant.

[0204] In the same first category, deviation time series can be further distinguished according to the number of deviations, where a multiple deviation time series has at least two / / s, such as {X}i and {X}N, and a single deviation time series has only one / / s. The number of deviations corresponds to the number of state transitions (multiple CORR-to-DEV transitions or only one transition).

[0205] In the second category, univariate time series (with deviations) can be compared to each other by criteria such as: The integral over time of the values ​​of the (deviant) segments that differ from the values ​​relative to the reference can be calculated for each (deviant) time series. If two time series use the same range (see range 510 in Figure 3), the integrals can be compared. The two series are distinguished by their integrals. The number of deviating segments can be used to classify time series and is simplified to distinguish frequently deviating series from occasionally deviating series. The criterion can be considered as an intervariate criterion since multiple variables (i=1 to N) are considered. The duration of the deviation can also be a reference. In a simplified example {0,1,1,1,0) against the reference {0,0,0,0,0}, the deviation has a relative duration of 60 percent.

[0206] Such a second category distinction may be useful to (subsequently) determine whether the deviation (DEV state after transition from CORR to DEV) is also a critical deviation (causing abnormal operation of the machine).

[0207] Detecting Importance Not all deviations (in a segment or otherwise) lead to DEV_CRITICAL. If multiple parameters are in the DEV state, the computer can implement a ranking. Ranking can be advantageous because it introduces robustness to the computer warning the operator by indicating potentially non-critical parameters.

[0208] As an example, the present specification now describes one (optional) approach to detecting importance.

[0209] A computer (predictor 220) can evaluate the deviation segments / / and calculate the impact of a deviation (or multiple deviations).

[0210] Taking {X}i as an example, the graphic area between the dotted and bold lines for the time interval from tDEV to tM (the duration of the deviation segment from the transition CORR to DEV) can quantify the deviation. This calculation is a simple integral calculation. The deviation should have a deviation value DEV(i).

[0211] Similarly, the graphical area between the dotted and bold lines for {X}N yields the value DEV(N). The deviation segment is shorter.

[0212] Rules can be defined to identify the importance, and the rules can be similar to the rules for identifying deviations. The following is given as an example: A first univariate time series is more critical than a second univariate time series if the deviation value (optionally, the amount of that value) is higher. For example, the deviation of parameter i is critical because DEV(i)>DEV(N). This is an example of importance identification, which compares deviation segments in different variables (here, i and N). If the value is higher than the threshold, the univariate time series deviates definitively. This is an example of significance identification that can remain within the variable, and comparison with other variables is not required. Univariate time series can be partially deterministic. For example, a time series can have non-critical deviations and then - later - critical deviations.

[0213] The deviance value DEV(i) can be considered as a variate-specific error. For example, it is possible to define the overall error ERROR as the sum of all DEV(i) from i=1 to i=N. In a modified form, the overall error can be calculated as the sum of the squares of all DEV(i). Other options exist.

[0214] Replacement Segment When the computer attempts to identify an action (see FIG. 1), the computer (the significance predictor 220 in combination with the simulator 230) can replace deviant segments with non-deviant segments for the simulation only.

[0215] This is illustrated by the example of Figure 4. By means of a dotted line and by the notation ~~, Figure 4 indicates a replacement segment that corresponds in time to the deviating segment / / . The value of the replacement segment can correspond to a reference (in this example, the midpoint between the upper and lower tolerance values). It is also possible to calculate the replacement segment by an autoencoder. This is explained in more detail below.

[0216] For example, a substitution segment ~X~i corresponds in time to a deviation segment / X / i (the segments start and stop at the same time), but not in value (the deviation "goes down" but the reference goes up, and the substitution dominates the reference).

[0217] For example, the replacement segment ~X~N corresponds in time to the deviating segment / X / N, but not in value.

[0218] The computer (predictor 220) can use the replacement segment .times. ...

[0219] The computer can calculate any of the following: · Described deviation value DEV(i) only for the deviating segment; "Difference to Reference" REF_DIFFERENCE(i) (difference between _op or _ft and _rf (relative to the upper boundary, lower boundary, or midline)

[0220] For the simulation, the computer can apply variations (v1...v4), for example, by selectively replacing or not replacing deviant segments. The computer calculates REF_DIFFERENCE(1) and REF_DIFFERENCE(2) for the unaltered time series, but the computer makes the difference for (i) and (N). Simulation (1) uses variation v1 in that / X / i and / X / N are both replaced by ~X~i, ~X~N. Simulation (2) uses variation v2 in that only / X / i is substituted. Simulation (3) uses variation v3 in that only / X / N is replaced. Simulation (4) uses variation v4 without replacement.

[0221] In general, there are K variations vk (k=1 to K), and the example of K=4 variations is merely a simplified example.

[0222] The computer can calculate ERROR across all variables {{X}} from i=1 to i=N, and the error will be specific to the case simulation. Since substitution (theoretically) reduces deviance (in the simulation), one of simulations (2) or (3) will point to the variable with the highest influence. (Simulations (1) and (4) can be ignored.)

[0223] Other options are also applicable.

[0224] Reimagined Actions In simplification, the most critical deviations may lead to the highest ranked actions. We will illustrate this with an example. Considering the topic of (non-)actuators introduced above, note that the highest impacts from a simulation may not (yet) lead to any action.

[0225] Simplified example of a time series Figure 5 shows the univariate time series of parameter X_1 in panels (i), (ii), and (iii). Figure (i) shows the time series for the complete window time interval T_WINDOW between the time points t_start and t_end. Figure (ii) shows the time series of the partial time interval applicable to the current operation of the industrial machine from the start time t_start to the observation time t_current. Figure (iii) shows the time series from the prediction time t_predict to the time t_end applicable to the future operation of the industrial machine.

[0226] In a simplified example, the univariate time series {X}1 in the figure should belong to the multivariate time series {{X}} introduced in Figure 3. Continuing the value-versus-time notation, {X}1 is shown with its numerical value running vertically and time progressing horizontally.

[0227] The T_WINDOW interval is advantageously distinguished into a first sub-interval from a starting time t_start to a time t_intermediate, and a second sub-interval that lasts until t_end. Both sub-intervals may have substantially equal duration, so that t_intermediate occurs approximately T_WINDOW / 2 after t_start.

[0228] As shown in diagram (i) of the figure, during the T_WINDOW interval, the value of {X}1 should rise from a minimum value ("min") to a maximum value ("max") during the first subinterval, and then fall back to the minimum value during the second subinterval. Because the graph is given by a straight line, it can be assumed that the rate of increase (or decrease) may be substantially constant. This is merely a convenient simplification.

[0229] It can be further assumed that such increase / decrease behavior can be derived from reference data, for example, from {{X}}_rf (i.e., more specifically, from multiple past occurrences of {X}1_rf). In other words, a training ML tool can establish {X}1_rf, and no further explanation is needed here.

[0230] As in figure (ii), the value of {X}1 should be a measurement value (see figure 2) belonging to {{X}}_op. It should rise from a minimum value (min) at t_start to at least t_current (the latest point in time for which its calculated value is available). The point in time t_current corresponds to the current time. The line does not appear to be a straight line as in figure (i), but simply symbolizes that {X}1_op is a measurement value.

[0231] Since no observations can be made about the future, {{X}}_op is not available for any time in the future. However, the action predictor 210 (see FIG. 2) predicts what will happen next.

[0232] As shown in diagram (iii) of the figure, the value of {X}1 should continue to rise (until t_intermediate) and then fall, as explained for diagram (i). Hence, the value belongs to {{X}}}_ft ("future").

[0233] In principle, it is possible to consider virtually all variables in {{X}} in terms of their reference (_rf), action (_op), and future (_ft) divisions.

[0234] In this univariate example, it is possible to detect deviations in {X}1 (and ultimately identify their significance), but in the following example with vehicles, we look at additional parameters.

[0235] Time series example Figure 6 shows the multivariate time series {{X}} with the individual univariate time series {X}1, {X}2, and X\{3\} in a slightly different view, considering the variate dependence rather than the time process (as in Figure 4).

[0236] As shown at the top of the diagram, the {X}1 value corresponds to the vertical axis (between "min" and "max"), the {X}2 value corresponds to the horizontal axis (again between "min" and "max"), and the {X}3 value is represented by the thickness of the line. In this example, the {X}3 value is a binary value: thin and thick lines.

[0237] Vehicle Examples It does not matter which physical parameters the univariate time series {X}1, {X}2, and {X}3 actually represent. The computer 200 is agnostic to the physical parameters, at least for most of the calculations. In one example, it is convenient to imagine the industrial machines 101 / 102 as vehicles moving within the factory 800. Such intra-factory vehicles can move more or less autonomously. The vehicles can be battery-powered. The vehicles may carry (intermediate) products, and deviations in product delivery can have different impacts on the factory.

[0238] Vehicle batteries can be easily replaced or recharged (if necessary) and arrival delays are negligible, but cargo destruction is very difficult to deal with. (Considering the ranking above, cargo destruction is the highest deviation.) Overall complexity is an issue here. As a side note, the factory itself can be seen as an industrial system with vehicles as one of its components. Vehicles can also be considered as an example of a machine that orchestrates maintenance with minimal interaction from human workers.

[0239] For that example, the figure also shows the factory 800 as seen from above. In a normal process step (or rather a transport step), a vehicle would move within the factory 800 from a first corner 801 (at min / min position) to a second corner 802 (at max / max position) and would return to the first corner 801. The vehicle could deliver intermediate products throughout the factory, for example from corner 801 to corner 802. In other words, the vehicle would return from corner 802 empty of a load.

[0240] {X}1 and {X}2 represent the two coordinates of the vehicle position. A transport step has an average duration of T_WINDOW. Not all transports require the same time.

[0241] In that scenario, {X}1 combined with X\{2\} represents the vehicle's path 805. {X}1 does not necessarily have a constant speed, but rather varies over time as shown in Figure 4, with the position {X}1 changing from "min" to "max" and back to "min" again. The same is true for {X}2.

[0242] Route 805 can pass through certain areas within factory 800, such as areas 811 and 812. In the figure, areas 811, 812 are symbolized by rectangles with rounded corners, but the shape of the areas does not matter. At any time within {{X}_op, the computer can detect vehicles that are within areas 811 and 812 based on their (X1, X2) coordinates.

[0243] For convenience of illustration, {X}3 can also be assigned a meaning, for example, mechanical vibration of a vehicle. To remain a binary value, there should be a "weak" and a "severe" magnitude of vibration, and again, a binary distinction should be present to keep the example as simple as possible. Of course, those skilled in the art can identify thresholds, etc., and even the difference between both binaries can be established by machine learning. Note that computers treat values ​​as numbers, and "weak" or "severe," or other semantics are used here only for convenience of explanation.

[0244] Reference data {{X}}_rf should be similar to FIG. 4 for {X}1 and {X2} (i.e., increase or decrease), and should also be available for {X}3_rf. As an example, {X}3 should be "intense" for the (X1, X2) coordinate within region 812. This also applies to {{X}}_rf as well as {{X}}_op.

[0245] It is also possible to classify the reference path by combining {X}1 and {X}2. For example, - A "wall path" class for vehicles moving near walls, as in case (A), A "diagonal route" class for vehicles traveling the shortest route, as in case (B), or "Wall diagonal path" class like (C) exists.

[0246] The class names in quotation marks " " are given here arbitrarily and the computer is not required to use them.

[0247] {{X}}_rf can be assumed to be a collection of historical data recording past movements, in the vehicle example, perhaps with a 50 / 40 / 10 percent distribution for cases (A), (B), and (C).

[0248] Figure 5 further illustrates the difference between the modalities, with the computed values ​​{{X}}_op given as open circles and the future values ​​{{X}}_ft given as filled circles. This diagram is also simplified by showing approximately 10 values ​​per trip. More realistic in-plant vehicles require several minutes, and a data sampling interval ΔT of 1 second is appropriate. Thus, a single transport step (i.e., a round trip) provides several hundred values.

[0249] As the time distance increases, the prediction accuracy may simply worsen: the black dot at the end of the path may be larger than the black dot at an earlier point in time.

[0250] Operational prediction The motion predictor 210 processes {X}_op and thus predicts the value of each univariate time series {X}1, {X2} (e.g., the parameter "position" having two coordinates) and {X3} (e.g., the parameter "vibration"). The last open circle (a) in (A), (b) in (B), and (c) in (C) symbolize the position (X1, X2) at the time t_current. (For example, if the motion predictor 210 is implemented as a machine learning tool trained on historical reference data, the reference time series {X}_rf also plays a role.)

[0251] For example, based on the received first position coordinates (X1, X2), the motion predictor 210 can predict whether the vehicle will take a "wall path" as in (A) or a "diagonal path" as in (B).

[0252] The motion predictor 210 also predicts the value of {X}3 depending on the path. For example, if the vehicle moves through region 811 (at t_current), it will experience some "intense" vibrations in region 812.

[0253] Diagram (A) shows the position (a) at t_current in region 811 by an open circle, and shows filled circles in region 811 (above thin line, "weak vibration") and region 812 (thick line, strong vibration). In diagrams (B) and (C), the motion predictor 210 predicts movement along a diagonal path, but the vibration is predicted to be "weak".

[0254] In other words, the motion predictor 210 has at least partially predicted or predicted the remaining path. The figure shows a black circle at the predicted location. It may not yet distinguish between the return path from corners (B) and (C) 802, but it at least predicts {X}3 with a predicted location.

[0255] Deviation It should be assumed that vibrations cannot be avoided. For {X}3_ft, which corresponds to region 812, the severity predictor 220 identifies the vibration value as a deviation segment (see the two black circles on the thick line). In other words, the severity predictor 220 predicts the transition of {X}3 from CORR to DEV for some time in the future (see T_event in FIG. 7). (In other words, "weak" corresponds to CORR, and "severe" corresponds to DEV.)

[0256] Detecting Importance It can also be assumed that at a relatively early time t_predict (see FIG. 7), the severity predictor 220 predicts that the new state will be DEV_CRITICAL (not just DEV, but DEV_CRITICAL). (Due to the additional parameter that the vehicle carries a load (from corner 801 to corner 802), there may be some other parameters, such as DEV_CRITICAL, or past observations. In other words, during T_event, an event (see bold arrow 11a) occurs, which causes the state of X3 to change.

[0257] The severity predictor 220 can identify CP=X3 ("vibration") at the estimated time interval T_event and output {{X}}_ft_CP (to the operator 190 via the user interface 290). For example, the computer can output (to the operator 190) the identified future event along with a warning. (In this example, {{X}}_ft_CP can also be accompanied by an indication of the vehicle's location.)

[0258] simulation Simply issuing a warning may not be enough: in factories, workers may be preoccupied with other tasks, so their attention to the vehicle takes second place.

[0259] The simulator can run a simulation with the position (X1, X2) as a varying parameter (actuator parameter) (here the two variations are going along the wall or diagonally). The ERROR is calculated and one simulation shows that the "vibration" can be minimized.

[0260] Of course, the computer operates semantically agnostic and calculates DEV(1) and DEV(2) as potentially contributing to error in some cases, but taking different paths does not contribute to the overall performance of the vehicle.

[0261] FIG. 7 shows a timeline of the steps performed by the computer 200 executing the computer-implemented method 403.

[0262] Method 403 is In step 413, the motion predictor 210 predicts the motion of the machine 101 and provides {X}_ft; In step 423, the importance predictor 220 predicts events whose parameters indicate deviations that will be critical (DEV_CRITICAL, critical transitions according to arrows 11a or 31 in FIG. 1). Includes:

[0263] 7 and the present specification is simplified by assuming that the critical transition occurs. However, the significance predictor 220 may also perform step 423 and obtain the result that the critical transition does not occur.

[0264] 7 further illustrates that the behavior predictor 210 and the importance predictor 220 can be implemented by machine learning tools, with steps 412 and 422 representing the training steps of the modules, respectively. The training can be considered as a training method 402. However, it should be noted that the training is only applicable to other implementations where such implementations have already been introduced (see FIG. 2). For training, the reference time series {{X}} serves as historical data.

[0265] Putting step execution in the context of data availability, Figure 7 shows the availability of the multivariate time series {{X}} in {{X}}_rf, {{X}}_op, {{X}}_ft, and {{X}}_ft_CP. The progression of time is indicated by the time axis from left to right, which is not scaled.

[0266] A step is symbolized by a box, the left side of which projects to the earliest point in time at which the execution of the step can begin, and the right side of which projects to the earliest point in time at which the results from the execution of the step can be used (for subsequent actions).

[0267] The availability of {{X}} is symbolized by a horizontal line drawn parallel to the time axis, which may be dashed to indicate that the time series is available but not necessarily used by module 210 / 220.

[0268] A vertical line (with an arrow) going into a box (usually from a horizontal line) indicates that a time series {{X}} in a particular modality is fed (at least in part) into module 210 / 220.

[0269] Time points and time intervals Here, the present specification (re)introduces, in chronological order, time points and time intervals.

[0270] The time t_collect symbolizes that collection of {{X}}_rf started in the past. It is not required to know the exact time t_collect; collection could have started several years ago.

[0271] Time instant t_train_1 symbolizes the start of training steps 412 and 422. Step 412, if performed by a machine learning tool, symbolizes the process of training the behavior predictor 210, and step 422 symbolizes the process of training the importance predictor 220. Training is typically enabled when a suitable time series {{X}}_rf is available. This is also indicated by the vertical arrows 1 and 2 from {{X}}_rf to the left of boxes 412, 422, respectively.

[0272] The time t_train_2 symbolizes that training has been performed such that the operator predictor 210 is able to predict the behavior of the machine 101 (see FIG. 2) and the severity predictor 220 is able to identify the condition DEV_CRITICAL.

[0273] In the diagram, training of both modules 210 / 220 is shown starting and stopping simultaneously at t_train_1 and t_train_2, respectively. This is merely a simplification of the diagram and is not actually required. Training can continue for new {{X}}_rf as they become available.

[0274] From time t_train_2, the behavior predictor 210 and importance predictor 220 are ready to be used, but no data needs to be provided by these modules yet.

[0275] The time t_start symbolizes that the industrial machine 101 begins a new operating cycle, such that {{X}}_op becomes gradually available from t_start (see the example of a vehicle exiting corner 801 in Figure 6).

[0276] The time t_current indicates the time when {{X}}_op is available with the latest data. T_op is the operation time interval currently in progress (i.e., from t_start to t_present). From the time t_current, the motion predictor 210 starts executing step 413 ("Predict"), which includes receiving {{X}}_op. The vertical arrow 3 symbolizes that the motion predictor 210 processes {{X}}_op (from t_start to t_current, as long as it is available). In this example, the vehicle is at the position of the last open circle (a), (b), or (c), see Figure 6.

[0277] The operation of the machine is ongoing after t_current, and the time it takes the motion predictor 210 to perform step 413 is relatively short (compared to T_op, the figure is not to scale).

[0278] The time t_predict indicates the earliest time point at which the motion predictor 210 gives {{X}}_ft as a result (see the right side of the box). Again, the diagram is simplified since the execution of step 413 can continue. Over time, as more and more data in {{X}}_op becomes available, {{X}}_ft should become more accurate. In the diagram of Figure 5, the black circles become smaller.

[0279] The time interval from t_current to t_predict can be considered the runtime of the motion predictor 210. In an ideal situation, {{X}}_op would still be available at t_predict so that the motion predictor 210 could repeat the steps. As a result, looking at Figure 6, the black circles become smaller.

[0280] The vertical arrow 4 symbolizes the data transfer of {{X}}_ft to the importance predictor 220 .

[0281] Thus, time t_predict indicates the earliest time point at which importance predictor 220 can start executing step 423. As indicated by vertical arrows 5 and 6, importance predictor 220 does not need to depend on {{X}}_ft, but can optionally process {{X}}_rf and {{X}}_op.

[0282] At time t_result, the severity predictor 220 has identified a critical condition DEV_CRITICAL for at least one parameter that will occur in the future (ie, the future is after t_result).

[0283] The diagram shows the result with a further horizontal line for {{X}}_ft_CP, i.e., in short, a computer-generated annotated multivariate time series of future {{X}}_ft. For example, the annotation can indicate the specific univariate time series {X}i where deviations are expected to occur, symbolized here by the "CP" critical parameter, the start and end times of the variation, a numerical estimate, etc. The vertical arrow 7 simply indicates that {{X}}_ft_CP comes from the significance predictor 220 performing step 423.

[0284] The time interval from t_predict to t_result can be considered the runtime of the importance predictor 220. (Because the importance predictor 220 processes {{X}}_ft but not {{X}}_op, the amount of data to be processed is relatively small compared to the amount of data available for processing.) Due to its relatively short runtime, t_result is before the deviation occurs.

[0285] The time interval T_event indicates the time interval during which the severity predictor 220 predicts a transition to DEV_CRITICAL, i.e., an event to be avoided. T_event is given as an interval.

[0286] The time t_end indicates the latest time point at which the motion predictor 210 can predict {{X}}_ft. In other words, there is a prediction time interval (T_ft) from t_future to t_end. The figure shows this time point to symbolize that the prediction is time-bounded.

[0287] To summarize this overview, at time t_result, computer 200 notifies operator 190 that a particular event (i.e., a parameter becoming critical) is expected during T_event (in the future relative to t_result). Operator 190 can then take appropriate action, typically with the aim of avoiding this event.

[0288] This document describes how computer 200 can assist merchant 190 in finding an appropriate remedy (or "action"). This document refers to Figure 8, but it will be helpful to look briefly at a vehicle example.

[0289] Timing aspects considering vehicle example The computer 200 (with its action predictor 210 and importance predictor 220) does not need to handle semantics, however it is convenient for illustration to briefly look again at Figures 6-7.

[0290] From the time t_trained, the motion predictor 210 is able to predict the vehicle path and also to predict some severe oscillations in the region 812. Assuming that the vehicle approaches a "wall" path at t_current, the motion predictor 210 provides {X}3_ft where severe oscillations are expected (see black dots on the thick line in FIG. 6, case (A)). If the vehicle enters a diagonal path at t_current, {X}3_ft shows "weak" oscillations (black circles on the thin line in FIG. 6, case (B)).

[0291] Because severe oscillations are not necessarily recognized as critical events (events that transition to DEV_CRITICAL, see the bold arrow in Figure 1), the deviation identifier may not identify such oscillation variances as deviations. The deviation identifier may recognize other deviations.

[0292] Given the predicted path (through region 812), the significance predictor 220 (optionally receiving {{X}}_rf) can identify {X}3 as deviating during T_event (i.e., when the vehicle passes through region 812).

[0293] simulation FIG. 8 shows a continuation of the timeline of further steps 433 performed by simulator 230, along with further data availability graphs.

[0294] FIG. 8 repeats steps 413 and 423 and introduces step 433 as a simulation step (by simulator 230).

[0295] As already explained, the behavior predictor 210 (at least initially) performs step 413 from t_current to t_predict, and the significance predictor 220 (at least initially) performs step 423 from t_predict to t_result.

[0296] From time t_result, {{X}}_ft_CP is available (horizontal line), simulator 230 can begin simulating (step 433), and simulator 230 identifies an action and a time (t_action) to take this action. Time t_action is before T_event. (The computer is again under "time constraints."

[0297] The simulator 230 performs the following sub-steps: (1) modifying the critical parameters to a set of variations (or modifying other parameters); (2) for each variation, by predicting the machine behavior for the modified parameters (e.g., implemented by invoking the behavior predictor 210 to update {{X}}_ft); (3) by evaluating {{X}}_ft to determine whether the variation still leads to the parameter becoming DEV_CRITICAL; (4) (e.g., avoiding the event (during T_event) or shifting the occurrence of the event further into the future (to the right in the diagram) - the impact can be selected according to a predetermined rule or by a pre-trained module - by selecting the variation that has the lowest impact on the behavior (i.e., the least significant); and (5) optionally by outputting the selection as a recommended action (or by outputting an identification of the parameter with the deviation segment / / replaced with ~~ so that the operator 190, see FIG. 2, understands what to do as an action); The simulation of step 433 is performed.

[0298] The simulator 230 was introduced herein above as an optional module (see FIG. 2) and will be described in detail below. In other words, by outputting a selection (optionally in the context of {{X}}_ft_CP), the operator 190 is able to modify a parameter (subject to the "actuator" being able to modify the parameter). Given the CP, the selection may correspond to the CP, but need not.

[0299] The sub-steps are examined in more detail.

[0300] (1) The simulator 230 does not actually modify the parameters, but modifies the data representing the parameters. For example, the simulator 230 does not change the temperature or reduce the vibration of the machine. The simulator only changes the data. Modifying a single parameter to have at least two different values ​​results in two sets of variations, but more realistically, it is a modification to multiple parameters, among which parameters are predicted to be critical (DEV_CRITICAL). This specification describes an example of a vehicle with modified parameters such as vibration and path, and an example of a bearing with four parameters in 12 variations.

[0301] (2) Note that simulator 230 invokes motion predictor 210, which in a variation provides a "planned" motion. The "planned" motion is not necessarily a "fixed" motion, since the parameters are not actually changed (i.e., they are changed only in the simulation, not in machine 101).

[0302] (3) In the following paragraphs, the specification refers to an example vehicle, where certain variations (particular route choices) prevent the parameter "vibration" from becoming critical. The specification provides a negative example here because the operator 190 is interested in preventing the parameter from becoming critical (see actions in FIG. 1, such as action 611a).

[0303] (4) and (5) Selecting the variation with the lowest impact (or, in other words, the least impactful) does not force the operator 190 to apply that variation. Outputting a selection should be understood as a recommendation. Although given in the singular, a selection may include multiple variations, so the operator makes the final selection.

[0304] Vehicle Examples As mentioned above, industrial machines can be very complex, and having approximately N=3 variables is merely a simplification for purposes of explanation.

[0305] Computer 200 can use simulator 230 to simulate the alternatives. In this example, X3 "vibration" cannot be modified, but the parameter pair (X1, X2) represents the vehicle position, which is a modifiable parameter. (In other notations, the parameter pair is an actuator parameter.)

[0306] The simulator identifies two paths with a re-forecast of X3. "Wall Passage" where X3 is expected to be a critical hit (CP=X3) "Diagonal path" where X3 is expected to remain in CORR

[0307] Simulator 230 then outputs "diagonal path" as the recommended action (to be executed as t_action). In other words, the position can be corrected by not allowing the vehicle to enter region 812 and simply forcing the vehicle to take an alternative path. Figure 6 shows this alternative path at (D).

[0308] Note that simulator 230 provided alternatives by varying actuator parameters (the vehicle has actuators to change its path), but not by varying non-actuator parameters (X3 cannot be simulated to be "weak" because vibrations cannot be avoided).

[0309] As a result, the worker 190 can interact with the vehicle (not in all cases, but only when the vehicle approaches region 812) to have the vehicle take an alternative route, as shown in case (D). Initially, the motion predictor 210 may already have provided {{X}}_ft, and the route change makes {{X}}_ft stale, but the motion predictor 210 can recalculate {{X}}_ft for the alternative route.

[0310] More specifically, the simulator 230 causes modules 210 and 220 to repeat several calculations. The simulator 230 corrects the remaining path predicted in {{X}}_ft, i.e., the position (a1) of case (A), the (simulated, not real) positions (X1, X2) of the first variation in the black circle. Similarly, the simulator 230 corrects (X1, X2) for positions that the vehicle can (soon) reach.

[0311] The simulator 230 requires an updated prediction, with the first variation indicating that X3 is "violent" (because the vehicle is predicted to be in region 812), and the second variation resulting in X3 being "weak" (outside region 812).

[0312] The simulator 230 re-evaluates both variations (by invoking the significance predictor 220), but there is no deviation in the second variation (X3 remains "weak"). In reality there is a shock (the vehicle takes an alternative route), but the shock is tolerated (the shock is, for example, a negligible delay in the arrival of the vehicle back at corner 801).

[0313] Since X3 is expected to be "weak," simulator 230 selects the second variation.

[0314] The simulator 230 outputs the selection as a recommended action, for example with modified position instructions. Figure 6 illustrates this for case (D), where the vehicle avoids region 812.

[0315] root cause Industrial machines provide a wide variety of data that can be used here. Continuing with the example of a vehicle, which can also be considered a machine, the present specification will explain further aspects by reference to this example. Position X1, X2 and vibration X3 are parameters, while the presence (or absence) of a load on the vehicle is a further vehicle parameter X4.

[0316] It can be assumed that further data about the factory is available, such as a collection of images per location as X5 (potentially in sufficient quantity to have images not for every location, but along the route (even for the predicted route). The images do not need to be available as a time series, but the collection of images can be thought of as a de facto time series as the vehicle moves along the route.

[0317] The images can be classified (image processing is state of the art) and the classifier (resulting from the classification) can be used as input data (for the predictor 210, for the detector 220, for the simulator 230). Images should be classified as they show two classes of floors. For example, each image is classified as "uneven floor" or "smooth floor". Uneven floors are the (root) cause of vibrations. The computer can eventually associate the floor with the vibration, so that an action (such as an actuator) will take an alternative path (like Figure 6(D)) unless the floor is repaired.

[0318] Bearing example It does not matter whether the action (see the output of simulator 230) relates to the operation of the machine (taking a particular path that avoids a particular area) or to a special mode (that can be shut down during maintenance in many situations, etc.) By referring to the example of a bearing, the present specification continues by considering actions (to avoid abnormal operation).

[0319] Figure 9 shows a symbolic diagram of a bearing (two concentric circles for the inner and outer rings) with parameters of data types "temperature", "speed", "torque", "oil pressure", and "oil type". This diagram also symbolizes the temperature increase and therefore the bearing failure. Such a typical time series can be considered as the {temperature}_rf for "failure". The failure can be associated with a reduction in the rotation of the rotor supported by the bearing (for example, to 600 rpm). Considering the deviations introduced above, a sudden temperature increase can be considered as a transition from CORR to DEV_CRITICAL (see Figure 1).

[0320] Adding or changing oil is a well-established approach to avoiding or delaying failure (see action 611 in Figure 1), but again, computers do not necessarily implement rules that take oil type into account, but this example is useful for illustration.

[0321] FIG. 10 references the bearing of FIG. 9 and shows a simulation of the temperature parameter {temperature} for variations v1...v12 in four actuator parameters ("load," "oil_type," "pressure," and "torque"). The simulator 230 (in cooperation with the motion predictor 210) simulates three different ranges #1, #2, and #3 for each input parameter, resulting in 4 x 3 = 12 temperature values ​​over a time interval (such as during the T_WINDOW or a shorter interval). In this example, there are K=12 variations and K=12 simulations. For simplicity, FIG. 10 shows only four temperature plots over time (instead of 12 plots for the K=12 simulation).

[0322] In other words, a time series is available from {temperature}|#1#1#1#1_ft (variation v1, where "load", "oil_type", "pressure", and "torque" all have value #1, dashed line) to {temperature}|#3#3#3#3_ft (variation v2, which is value #3, also dashed line).

[0323] The value # is parameter specific. For example, for the parameter "load", the values ​​can be #1 = "not loaded", #2 = "50% loaded", and #3 = "full". There can be three different oil types: Type #1, Type #2, and Type #3. The acronym _ft refers to the future. The simulation data does not have to be equivalent to the operational data _op.

[0324] One of the time series corresponds to the failure scenario in Figure 9, where the computer can recognize certain temperature patterns as corresponding to the failure criteria. The other time series shows a scenario where a different oil type is used (as an action).

[0325] With respect to the states and transitions in FIG. 1, {temperature}|#1#1#1#1_ft corresponds to {temperature}_rf (in FIG. 9).

[0326] Selected Parameter Values When performing method step 433 (simulation), the simulator 230 can select the parameter variation that has the least impact on the operation of the particular industrial machine (101). There can be multiple variations. In this example, the variation of oil type #2 has the least impact. FIG. 10 shows, as an example, a variation v2=#1, #2, #1 (#, #2, # is more common: oil type matters, other parameters do not). Because the operator 190 (see FIG. 2) can see from the temperature curve (no substantial heating) that selecting oil type #2 is potentially beneficial, the curve is also given as "{temperature}_ft_action," and this notation is a specific instance of {{X}}_ft_action (see FIG. 2).

[0327] In other words, the action of running the bearing with Type #2 oil can be considered action 611 (see FIG. 1) to keep the temperature at CORR.

[0328] causation As already mentioned above in FIG. 1, the parameters are interdependent in their effect.

[0329] Although FIG. 1 distinguishes actions 611, 612, etc. according to transitions (to prevent or support the transition), a computer need not consider causality in all its precision.

[0330] Simulation allows for parameter changes to be compared taking into account their effect (on the operation of the machine in general and / or on the parameter's state change in particular). In the vehicle example, simulator 230 can modify further (actuator) parameters such as vehicle speed {X}7, which can also reduce vibrations by going slower (actions 611, 612 in FIG. 1). Since the computer may not consider semantics, there will only be the further parameter {X}7 that changes (action 612 in FIG. 1). The computer can then rank the different actuator parameters.

[0331] The computer can also rank different actions (equivalent to a rank parameter), with higher ranked actions being executed in preference to lower ranked actions.

[0332] The differences between the actions introduced above (see Figure 1) can be taken into account in the ranking: Action 632 (which removes importance) may have a higher rank than Action 611 (which maintains CORR).

[0333] Both actions "slow down" and "take an alternate route" reduce the likelihood of encountering vibrations (and the risk of the load falling off the vehicle), but the actions may have different efficiencies. The computer may give a higher rank to "take an alternate route" (and the vehicle will arrive substantially in time because the delay impact is negligible for that action). Both actions can be considered as actions 611 or 612 in Figure 1.

[0334] In the bearing example, changing the oil type (eg, to #3) may have a higher ranking than, for example, changing the oil pressure.

[0335] Thoughts on the Director This approach presents a solution for prescriptive maintenance, but does not require that any history about past actions (e.g., maintenance) be available. Such an approach is unsupervised.

[0336] In the vehicle example, processing the reference data {{X}}_rf makes it possible to predict that X3 will become a DEV (or even a DEV_CRITICAL) in the future, but there is no history of available actions. There is no historical data available for the vehicle to take an alternative route (see Figure 6(d)).

[0337] In the simulation, the "best" actuator parameters are those for which the simulation yielded different importance with the sharpest contrast: a simulated "wall path" leads to (simulated) "intense" vibration, and a simulated "diagonal path" leads to (simulated) "weak" vibration. The simulation may have yielded other parameters, such as different transport durations (i.e., T_WINDOW as a parameter expected to vary from path to path). However, this difference was greatest for vibration (from "intense" to "weak"), but not for duration (perhaps a one-minute earlier arrival time). This difference in parameter evaluation does not need to be communicated to the computer by supervision; the computer automatically selects the parameters with the sharpest contrast, without the need for supervision.

[0338] Similarly, the historical data (part of {{X}}_rf) includes data about maintenance (e.g., to recharge the vehicle battery) and about failures (e.g., battery failures leading to recharging). However, the historical data does not need to include data about all types of failures. A loaded vehicle may not have rolled over (in the past), but a potentially dangerous situation may have been prevented (by the vehicle taking an alternative route).

[0339] While avoiding supervision (by human experts) is not possible in all situations, computer-implemented methods such as those described can be used to identify actions in factories that are increasingly operated autonomously.

[0340] timing As explained, the computer 200 (through the simulator 230) suggests actions and there should be no substantial delay. The time interval (t_current to t_predict, action predictor; t_predict to t_result, significance predictor) needs to be short enough to allow the computer to deliver results before an event can occur ({{X}}_ft_OP) and before an action can be taken ({{X}}_ft_OP_action). The operation of the computer modules can be sped up by simplifying: The identification of the DEV state makes it possible to ignore non-deviant parameters, and therefore the severity predictor 220 does not have a check CORR parameter. Actions can only be identified for actuator parameters. Such an approach reduces the number of candidate actions to be identified. This approach may involve restricting variations to actuator parameters. In other words, actuators that cannot be changed in reality remain "no action" during the simulation. For example, the parameter "vibration" with two values ​​"intense" and "weak" results from changing other parameters (e.g., position, path, etc.), and changing the vibration to a specific value does not correspond to reality and can be avoided.

[0341] Autoencoder Deep autoencoders can be used to identify features that can be used to train architectures such as those described in papers such as Karl-Philipp Kortmann, Moritz Fehsenfeld, and Mark Wielitzka, "Autoencoder-based Representation Learning from Heterogeneous Multivariate Time Series Data of Mechatronic Systems", arXiv:2104.02784.

[0342] Autoencoders are described in scientific papers such as "Cyriana MARoelofs, Marc-Alexander Lutz, Stefan Faulstich, Stephan Vogt: Autoencoder-based anomaly root cause analysis for wind turbines. Energy and AI 4 (2021) 100065."

[0343] The autoencoder can be a convolutional autoencoder, or an LSTM autoencoder, or any other structure capable of processing sequential data, an example of which is described in "Roy Assaf, Ioana Giurgiu, Jonas Pfefferle, Serge Monney, Haris Pozidis, and Anika Schumann, "An Anomaly Detection and Explainability Framework using Convolutional Autoencoders for Data Storage Systems," Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-2000) Demonstrations Track."

[0344] FIG. 11 repeats the illustration of industrial machines 101 and 102 and computer 200 of FIG. 2, but adds an example of the optional use of autoencoders 201, 202, and 203. In this example, the autoencoder 201 receives {{X}}_op and provides {{X}}_rf (in addition to, or as an alternative to, {{X}}_rf from the machine 102). In this example, the autoencoder 202 receives {{X}}_ft and provides a reference not as {{X}}_rf from the machine 102, but rather as a reference based on prediction. In this example, the simulator 203 uses an autoencoder for various purposes, such as identifying a replacement segment ~X~ (which replaces the deviant segment in the above-mentioned variation before the simulation). In other words, the autoencoder 203 can also be used to calculate the replacement segment ~~ (see the examples of ~X~1 and ~X~N in Figure 4).

[0345] For the sake of simplicity of illustration and explanation, for example, an autoencoder does not need to encode a multivariate time series using all variables, but can apply autocoding to only some variables.

[0346] In the approach we have described, the use of an autoencoder is favorable for established {{X}}_rf (or other data) even in situations where historical data is sparse. Also, the use of an autoencoder can be advantageous as it reduces the effort required for human experts to annotate relevant features (in {{X}}_rf).

[0347] For example, deviations / X / i as in Figure 4 are indicated by values ​​that differ from the values ​​of the reference bands, but potentially a human expert needs to define these reference bands (e.g., with upper and lower boundaries).

[0348] In another example, the autoencoder can distinguish between the vehicle's future path following a wall or following a diagonal line. In that sense, the motion data {{X}}_op (for position, 6 open circles) acts as a code that causes a computer (here, the motion predictor 210) to derive the future path. Multimodality and variable preprocessing

[0349] Figures 12-14 illustrate variable preprocessing. More specifically, Figure 12 illustrates adapter 280 with image data to be adapted, Figure 13 illustrates the image data being adapted, and Figure 14 illustrates data aggregation by adapter 280.

[0350] The adapter 280 can be implemented as part of a module that performs the predicting step (see step 413). By way of example, Figure 12 shows an adapter 280 for pre-processing data from an industrial machine 101 (or from a machine 102). The adapter 280 receives multivariate measurement time series(s) from the industrial machine 101 or from a reference machine 102. Although the illustration is simplified by passing all N variables through the adapter, in an implementation not all variables need to be adapted.

[0351] As noted above, the notation Xmi represents the numerical value of the parameter sample 505 in the variable {X}i at time tm, see Figures 3-4, which show the parameter sample as a scalar. Those skilled in the art can receive (or obtain) the sample by well-known approaches, for example, by collecting measurement data, by collecting metadata (e.g., from a worksite application), etc.

[0352] However, data capture is not limited to scalar data: it is possible to apply data pre-processing to adapt the data modality, e.g., from image data to scalar data, from audio data to scalar data, from vector data to scalar data, etc.

[0353] In other words, the adapter 280 can optionally perform modality adaptation of data originally available as non-scalar data. As an example, this adaptation is shown for the variable {X}p.

[0354] The data for the variable {X}p can be a set of digital images 281 (i.e., a matrix with pixels that indicate color). The images can be taken by a camera or a scanner. The images 281 can be available as a sequence of images (e.g., for images every Δt from t1 to t8 in the figure). Considering the vehicle example above, the images could show, for example, a factory floor.

[0355] Adapter 280 can process the images and assign them a scalar value, shown here as the time series {X}p'. The dash ' merely indicates that an assignment has been made. One skilled in the art can apply image processing techniques such as classification by pre-trained neural networks or other methods.

[0356] For example, the image may show a moving machine component (or a non-moving part such as a floor). The adapter 280 may classify the component as "rough" or "not rough" (a binary classification for a floor) and moving at a speed of "0", "1", "2", "3", etc. (for a moving part).

[0357] Because floors can be uneven, images can be classified accordingly. Very simply, images showing a bump can be coded as 1 ("bump present"), and images without a bump can be coded as 0 ("absent"). Figure 16 shows a bump symbolically with a triangle. The resulting univariate time series can be {X}p' with parameter samples {0,0,0,0,0,1,1,1}', see 505, as "op," and the computer executes the described method using {X}p'. Assuming the reference is {X}p'_rf with parameter samples {1,1,1,1,0,0,0,0}, deviations can be determined initially as unexpected "no bump" and at the end of the window as unexpected "bump."

[0358] Those skilled in the art can adapt the sampling rate as needed: for example, image data may arrive as a video stream (25 image frames per second), and some frames may be removed due to arriving at Δt (which may be much longer than 1 / 25 of a second).

[0359] In a further scenario (not shown by the drawing but easily imaginable), the non-scalar element could be an audio sequence. For example, {X}p could be a collection of audio recordings (or "voice recordings", e.g., each having a duration less than or equal to Δt) from a microphone sensor. Those skilled in the art can apply appropriate audio processing, for example, by sampling the sound at a frequency of 20 kHz, which leads to 60*20,000 audio samples per minute. Note that in the vehicle example, a vehicle passing over an uneven floor may emit some characteristic sound.

[0360] In the context of providing Δt spacing to a time series, adapter 280 processes these millions of samples into a single scalar. For example, the scalar may indicate a pitch rise for Δt, a pitch fall Δt, a pitch constant for Δt, a pitch rise and fall for Δt, etc. Such patterns may be assigned to integers, or the varying pitch itself may be assigned to the parameter sample.

[0361] FIG. 13 shows image data adapted as in FIG. 12, but modified. Here, adapter 280 also receives the image sequence (shown for times t=t, t, and t), but now assigns scalar values ​​in two or more (univariate) time series given as {X} and {X} (again, a prime denotes an assignment). Adapter 280 identifies regions 282, 283 within image 281 and processes the data for the regions separately. In this example, region 282 indicates a machine component that rotates with a sense of direction "+1," or with a sense of direction "-1," or does not rotate at all ("0"), as shown for the assignment of {X}. Region 283 indicates a "fire" (triangle symbol) corresponding to "1," or "no fire" at "0." In this example, the diagram shows image 281 at time t ("+1" rotation, with fire).

[0362] Processing the data requires computational resources (e.g., CPU, memory, etc.), and the overall time to identify critical parameters (involving state transitions) can be critical (see real-time requirements above). The examples in Figures 12-14 show that preprocessing (by mapping non-scalar data to scalar data) can save on resource consumption.

[0363] Note that pre-processing the audio samples or images by the adapter simplifies the implementation: by being specific to audio samples or images, the adapter removes complexity from the predictor, which had to process scalars, not audio, not images.

[0364] FIG. 14 illustrates data aggregation, and applying such aggregation can also contribute to resource savings. As explained above (FIGS. 4-5), a multivariate time series {{X}} is a set of univariate time series {X}i. It is possible to identify subsets (of the univariate time series) and aggregate them before the method is performed. To perform the method, a computer can use, at least in part, the aggregated time series.

[0365] The figure shows an example where three univariate time series {X}1, {X2}, {X3} belong to {{X}}. They are aggregated into a univariate time series {X}'. {X}' is part of the multivariate time series on which the method is performed.

[0366] In this example, {X}1 and {X}2 indicate the position (of the vehicle) and {X}3 indicates the vibration.

[0367] To illustrate this with an example, a motor (or vehicle) that is forced to stop (or a vibrating vehicle) will make some characteristic noise and potentially heat up. A domain expert can define an appropriate rule so that {X}' changes (e.g., from t6, from 0 to 1).

[0368] Additionally, data aggregation (i.e., N to 1, N to 2, N to a few) complexity reductions are discussed in papers such as Zhihua Zhang and Michael I. Jordan, "Latent Variable Models for Dimensionality Reduction," Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, PMLR 5:655-662, 2009; and W. Wang, Y. Huang, Y. Wang, and L. Wang, "Generalized Autoencoder: A Neural Network Framework for Dimensionality Reduction," 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2014, pp. 496-503.

[0369] In the scenarios introduced in Figures 12-14, adapter 280 can be implemented by a neural network that has been previously trained, potentially under supervision (by a human expert). For example, adapter 280 may have been trained on annotated images showing bumps, annotated audio sequences, etc. Adapter 280 can also be trained in an unsupervised manner to identify abstract representations of images (or audio), ensuring maximization of the informative content of the images. A sequence of images (or audio) can then be reduced to a sequence of scalars by this process.

[0370] To identify regions in an image (see Figure 13), one skilled in the art can apply techniques from autonomous driving (e.g., car computers distinguish traffic signs from pedestrians). Statistics can play a role in training the network, applying rules, etc.

[0371] Similar expert involvement can be applied to the selection of subsets to be aggregated (see Figure 14). By selecting (time-series) subsets, the industrial machines are virtually divided into groups and the selection can take into account the components (see 110, 120, 130).

[0372] The univariate time series provided by the adapter (e.g., {X}', {X}p', and {X}p' in Figures 16-18) can optionally be semantically related. Such semantics are often referred to by metaphorical terms such as "health index" or "operating status." In the motor example, {X}' is obtained by aggregation and is rather a "motor problem index" that symbolizes the potential malfunction of one machine component (i.e., a specific motor). In that sense, {X}' does not yet indicate a critical parameter, but can enable a computer to find the (root) cause faster (see the vehicle and bearing scenario). In other words, {X}' may indicate a CR with relatively low accuracy, but is accurate enough to allow a computer to further investigate the machine, but not enough to investigate the machine component. {X}' enables prioritization, thus saving computational resources.

[0373] general-purpose computer FIG. 15 illustrates an example of a general-purpose computing device that can be used with the technology described herein. FIG. 15 illustrates an example of a general-purpose computing device 900 and a general-purpose mobile computing device 950 that can be used with the technology described herein. Computing device 900 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. General-purpose computing device 900 may correspond to computer system 200 of FIG. 2. Computing device 950 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, driver assistance systems or vehicle on-board computers, and other similar computing devices. For example, computing device 950 can be used as a front end by a user (e.g., a blast furnace operator) to interact with computing device 900. The components, their connections and relationships, and their functions illustrated herein are intended to be exemplary only and are not intended to limit the implementation of the invention(s) described and / or claimed herein.

[0374] Computing device 900 includes a processor 902, memory 904, a storage device 906, a high-speed interface 908 connecting to memory 904 and a high-speed expansion port 910, and a low-speed interface 912 connecting to a low-speed bus 914 and storage device 906. Each of the components 902, 904, 906, 908, 910, and 912 are interconnected using various buses and may be mounted on a common motherboard or otherwise as needed. Processor 902 can process instructions for execution within computing device 900, including instructions stored in memory 904 or on storage device 906 to display graphical information for a GUI on an external input / output device, such as a display 916 coupled to the high-speed interface 908. In other implementations, multiple processors and / or multiple buses may be used, along with multiple memories and multiple types of memory, as needed. Multiple computing devices 900 may also be connected, each providing a portion of the required operations (e.g., as a server bank, a group of blade servers, or a multiprocessor system).

[0375] The memory 904 stores information within the computing device 900. In one implementation, the memory 904 is one or more volatile memory units. In another implementation, the memory 904 is one or more non-volatile memory units. The memory 904 may also be another form of computer-readable medium, such as a magnetic or optical disk.

[0376] The storage device 906 is capable of providing mass storage for the computing device 900. In one implementation, the storage device 906 can be or include a computer-readable medium such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or an array of devices including devices in a storage area network or other configuration. A computer program product can be tangibly embodied on an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as memory 904, the storage device 906, or memory on the processor 902.

[0377] The high-speed controller 908 manages bandwidth-intensive operations of the computing device 900, while the low-speed controller 912 manages less bandwidth-intensive operations. This allocation of functionality is merely exemplary. In one implementation, the high-speed controller 908 is coupled to the memory 904, the display 916 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 910 that can accept various expansion cards (not shown). In this implementation, the low-speed controller 912 is coupled to the storage device 906 and the low-speed expansion port 914. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled, for example, via a network adapter, to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router.

[0378] Computing device 900, as shown, can be implemented in several different forms. For example, it may be implemented as a standard server 920, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 924. Additionally, it may be implemented in a personal computer, such as a laptop computer 922. Alternatively, components from computing device 900 can be combined with other components in a mobile device (not shown), such as device 950. Each such device can include one or more of computing devices 900, 950, and the entire system can be composed of multiple computing devices 900, 950 communicating with each other.

[0379] Computing device 950 includes, among other components, a processor 952, memory 964, input / output devices such as a display 954, a communications interface 966, and a transceiver 968. Device 950 may also include a storage device, such as a microdrive or other device, to provide additional storage. Each of components 950, 952, 964, 954, 966, and 968 are interconnected using various buses, and some of the components may be mounted on a common motherboard or otherwise as desired.

[0380] The processor 952 can execute instructions within the computing device 950, including instructions stored in the memory 964. The processor may be implemented as a chipset of chips including separate analog and digital processors. The processor can provide coordination of other components of the device 950, such as control of a user interface, applications run by the device 950, and wireless communications by the device 950.

[0381] The processor 952 can communicate with a user via a control interface 958 and a display interface 956 coupled to a display 954. The display 954 can be, for example, a TFT LCD (thin film transistor liquid crystal display) or an OLED (organic light emitting diode) display, or other suitable display technology. The display interface 956 can comprise appropriate circuitry for driving the display 954 to present graphics and other information to the user. The control interface 958 can receive commands from the user and convert them for submission to the processor 952. Additionally, an external interface 962 can be provided in communication with the processor 952 to enable near-field communication of the device 950 with other devices. The external interface 962 can provide, for example, wired communication in some implementations or wireless communication in other implementations, and multiple interfaces may also be used.

[0382] Memory 964 stores information within computing device 950. Memory 964 may be implemented as one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Expansion memory 984 may also be provided and connected to device 950 via expansion interface 982, which may include, for example, a SIMM (single in-line memory module) card interface. Such expansion memory 984 may provide additional storage space for device 950 or may store applications or other information for device 950. Specifically, expansion memory 984 may include instructions for performing or supplementing the processes described above and may also include secure information. Thus, for example, expansion memory 984 may function as a security module for device 950 and be programmed with instructions that enable secure use of device 950. Additionally, secure applications may be provided via a SIMM card along with additional information, such as placing identifying information on the SIMM card in an unhackable manner.

[0383] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied on an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as memory 964, expansion memory 984, or memory on processor 952, which may be received, for example, via transceiver 968 or external interface 962.

[0384] Device 950 can communicate wirelessly via communication interface 966, which may include digital signal processing circuitry as needed. Communication interface 966 can provide for communication under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, via radio frequency transceiver 968. Additionally, short-range communication may occur, such as using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, GPS (Global Positioning System) receiver module 980 may provide device 950 with additional navigation- and location-related wireless data that may be used as needed by applications executing on device 950.

[0385] Device 950 can also communicate audibly using audio codec 960, which can receive verbal information from a user and convert it into usable digital information. Audio codec 960 can likewise generate audible sounds for the user, such as through a speaker in the handset of device 950. Such sounds can include sounds from voice calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications running on device 950.

[0386] The computing device 950, as shown, can be implemented in several different forms. For example, it can be implemented as a mobile phone 980. It can also be implemented as part of a smartphone 982, personal digital assistant, or other similar mobile device.

[0387] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from, and send data and instructions to, a storage system, at least one input device, and at least one output device.

[0388] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language and / or in an assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0389] To provide for user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0390] The systems and techniques described herein can be implemented on computing devices that include back-end components (e.g., as data servers), or that include middleware components (e.g., application servers), or that include front-end components (e.g., client computers having a graphical user interface or web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the systems can be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0391] Computing devices may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0392] Although several embodiments have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0393] Additionally, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Additionally, other steps may be provided or steps may be eliminated from the described flows, and other components may be added to or removed from the described systems. Furthermore, other embodiments are within the scope of the following claims. [Explanation of symbols]

[0394] CORR, DEV state DEV_CRITICAL DEV_NON_CRITICAL 11,11a,11b,12,31,32 State transition 101 Industrial machinery under observation 102 Industrial machinery that fulfills the standard role 105 Machine Controller 190 workers 200 computers 210 Behavior Predictor 220 Importance Predictor 230 Simulator 280 adapter 281,282,283 Images, Regions within Images 402 Training method 403 Methods for identifying parameter state transitions 412,422 Method Steps (Training) 413, 423, 433 Method Steps (Identification) 500,501,502 Multivariate Time Series 505 Parameter Samples 510 range 611,612,631,632 Actions 800 factories 801,802 angle 811,812 areas General-purpose computers with 9xx components [Prior art documents] [Patent documents]

[0395] [Patent Document 1] US Patent Application Publication No. 2019 / 0147300

Claims

1. A computer-implemented method (403) for identifying a parameter state transition (11a, 31) predicted to occur in the future, the parameter state transition being a critical transition resulting from a predicted increase in the likelihood of a future change in an operating mode (X_mode) of an industrial machine (101), the operating mode being a technical state of the machine; the computer (200) processes a multivariate time series ({{X}}) that is a plurality of univariate time series ({X}i), each univariate time series ({X}i) representing a parameter (X_i) of the industrial machine (101); The method (403) comprises: a step (413) of predicting the behavior of said particular industrial machine (101), by processing an operational multivariate time series ({{X}}_op) representing the operation of said particular industrial machine (101) during a particular operational time interval (T_op) currently in progress (t_current); and by providing a future multivariate time series ({X}_ft) representing the predicted behavior of said particular industrial machine (101) during a particular prediction time interval (T_ft) extending into the future; A step (413) of predicting the operation of the specific industrial machine (101); a step (423) of processing the future multivariate time series ({X}_ft) to predict the parameter state transitions (11a, 31), The computer (a) determining, for parameters represented by univariate time series as part of the future multivariate time series ({X}_ft), that at least one particular parameter (X3) is predicted (DEV) to have a value different from a reference value in at least one deviation segment ( / X / i); and (b) determining a predicted likelihood of the change in the operating mode of the industrial machine (101) by quantifying the deviation (DEV(i)) in the deviation segment for the at least one particular parameter and by applying a predetermined rule; a step (423) of predicting said parameter state transitions (11a, 31) under the associated condition: A method (403) comprising:

2. 2. The method of claim 1, wherein the computer performs the step of predicting the behavior of the particular industrial machine with a behavior predictor module implemented by one selected from: a simulator; a machine learning tool that has been pre-trained using multivariate time series of behavior from past operations; and a computer function that applies mathematical relationships, thereby processing data by formulas or lookup tables.

3. the computer (200) performs a step (423) of predicting the parameter state transitions (11a, 31) by a significance predictor module (220) adapted to identify deviant segments in the univariate time series separately for each univariate time series by comparing the future multivariate time series ({X}_ft) with a reference multivariate time series ({X}_rf); 3. The method (403) of claim 1 or 2, whereby the deviation segments identify parameter deviations, quantify deviation values, or indicate variable-specific errors predicted to occur during future segment intervals.

4. 4. The method (403) according to any one of claims 1 to 3, wherein the computer (200) executes a step (423) of predicting the parameter state transitions (11a, 31) by means of a significance predictor module (220) adapted to identify differences between numerical values ​​obtained by preprocessing image or audio samples.

5. 5. The method according to claim 1, wherein the computer executes a step of predicting the parameter state transitions by a significance predictor module that has been pre-trained to determine the increased likelihood of a change in an operating mode of the industrial machine, the significance predictor module having been trained with historical data relating to the machine mode, the machine mode serving as a ground truth for output, and data relating to parameters and quantified deviations serving as input data.

6. The computer (200) - modifying the representation of said at least one specific parameter (X3) to obtain a set of parameter variations (v1...v4, v1...v12), wherein in at least one univariate time series representing said at least one specific parameter (X3), said computer replaces deviant segments ( / X / 1, / X / N) by replacement segments (~X~1, ~X~N) obtained from a reference or generated by an autoencoder (203); - predicting the operation of the specific industrial machine (101) separately for each parameter variation (vk) and evaluating whether the at least one specific parameter (X3) still has a value different from the reference value and still changes the likelihood of the change in operating mode (X_mode) of the industrial machine; selecting the parameter variation that has the least effect on the operation of the particular industrial machine (101); The method (403) of any one of claims 1 to 5, further comprising the step of simulating (433) by:

7. 7. The method (403) of claim 6, also a method for identifying an action and outputting the selected parameter variation as a recommended action to an operator (190) of the industrial machine (101).

8. The method (403) of claim 6 or 7, wherein the computer (200) limits data processing during the simulation to parameters that are actuator parameters.

9. The method (403) of any one of claims 1 to 8, wherein the parameter state transition indicates that the particular industrial machine is switching into the particular operating mode that is an abnormal machine operation.

10. 20. A computer program product that, when loaded into a memory of a computer system and executed by at least one processor of said computer system, causes said computer system to perform the steps of the computer-implemented method (403) of any one of claims 1 to 19.

11. A computer system (200) comprising a plurality of modules (210, 220, 230) that, when executed by the computer system, perform the steps of the computer-implemented method (403) of any one of claims 1 to 9.

12. 12. Use of the computer system (200) of claim 11 to distinguish parameters of a particular industrial machine (101) to identify a subset of parameters that are critical parameters (CP) that cause abnormal operation of said industrial machine (101).

Images