Generating virtual sensors for use in industrial machines

Abstract

The industrial machine (123) may not have a sensor for a particular parameter, so the computer uses the neural network (473) to virtualize the missing sensor. The computer trains a neural network (373) in steps of receiving a measurement time series with historical measurement data from a reference machine, processing the time series into a feature series that is invariant to differences in the domain of the reference machine, transforming the time series using the transformation rules, receiving a univariate time series of the further process parameter (z), and training the neural network with the feature series at the input and with the univariate time series at the output to provide parameter indices (Z') of a further process parameter (173, z) for the industrial machine (123).

Description

Technical Field

[0001] The present disclosure generally relates to industrial production processes, and more particularly to a computer system, method, and computer-program product for emulating sensors and emulating measurement data for process parameters of industrial machines that execute these processes.

Background Art

[0002] Simultaneously measuring the operation of an industrial machine and its process parameters is homogeneous. The measurement includes measuring parameters directly related to the process ongoing in the machine, such as measurement of temperature, pressure, sound, illuminance (such as readings of an infrared camera), reflection of light from a surface, gas concentration, and other physical properties or phenomena. However, the measurement can also relate to process parameters indirectly related to the process, such as measurement of the physical properties of the materials entering and leaving the machine (resulting from the process).

[0003] In that sense, an industrial machine is a machine under surveillance.

[0004] Furthermore, process parameters can be distinguished by their locations inside and outside the machine. For example, temperature distributions can be measured at different parts of the machine.

[0005] From a more general perspective, an industrial machine can be a machine that executes an industrial process. For example, the machine can be a reactor, a metallurgical furnace, a container, an engine. More specifically, from the perspective of the operating principle, the furnace can be a blast furnace. Put very simply, a blast furnace receives ore and coke as well as hot air (through tuyeres) and supplies molten metal.

[0006] Measurement is the main purpose of sensors related to the machine. For example, a machine can be equipped with a temperature sensor. Such a sensor can be implemented by a thermometer, a PT100 resistance temperature detector, or other types of temperature sensors.

[0007] The measurement results show at least two measurement modalities, namely, • By sensors associated with the machine, and • Through interaction between human operators and machines, It is well known that they can provide this.

[0008] In the case of a blast furnace, sensors are distributed in various locations, and in addition, operators typically take samples of the molten material. Simply put, the operator opens a portion of the blast furnace hearth (using a drilling machine in a step called tapping) to extract the molten material. The material then flows into a runner (called casting), and the operator can then examine the chemical composition, temperature, appearance, etc., of the sample.

[0009] Most measurement results are converted into measurement data. Regarding data modalities, measurement data is, for example, • Scalar as a function of temperature or pressure • Vectors for the movement patterns of machine parts • A matrix for images from surfaces, etc. It can have complex forms like this.

[0010] The measurement data will be processed by one or more computers. These computers can process the measurement data to obtain indicators for the machine's operating status. Based on the measurement data and indicators, the operator can modify the operating parameters. For example, the aforementioned sampling may reveal that the temperature is insufficient. Along with other data, the computer will indicate to the operator that more coke needs to be added to the furnace (the amount of coke is an operating parameter).

[0011] In many other scenarios, a computer can act directly with the machine through the functions of a process controller (e.g., a control loop for temperature parameters).

[0012] However, measurement data is not always available for various reasons and limitations. Some of these deserve further attention: • Certain sensors may become disconnected, or data connections to the computer may fail. • Due to differences in the development stages of the machines, the distribution of sensors may differ from machine to machine, even if the machines are substantially the same type. For example, in the case of furnaces, they are composed of individual units or very small series. Therefore, when viewed as a group of furnaces, they are disparate. Consequently, even furnaces with the same operating principle (e.g., thermal control by coke feeding and / or coal injection) and the same materials (e.g., iron ore, coke) will differ (e.g., in terms of volume). Such differences between machines can also lead to a lack of measurement data. Human-machine interaction plays a role. There are further technical issues involving hazards or potential dangers to the operator. Opportunities to open the furnace for the aforementioned sampling should be limited to minimize occurrences. Several measurements may be required during critical furnace operating procedures, such as measuring the temperature of the molten material during casting. Human involvement in measurements should be minimized so that the operator can fully concentrate on the operation of the furnace. Some of these constraints come in combination; with older analog thermometers, the operator needs to read the data, whereas with digital thermometers, this is not necessary.

[0013] If the relationships between measurement data are known, a computer can calculate indicators of a machine's operating state. For example, a car's computer might tell the driver to shift gears. However, in many cases, the relationships between measurement data are too complex to be modeled mathematically.

[0014] Using a neural network is one option. The network receives measurement data and provides metrics. Simply put, a network contains so-called neurons arranged in multiple layers. The interconnections between neurons are managed by so-called weights. A set of weights within a network can be called a configuration.

[0015] However, the network needs to be trained (to set the weights and other network parameters). Again, simply put, training requires feeding the network measurement history data.

[0016] However, historical data cannot be used for certain machines, especially if the machine does not have suitable counterparts or peers. This lack of historical data is particularly pronounced for machines (such as furnaces) that were manufactured at different development stages. Further background information can be found in the following paper: LI XIANG et al.: “Diagnosing Rotating Machines With Weakly Supervised Data Using Deep Transfer Learning”, IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, IEEE SERVICE CENTER, NEW YORK, NY, US, vol.16, no.3, July 8, 2019 (2019-07-08), pp. 1688-1697. This paper discusses data-driven machine diagnostics. A challenge exists in that sufficient training data is not available for machines within the target domain. However, it is possible to use training data from multiple source domains instead. This paper describes an approach that adapts training data using transfer learning. LI XIANG et al.: “Domain generalization in rotating machinery fault diagnostics using deep neural networks”, NEUROCOMPUTING, ELSEVIER, AMSTERDAM, NL, vol. 403, May 8, 2020 (2020-05-08), pp. 409-420. This paper also discusses data-driven machine diagnostics. There is a task of extracting domain-general or domain-invariant features, and this paper describes an approach using domain adversarial networks. LIAO YIXIAO et al.: “Deep Semisupervised Domain Generalization Network for Rotary Machinery Fault Diagnosis Under Variable Speed”, IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, IEEE, USA, vol.69, no.10, May 6, 2020 (2020-05-06), pp. 8064-8075. This paper discusses a rotating machine and the use of neural networks for fault detection. However, labeled training data can be used for one rotation speed but not for another. Nevertheless, this challenge can be overcome by using a Wasserstein Generative Adversarial Network. [Overview of the project]

[0017] Industrial machines operate with multiple process parameters, but they may lack appropriate sensors for certain parameters, or measurement data for those process parameters may be unavailable for other reasons. A neural network provides a parameter index corresponding to the process parameters of that machine. The network is trained based on historical data from reference machines. At least one reference machine (source reference) has sensors for its process parameters.

[0018] The reference machines are different, and from a machine learning perspective, they belong to different domains. Training them requires transfer learning, or more precisely, unsupervised domain adaptation.

[0019] By implementing a computer-implemented method, the computer trains a neural network to enable post-processing of multivariate measurement time series with measurement data. The measurement data represents instances of specific process parameters of an industrial machine. The trained network is then trained to provide parameter indices for further process parameters of the industrial machine.

[0020] In the first receiving step, the computer receives a first multivariate measurement time series containing measurement history data from a first reference machine, and a second multivariate measurement time series containing measurement history data from a physically different second reference machine.

[0021] In the acquisition step, the computer acquires a set of transformation rules by processing the first and second multivariate measurement time series so that a transformer module can transform the first and second multivariate measurement time series into the first and second multivariate feature time series respectively according to the transformation rules. The multivariate feature time series is invariant to the differences in the domains of the first and second reference machines.

[0022] In the transformation step, the computer transforms the first multivariate measurement time series into the first multivariate feature time series by means of a transformer module that applies the transformation rules.

[0023] In the second reception step, the computer receives a univariate measurement time series with measurement data of further process parameters from the first reference machine.

[0024] In the training step, the computer trains a neural network with the first multivariate feature time series at the input of the network , and and the univariate measurement time series at the output of the network Using

[0025] Optionally, the computer can use a discriminator module that operates until the adversarial loss reaches a maximum value and then determines that it can no longer distinguish the data origin.

[0026] Optionally, the computer can use a discriminator module that operates until the adversarial loss reaches a maximum value and then determines that it can no longer distinguish the data origin.​

[0027] Optionally, the computer can repeat the transformation rule acquisition and training steps, during which the transformer module receives network loss provided by the neural network being trained, so that the iteration stops in response to the minimum network loss.

[0028] With any choice, the combination of transformer and discriminator modules operates as a generative adversarial network.

[0029] Optionally, the computer receives a multivariate measurement time series, obtains a set of transformation rules, transforms it, and trains a neural network on the multivariate measurement time series, for which the data selector module has determined that the variables are associated with further parameters.

[0030] Optionally, the data selector module is trained by a selector trainer module that instructs the computer to repeatedly perform the acquisition, transformation, and training steps using a criterion for minimizing the loss function during neural network training.

[0031] Optionally, the computer receives first and second multivariate measurement time series with historical data, after adapting the data modality of the measurement data.

[0032] Optionally, the computer can adapt the data modality by at least one of the following: analyzing measurement data with sound samples by categorizing sound samples, and analyzing measurement data with images by categorizing images.

[0033] At will, the computer iteratively adapts to different goals selected while minimizing the loss function during training of the neural network.

[0034] By choice, the computer will use a neural network, specifically a regression neural network.

[0035] Optionally, the measurement data is associated with process parameters, which are physical properties selected from temperature, pressure, chemical composition of materials within the machine, the visual appearance of parts of the industrial machine, and sounds emitted from the industrial machine.

[0036] At will, industrial machines can be selected from reactors, metallurgical furnaces, vessels, and engines.

[0037] Once the neural network is trained, the computer (or a physically separate computer) can perform a computer-based execution of the neural network, which involves receiving a multivariate measurement time series containing measurement data from an industrial machine under surveillance, converting the multivariate measurement time series into a multivariate feature time series, and operating the network to provide parameter indices.

[0038] At will, the neural network will operate under the following conditions, which are selected from: sensor failure, data connection failure, and minimizing human involvement for measurement.

[0039] A computer program product, when loaded into the memory of a computer system and executed by at least one processor of the computer system, causes the computer system to perform steps(s) of the computer's execution method.

[0040] A computer system containing executable modules for multiple computers can perform multiple steps of the computer execution method when implemented by the computer system. Or, from another perspective, the computer system performs multiple steps of the computer execution method.

[0041] Furthermore, there is the use of computers that act as virtual sensors to obtain parameter indicators representing the parameters of industrial machines. [Brief explanation of the drawing]

[0042] Next, embodiments of the present invention will be described in detail with reference to the accompanying drawings. [Figure 1] This document presents an industrial machine equipped with multivariate measurement time series, as well as a neural network with a configuration to explain the notation rules. [Figure 2] This shows industrial machines and computer modules in the preparation stage for obtaining a set of conversion rules. [Figure 3] This shows the standard industrial machine, transformer module, and network used during the training phase. [Figure 4] Figure 1 shows the industrial machine and Figure 3 shows the transformer module in operation to provide parameter indicators. [Figure 5] This document provides flowcharts illustrating neural network training methods and their use in industrial machines during operation. [Figure 6] This shows an arbitrary modality adaptation module that performs modality adaptation of any selected measurement data. [Figure 7] This shows an arbitrary selection data selector module that allows you to select related measurement data arbitrarily. [Figure 8] This shows an example of its use when the industrial machine is a blast furnace. [Figure 9]Here are some examples of general-purpose computing devices that can be used with the technologies described here. [Modes for carrying out the invention]

[0043] Terms and Conditions Summary Figure 1 shows an industrial machine 123 with a multivariate measurement time series 223, and a neural network 373 with a configuration 283, to illustrate the writing conventions.

[0044] The diagram also shows—though merely symbolically—that industrial machine 123 performs an industrial process with process parameters 173 / 183. The process parameters can be divided into two groups: • Measurement parameter 183 (or x-parameter) and • 173 non-measured parameters (or z-parameters, at least one)

[0045] The attribute "measurement" is associated with a specific industrial machine 123, meaning that the measurement data is included in the multivariate measurement time series 223 for that machine 123.

[0046] No measurement data is available (i.e., unavailable) for at least one process parameter z. In principle, parameter z is a measurable parameter, but the data is unavailable. The output of neural network 373 provides a parameter indicator Z', which is a representation of that particular process parameter z. The parameter indicator Z' can be considered an "artificial measurement" or artificial or virtual measurement data. In other words, the computer emulates a sensor.

[0047] The parameter index Z' functions as a status index for machine 123 (Z' indicates the status), or as an input variable for a computer or other module that determines the machine's status (Z' is a contributor to the status). The explanation will focus on calculating Z'.

[0048] To the operator 193 (human) of industrial machine 123, network 373 appears as a virtual sensor that outputs a parameter index Z'. In a theoretical and ideal situation, index Z' would represent a physical entity, just like an actual sensor (hardware sensor, physical sensor).

[0049] In actual, non-ideal situations, the accuracy of the parameter index Z' (calculated by network 373) would be defined as the difference between the index Z' from network 373 and the measured data Z from a suitable hardware sensor.

[0050] The explanation describes the approach to determining Z' (Figures 1-5) and, in relation to (Figures 6-7), describes optional measures to improve accuracy.

[0051] An industrial machine 123 (or industrial equipment) is a machine that continuously carries out an industrial process. The above example was described in the explanation, but this explanation will focus on the example where machine 123 is a blast furnace.

[0052] Considering this blast furnace example, the process parameter z could be the temperature of the molten material. Although there are no available direct measurements (as mentioned above, either because a temperature sensor is not found or for other reasons), the parameter index Z' provides a calculated value. Z' could then be further processed to become a status index (for example, if the furnace is ready for casting, since temperature and other criteria allow it).

[0053] Several method steps, including machine learning, must be performed beforehand so that network 373 can provide Z'. For convenience, the explanation describes each step within the context of each stage. The explanation distinguishes each component that is active in the Preparation phase**1 (see Figure 2), the Training phase**2 (see Figure 3), and the Operation phase**3 (see Figure 4). In other words, throughout this explanation, references indicated as **1 / **2 / **3 represent similar but different components in these stages. For example, machine 121 in Figure 2 and machine 123 in Figure 4 are physically identical machines, but they operate as a single reference machine (or as a single machine under supervision / control).

[0054] In this explanation, stages **1 / **2 / **3 are distinguished by considering the network (which provides the parameter index Z'). With this in mind, training occurs in stage **2**.

[0055] However, other training loops may exist. • Preparation stage**1 may optionally include a training loop (see Figure 5, with repeating loops for substeps in obtaining rule 421). • Training can also be applied to modality adaptations with loops spanning **1 and **2 (see Figure 6). • Further training loops can be applied at a larger time granularity, as several steps of **1 and **2 are repeated (see Figure 7). • To train the discriminator and transformer, an additional training loop can be applied by including the network loss (LOSS) as a parameter.

[0056] The process parameters being measured (or at least emulated) are denoted by lowercase letters such as "x" or "z" (symbolized on the left side within machine 123).

[0057] The diagram shows this as parameters x_1 (which could be the temperature of a specific location within the machine, e.g., the top) and x_2 (which could be the temperature of another location, the bottom). There are N process parameters {x_n}N and at least one further parameter z. The underscore "_" is a placeholder.

[0058] Measurement data is indicated by uppercase letters such as X and Z. The parameter index Z' is indicated by a prime symbol (') because it represents de facto measurement data or artificial measurement data.

[0059] The uppercase convention also applies to the intermediate data "Y" (or "feature data," which is not measured and does not have output data).

[0060] Timeline Data (measurement data, intermediate data) is available in the form of a time series, that is, a series of data values ​​indexed in chronological order at subsequent points in time.

[0061] This diagram presents a time series using a short notation (rectangle 223) and a matrix.

[0062] The notation {X1...XM} represents a single (i.e., univariate) time series with a measured data element Xm (or simply "element"). This element Xm is available from time point 1 to time points M: X1, X2, ..., Xm, ... XM (i.e., the "measured time series"). The index m is the time point index. Time point m is usually followed by time points (m+1) at equal intervals Δt.

[0063] For example, there is a temperature rise over M time points: {1400°C...1500°C}. Those skilled in the art can pre-process the data values, for example, into normalized values ​​[0,1] or {0.2...1}. The format of the measurement data is not limited to scalars or vectors; {X1...XM} can also represent a series of M images taken from time point 1 to time point M. If the machine is a blast furnace, the images can be taken from inside the furnace tuyeres or from the direct environment of the blast furnace. The images are taken to detect anomalies.

[0064] The notation {{X1...XM}}N represents a multivariate time series (reference 223 above the matrix in Figure 1) with data element vectors {X_m}N from time point 1 to time point M. The vector has cardinality N (the number of variables, i.e., the number of process parameters for which the measurement data is available), meaning that at any given time point from 1 to M, N data elements are available. The matrix shows the variable index n as the row index (x_1 to x_N).

[0065] For example, a single time series for temperature can be accompanied by other time series such as measurement data for pressure or the chemical composition of a material.

[0066] Those skilled in the art will understand that this explanation is simplified. In reality, the number of variables N can reach or exceed several thousand. Time series are not ideal. In some cases, elements may be missing, but skilled individuals can handle such situations.

[0067] The choice of time interval Δt and the number of time points M varies depending on the process. The total time series duration Δt*M (i.e., the window size) corresponds to the longest-running process parameter shift. For example, in a furnace, processing the material takes several hours, so Δt*M is selected accordingly.

[0068] Conveniently, the interval Δt can be selected within the range of 1 to 60 minutes, and conveniently, Δt*M can be selected within the range of 4 to 24 hours. In the blast furnace example, Δt=15 minutes and M=32 can be applied to correspond to an 8-hour work shift.

[0069] Since time tm specifies the time for network processing, some measurement data can be preprocessed. For example, a temperature sensor may provide data every minute, but at Δt=15 minutes, some data will be discarded, averaged over Δt, or subjected to other preprocessing.

[0070] Since the data may originate from different industrial machines, the data origin is sometimes indicated by 1, 2, etc., before {} or {{...}}. In the example in the figure, 2{{...}} indicates that the multivariate time series came from machines 1, 2, and 3.

[0071] As explained, the computer applies a transformation from X to Y. The resulting values ​​can also be a multivariate time series. Cardinality is denoted by K.

[0072] reference As used here, reference 1** refers to an industrial machine in the real world (physical world), reference 2** refers to data (such as configuration 283), reference 3** refers to hardware and computer execution modules (such as neural network 373), and reference 4** or 5** refers to a method step (see Figure 5).

[0073] computer The term "computer" (singular, without reference) refers to the functions of computing power or computer-implemented modules (such as processing units and their time-sharing resources). These functions can be distributed across different physical computers, in which case the computer-implemented modules are distributed across those different computers.

[0074] The diagram also shows a computer program or computer program product. A computer program product—when loaded into the computer's memory and executed by at least one of the computer's processors—performs steps of the execution method by the computer. In other words, the program provides instructions to modules.

[0075] From another perspective, the diagram illustrates a computer system module that, when executed by a computer system, contains multiple computer-executable modules that perform steps in a computer execution method. Industrial machines are not considered computer-executable modules.

[0076] Machine learning tools and their configurations The diagram also symbolically represents neural network 373, which has neurons arranged in layers (multiple neurons within a single perpendicular line) (circle symbol). Neural network 373 is an example of a machine learning tool. Some of the other modules described here also use machine learning.

[0077] For such networks, there are two basic descriptors: • Hyperparameter set and Configuration.

[0078] The hyperparameters are, • Define a function to be performed by the neuron (for example, sum weighted inputs and apply a threshold or other function). • Define the function that the combined neurons will execute (for example, the selection of a kernel function between convolution kernels, Gaussian kernels, etc.). • Define the arrangement of neurons within a layer and the interconnections between layers (e.g., full inter-layer connections, such as between layer 1 and layer 2).

[0079] Hyperparameters are typically described as part of the network architecture in this technical field. Hyperparameters are not trained.

[0080] Configuration 283 defines weighted data transmission from neuron to neuron and defines bias (or offset) values ​​for the neurons. Configuration 283 is obtained through training.

[0081] Neural network 373 is illustrated here as an example only. Figures 6-7 show further tools (such as applying data modalities, selecting relevant data, and using data transformers) for applying architecture and configuration conventions to it.

[0082] Domain differences between machines Industrial machine 123 (Figure 1) is not the only machine available for its measurement data. Other industrial machines are also available for the same measurement data, hereinafter referred to as "peer machines." Peer machines operate according to the same principles, but may differ in some respects.

[0083] In relation to these differences, industrial machines can be considered to belong to the first and second domains.

[0084] One of the peer machines can provide additional measurement data for training the network. In this explanation, machines 111 / 112 are referred to as the source (which provides the actual measurement data Z for parameter z), and machine 123 (which the network uses to calculate the index Z') is referred to as the target. The peer machine that provides training data is also called the "reference machine." Both the source machine and the target machine are reference machines in the sense that they can provide the measurement data used in the preparation phase**1. One of them (the source machine) has a sensor that the other (the target machine) does not. The source machine is the source of the data used in the training phase**2.

[0085] Domain differences between reference machines are characterized by different operational modes. Such operational differences are induced by differences in machine parameterization or by different environments that affect the underlying processes on which the machines operate.

[0086] In the example of a blast furnace, two peer machines process the same metal, use the same materials (such as ore or coke), and have sensors located in comparable positions for the same parameters.

[0087] However, the first furnace may be larger (difference in size) or have a slightly different shape than the second furnace, which may result in slightly different measurements of the volume of molten material, melting time, etc. The difference in size is more pronounced in furnaces of different heights (see Figure 8 for an example).

[0088] Different domains result in different measurement data for the same equivalent parameters.

[0089] The differences in domains are also a result of applying different measurement methods.

[0090] Due to differences in domains, computers apply data transformations.

[0091] To illustrate, let's briefly mention music. Both organ pipes and blast furnaces produce sound through the movement of air. Longer organ pipes play at lower frequencies than shorter ones. However, human listeners have learned to recognize melodies, which are sequences of tones with different relative frequencies. Melodies are invariant with respect to absolute frequencies.

[0092] Since the sound from the furnace can be used as measurement data, the melody is separated from the pitch (highness of the sound) through conversion. In other words, a properly adapted transformer provides a melody (domain invariant) that can be received from the first and second furnaces, receiving two time series 1{X1...XM} and 2{X1...XM}, and further processed (by comparison, etc.).

[0093] Data Modality For the sake of simplicity, the explanation initially assumes that the measurement data have the same data modality. Time-series measurement data is assumed to contain only scalars. Any modality adaptation will be discussed in Figure 6.

[0094] Regarding the determination of the parameter index Z' To further simplify the explanation, the explanation first assumes that all parameters x_1 through x_N may also affect parameter z. Therefore, the computer processes all of them. At any discretion, relevant differences can be considered, and these are illustrated in detail with Figure 7.

[0095] Steps to generate the configuration The following describes the steps that one or more computers perform to obtain the configuration. The diagrams are explained in the order of the steps. However, it is helpful to view Figures 2-4 starting from the end.

[0096] During this operation (see Figure 4), the industrial machine 123 provides measurement data to the neural network 373, which provides the parameter index Z'.

[0097] Network 373 can provide Z' for the following reasons: Configuration 283 was acquired earlier in the training phase (Figure 3) by processing measurement history data from a virtually complete but industrial machine 112 (for which appropriate data about z is available), which has data about z. More specifically, time series 212 and 272 contain common measurement data 1{{X1...XM}}N and additional data 1{Z1...ZM}. Network 373 with configuration 283 cannot directly process the measured time series 223 with data 2{{X1...XM}}N, but transformer 323 converts the time series 223 with data 2{{X1...XM}}N into an intermediate time series 243 2{{Y1...YM}}K.

[0098] Transformer 323 can provide intermediate time series 243 for the following reasons: During the preparation phase [see Figure 2], the predecessor transformers 311 / 321 of transformer 323 are configured to convert the time series {{X1...XM}}N to the intermediate time series {{Y1 ...YM}}K. The configuration of transformers 311 / 321 is established by discriminator 361, which optimizes common transformation rules so that the intermediate time series are no longer specific to the domain differences between machines 111 and 121, which are physically different.

[0099] Training network 372 (see Figure 3) to obtain configuration 283 is possible for the following reasons: • The network during training can receive measurement history data (from peer machines, in this case from machine 112) because transformer 322 can transform that data. Time series 1{Z1...ZM} does not need to be transformed.

[0100] Origin of measurement data To keep the display simple, Figures 2-4 show the measurement data arriving from the sensors. However, some measurement data may arrive via human-machine interfaces, etc. For convenience, the sensors are referred to as providing measurement data X1...XN, and also Z, within small round-shaped square symbols.

[0101] Figure 2 shows an industrial machine and computer module in the preparatory stage**1 for obtaining a set of conversion rules (421).

[0102] Industrial machines 111 and 121 each have sensors X1 to XN (common sensors, for the reason of providing data on common process parameters). Machine 111 has a sensor Z (in bold box) that provides a univariate measurement time series 211 1{{X1...XM}}N, but machine 121 does not have such a sensor.

[0103] Information measurement data Z is "exchanged" between peers, albeit indirectly, during the establishment of rules. Machine 111 is the "source," and machine 121 is the "target." Machine 111 is the only machine equipped with sensor Z.

[0104] Parameter equivalence Process parameter xn is available for both reference machines (1xn for the source and 2xn for the target; see the matrix in Figure 1). For example, both machines have a temperature parameter for the top of the furnace. Process parameter x(n+1) is, for example, the temperature at the bottom, and is available for both machines.

[0105] In other words, process parameters that can be used on both reference machines are equivalent parameters. Equivalent parameters are characterized by equal variate index n.

[0106] Equivalent parameters lead to equivalent measurement data.

[0107] Sensor Xn of machine 111 ("source") and sensor Xn of machine 121 ("target") are equivalent because the resulting measurement data both refer to the same parameter, for example, the upper temperature sensor. Sensor Xn of machine 111 and sensor X(n+1) of machine 121 are not equivalent.

[0108] The parameter is singular (e.g., top temperature), but the measurement data is multifaceted (e.g., two furnaces).

[0109] Equivalence of sensors Sensors from different machines that provide measurement data for equivalent parameters are equivalent sensors. Equivalent sensors do not need to operate according to the same principle.

[0110] For example, both the (source) sensor Xn and the (target) Xn provide the temperature of the top gas. Xn (the source machine) does this by an invasive approach (moving a bar with the sensor attached over the ore), while Xn (the target machine) does this above the ore (i.e., above the furnace charge) by a sound transmitter and receiver.

[0111] Historical data Generally, training machine learning tools (such as networks) requires historical data. The operation of the Transformer 311 / 321 is as follows: • Multivariate measurement time series 211, 1{{X1...XM}}N) with measurement history data from reference machine 111, and • A second multivariate measurement time series 221, 2{{X1...XM}}N), with measurement history data from a physically different second reference machine 121. It begins with receiving it.

[0112] The diagram shows receipt indicated by arrows, but a person skilled in the art would store the time series in a database. No further explanation is needed here.

[0113] However, simply feeding {{X1...XM}}N to a training tool (such as a transformer for learning rules) is insufficient for M*N data elements (see the matrix in Figure 1). There are two possible choices:

[0114] Firstly, historical data could come from multiple physical machines 111 / 112 (not just from two machines, but from more machines). This arbitrary selection is simply symbolically represented by the repetition of dashed lines in the box. However, in practical terms, such an approach (heterogeneous fleet) is not available in the reactor.

[0115] Secondly, physical machines 111 / 112 repeatedly provide data. The second option is preferred.

[0116] Figure 2 shows multiple {{...}}N (and {{...}}K as well) represented by two overlapping circular rectangles. The figure also shows the arrow Q. Further notation is {{{...}}}Q.

[0117] Q is the number of multivariate measurement time series. It would be convenient if all machines had the same Q. However, in practice, this is not always possible. A machine may be "younger" (have a smaller Q) than its peer machine (which has a larger Q).

[0118] In the example described above (Δt*M = 8 hours), the machine collects a multivariate measurement time series with Q = 3*365 over a year. In other words, Q can have a magnitude of 1000 per year. It is not necessary to consider all available data. Measurement history data may not be available because the machine may be in maintenance or repair mode.

[0119] An 8-hour time slot is convenient for illustration. However, computers can apply a window slot T_WINDOW to compute the features of a multivariate time series. Such a window slot can be defined as a moving time window with a duration of several hours (e.g., 8 hours). The window slot can move over overlapping data collected over longer periods (e.g., more than a year) using T_OVERLAP. A favorable relationship between T_WINDOW and T_OVERLAP is, for example, T_WINDOW > T_OVERLAP (or equal to). It is convenient if T_OVERLAP is between 1 and 3 hours and T_WINDOW is around 6 to 10 hours.

[0120] Removing domain differences from historical data Despite the aforementioned equivalence (in parameters and sensors), historical data still reflects differences in domain.

[0121] The computer obtains a set of transformation rules 251 (Figure 2) / 252 (Figure 4) by processing the multivariate measurement time series 211 and 221 (step 421 in Figure 5). Transformation rules 251 / 252 enable transformers 311 / 321 to transform the multivariate measurement time series 211 and 221 ({{...}}) into multivariate feature time series 231 and 241, respectively. These feature time series 1{{Y1...YM}}K and 2{{Y1...YM}} are invariant to domain differences in reference machines 111 / 121.

[0122] In other words, domain-invariant features have already been extracted.

[0123] As mentioned earlier, we use multiple multivariate measurement time series (see Q), and the transformation does not change that.

[0124] Transformers 311 / 321 are shown here as two boxes. This is because time series 211 is transformed into time series 231, and time series 221 is transformed into time series 241. It does not matter whether the transformations are performed in parallel (as shown in the diagram) or sequentially. Both transformers use the common rules 251 / 252.

[0125] The rules are acquired through training. There is no need to involve human experts here. In other words, domain adaptation (having features instead of measured data) is unsupervised. Expert annotations or labels are not required. Optionally, experts can manually define some rules based on their existing expertise. Manually defining some rules further ensures the convergence of the training.

[0126] Conversion rules Since the network is trained using training data from different domains, transfer learning is involved.

[0127] Simply put, there exists a transformation that converts the first and second multivariate measurement time series 211 and 221 into the first and second multivariate feature time series 231 and 241, respectively. The multivariate feature time series 231 and 241 are invariant to the differences in domains of the reference machines 111 and 121.

[0128] In other words, it extracts features that are no longer domain-specific due to the transformation. The measurement time series can be considered to belong to the first space (or input space), and the feature time series can be considered to belong to the second space (or feature space). The input space is sensitive to domain differences, but the feature space is not.

[0129] There is a presumed causal relationship between space X and space Y, which represents a contiguous space for regression.

[0130] The marginal distributions of both domains (source=s, target=t) are similar: ps(ys|T(xs))≈pt(yt|T(xt)), where ys is the output data of the source domain ("s") and yt is the output data of the target domain ("t").

[0131] The transformation rules follow the principle Y=T(X). More specifically, transformers 311 and 321 operate according to a time interval Δt: at t=1, transformer 311 transforms data vector 1{X1}N into data vector 1{Y1}K, and transformer 321 transforms data vector 2{X2}N into data vector 2{Y2}K. They then transform 1{X2}N into 1{Y2}K and 2{X2}N into 2{Y2}K at t=2, and so on until t=M.

[0132] The variates of {{X1...XM}}N are not necessarily identical to the variates of {{Y1...YM}}K, and the variate cardinality (usually N > K) can be changed by transformation.

[0133] Here, the transformation rules are introduced to operate on data at time tm (vertically, a processing data vector relative to the time interval tm), but at any choice, the rules can be extended to process data from preceding time points (m-1), (m-2), etc. Those skilled in the art are familiar with such recurrent networks.

[0134] Since the conversions in this field have already been explained, those skilled in the art can select from existing conversion schemes.

[0135] For convenience, the following further papers will be discussed: · Hochreiter, et al. “Long Short-Term Memory” Neural Computation November 1997, https: / / doi.org / 10.1162 / neco.1997.9.8.1735(1997) · Ashish Vaswani et al., “Attention Is All You Need”, Advances in Neural Information Processing Systems 30, NIPS 2017, (2017) · Anastasia Borovykh et al., “Conditional Time Series Forecasting with Convolutional Neural Networks”, arXiv:1703.04691(2017) · Aaron van den Oord et al., “WaveNet: A generative model for raw audio”, arXiv:1609.03499, (2016)

[0136] Features of multimode multivariate time series can be extracted by algorithms such as recurrent neural networks (Hochreiter), which may be combined with convolutional neural networks, or transformers (Vaswani) or wavenets (Borovykh, van den Oord), although these are not the only such methods.

[0137] The transformation function T can be identified by minimizing the dissimilarity of the two conditional probability distributions ps(t(xs)|ys)≈pt(t(xt)|yt).

[0138] A transformation function t() for defining domain-invariant features is typically found by training an adversarial deep learning model. During training, the transformation is performed to minimize the loss of the regression model predicting the output, ultimately generating features that are indistinguishable between the source and target domains.

[0139] The following publications provide further background information. Y. Ganin et al., “Domain-adversarial training of neural networks,” Journal of Machine Learning Research, vol. 17, no. 59, pp. 1-35, (2016)

[0140] Examples of Transformer training This explanation assumes that the transformer has already acquired rules 251 / 252. Below, we describe any optional approach to acquiring the rules by applying adversarial (deep) learning.

[0141] This approach can be summarized by two method steps presented as substeps of the rule acquisition 421 (see Figure 5). Acquisition 421 of the set of transformation rules 251 / 252 can be performed by a computer repeating the following:

[0142] In conversion step 431, the computer uses a set of preliminary conversion rules to convert the (first) multivariate measurement time series 211 to the (first) multivariate feature time series 231, and the (second) multivariate measurement time series 221 to the (second) multivariate feature time series 241. The details have already been explained above.

[0143] In the discriminating step 441, the computer uses the discriminator module 361 to distinguish (or "identify") the origins of the first and second feature time series 231, 241 as originating from the (first) and (second) multivariate measurement time series 211, 221. In the first iteration, the discriminator 361 can "easily" identify them. Figure 2 symbolically illustrates this with the learning time (total time for iterations). On the left (early learning), the discriminator distinguishes X and Y from either 1 (first machine / domain) or 2 (second machine / domain). On the right, the discriminator cannot.

[0144] In modification step 451, the computer modifies the set of preliminary translation rules (or modifies the rules of the current iteration).

[0145] The computer repeats steps 431, 441, and 451 until the discriminator 361 can no longer identify the data origin (i.e., from 1 or 2). To determine that the data origin can no longer be identified, the discriminator 361 can calculate the adversarial loss ADLOSS. The discriminator 361 continues to operate (and modify the rule) until ADLOSS reaches its maximum value.

[0146] Conversion rules 251 / 252 will be the final rules applied in the next step.

[0147] In this particular approach, rules are acquired through a learning process that includes paradoxically emerging learning objectives.

[0148] In other words, the combination of transformers 311 / 321 and discriminator 361 is trained (with settings modified in accordance with the rules) to classify (or "predict") whether a feature is associated with a source machine or a target machine.

[0149] In other words, the same encoder (or transformer with rules prepared during iteration) is shared between the source domain and the target domain (or the first and second reference machines) and the encoder that is expected to extract domain-invariant representations with the help of an adversarial discriminator (i.e., discriminator 361).

[0150] For further details, please refer to the following paper. · Zhang et al., “Domain-Invariant Adversarial Learning for Unsupervised Domain Adaptation”,arXiv:1811.12751,(2018) · Xie et al., “Learning Semantic Representations for Unsupervised Domain Adaptation@, Proceedings of the 35th International Conference on Machine Learning, PMLR 80:5423-5432, (2018) · Avisek Lahiri et al., “Unsupervised Domain Adaptation for Learning Eye Gaze from a Million Synthetic Images: An Adversarial Approach”, In Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing (2018)

[0151] Adversarial training is further outlined in: Goodfellow, Ian J., Pouget-Abadie, Jean, Mirza, Mehdi, Xu, Bing, Warde-Farley, David, Ozair, Sherjil, Courville, Aaron C., and Bengio, Yoshua. Generative adversarial nets. NIPS, 2014.

[0152] The acquisition of rules through a learning approach belongs to the preparation stage**1. This is not the same as training the network, which will be described next.

[0153] Figure 3 shows the reference industrial machine 112, the transformer module 322 (abbreviated as "transformer 322"), and the network 372 (in training) during training phase**2.

[0154] Transformer 322 transforms the multivariate measurement time series 212 into a multivariate feature time series 323 (step 452 in Figure 5). Time series 212 may be the same as time series 211 (see Figure 2). Transformer 322 applies transformation rule 252 (see rule 251 in Figure 2 for confirmation). As symbolized by multiple overlapping rectangles, transformer 322 processes multiple multivariate measurement time series (see Q introduced in Figure 2).

[0155] The computer receives a univariate measurement time series 272 (i.e., 1{Z1...ZM}) from the reference machine 112, which includes measurement data (Z) of further process parameters (z), usually multiple series (step 462 in Figure 5). In this sense, the industrial machine 112 serves as a reference providing measurement data for actual measurements (from sensors X1 through XM, and from sensor Z, etc.).

[0156] Network 372 takes the first multivariate feature time series (232) as input. , and At the output, there is a univariate measurement time series (272). UsingTraining is underway (Step 472 in Figure 5).

[0157] Those skilled in the art can perform training using appropriate hyperparameters, optimization goals, etc. For example, training may be terminated when the loss function reaches a specific predefined value. During training, the intermediate calculated value LOSS (or "network loss") changes until it reaches a minimum value, and as a result, the weights and bias values ​​can be carried over to the configuration. In other words, LOSS is a control parameter of the network's internal processing loop.

[0158] LOSS can also be used to control loops outside of network 372. The figure shows that the intermediate calculated value LOSS can be provided by network 372. As explained with respect to Figure 7, LOSS can be used in further loops by the selector trainer 380. In other words, LOSS can also be a control parameter for one or more network-external processing loops.

[0159] Furthermore, LOSS can be provided to the transformer (see Figure 2) as a parameter to optimize the transformation rules.

[0160] Figure 4 shows the industrial machine 123 (in Figure 1) and transformer module 323 (see Figure 3) in operation stage**3 to provide the parameter index Z'. Figure 4 is an updated version of Figure 1, which has been described in detail above.

[0161] The computer receives a multivariate measurement time series 223 (also denoted as 2{{X1...XM}}N) containing measurement data for the industrial machine 123 (see step 513 in Figure 5). Here, machine 123 is a machine under surveillance and does not have sensors to measure parameter z.

[0162] Transformer 323 converts the multivariate measurement time series 223 into a multivariate feature time series 243 (similarly 2{{Y1...YM}}K).

[0163] The network receives a multivariate feature time series 243 and provides a parameter index Z'. Network 373 is illustrated with configuration 283.

[0164] Note that the time series is a single time series (i.e., a specific work shift, etc.), regardless of Figures 2 and 4.

[0165] Figure 5 shows flowcharts of the computer execution method 400 for the train neural network 372 / 373 and the computer execution method 500 for using the network during the operation of the industrial machine 123.

[0166] Method 400 is a computer-based method for training a neural network 373. The network 373 is trained with measurement data representing instances of specific process parameters (see Figure 1, 183) of an industrial machine (see Figure 1, 123) in order to later process multivariate measurement time series (see Figure 1, 223) and thereby provide a parameter index Z' of further process parameters (see Figure 1, 173, z) for the industrial machine.

[0167] In receiving step 411, the computer receives a first multivariate measurement time series with measurement history data from a first reference machine and a second multivariate measurement time series with measurement history data from a physically different second reference machine (1{{X1...XM}}N) and 2{{X1...XM}}N), see left side of Figure 2.

[0168] In the obtaining step 421, the computer obtains a set of transformation rules 251 / 252 by processing the first and second multivariate measurement time series, thereby enabling the transformer (see 311 / 321 in Figure 2) to transform the first and second multivariate measurement time series into the first and second multivariate feature time series, respectively, according to transformation rules 251 / 252 (see 231, 241 in Figure 2, also denoted as 1{{Y1...YM}}K, 2{{Y1...YM}}K). The multivariate feature time series are invariant to the domain differences between the first and second reference machines (see 111, 121 in Figure 2) (the two reference machines belong to different domains and have the details described).

[0169] In transformation step 452, the computer transforms the first multivariate measurement time series into the first multivariate feature time series using a transformer (322 in Figure 3) that applies transformation rules.

[0170] In the receiving step (462), the computer receives a univariate measurement time series containing measurement data Z of further process parameters z from the first reference machine (see 112 in Figure 3).

[0171] In the 472 steps of training the neural network 372, the computer trains the network using a first multivariate feature time series as the network input and a univariate measurement time series as the network output.

[0172] Figure 5 also shows details (dashed boxes) of any selection for obtaining a set of transformation rules 421. The computer repeats steps 431, 441, and 451.

[0173] In transformation step 431, the computer uses a set of transformation rules, which are preliminary rules, to transform the first multivariate measurement time series into the first multivariate feature time series and the second multivariate measurement time series into the second multivariate feature time series.

[0174] In the identification step 441, the computer runs the discriminator 361 to distinguish (or identify) the origins of the first and second feature time series as the first and second multivariate measurement time series.

[0175] In modification step 451, the computer modifies the set of preliminary translation rules.

[0176] The iterative step stops when the discriminator (361) can no longer identify the data origin (i.e., the loop repeats until or while).

[0177] The details of the steps in Method 400 will be explained in the context of this description.

[0178] For a computer-based execution method using a network during the operation of an industrial machine, see Figure 4, which illustrates the steps involved.

[0179] In the receiving step 513, the computer receives a multivariate measurement time series containing measurement data from the industrial machine under surveillance; in the conversion step 523, the multivariate measurement time series is converted into a multivariate feature time series; and in the operation step 533, the network 373 is operated to provide the parameter index Z'.

[0180] Please note again that the computers used for methods 400 and 500 may be physically different computers. The common data will be configuration 283 (obtained during training).

[0181] Improved accuracy As briefly explained above, the accuracy of the index Z' corresponds to the difference between it and the measured data Z (if available). Simply put, the smaller the difference, the higher the accuracy.

[0182] Here, we will explain two approaches to improving accuracy. One approach (adaptation) increases the number of variables being processed, while the other approach (selection) decreases the number of variables.

[0183] At first glance, the two approaches may seem contradictory, but they are related. For convenience, the explanation will introduce them separately, and then discuss their relationship later.

[0184] Data modality adaptation Up to this point, we have described an approach to measuring time series data that contains only scalars, where {X1...XM} is a sequence of M consecutive scalar values. It is irrelevant whether the computer represents scalars as integers or real numbers.

[0185] This explanation discusses how to adapt measurement data from other modalities to the processing.

[0186] Figure 6 shows an optional modality adaptation module 393 (or "adapter") that performs modality adaptation of measurement data of any choice. For example, the figure shows the adapter 393 during operation phase**3, but the adapter 393 may operate similarly in other phases as well.

[0187] Adapter 393 receives a multivariate measurement time series (also denoted as {{X1...XM}) 213 from an industrial machine (shown as machine 113 for example, but the data origin "1" or "2" is irrelevant). The term "receive" corresponds to method steps 411 and 513 (see Figure 5). Thus, adapter 393 can be executed as part of a module that performs the receiving.

[0188] A multivariate measurement time series {{X1...XM}}N contains a univariate measurement time series {X1...XM} which is a sequence of scalar or non-scalar elements. In this example, the univariate time series {X1...XM}n must contain a non-scalar element Xn.

[0189] As a side note, in many situations, 1{Z1...ZM} is only a scalar. However, this is not mandatory, so 1{Z1...ZM} can also be a vector, etc.

[0190] Adapter 393 passes time series containing scalars to other modules virtually unchanged. These time series don't even need to go through the adapter.

[0191] Adapter 393 processes the non-scalar elements Xn individually, but each n follows a rule. This diagram shows the non-scalar elements Xn in a highly symbolic way. This allows the computer to process the elements and assign them (by classification) scalars denoted as Xn~. ~ indicates that an assignment has been made.

[0192] In the first scenario, the non-scalar element is an image (i.e., a matrix with pixels that indicate color). For example, the image shows inspection holes of the machine taken at time intervals Δt (the exposure time for the image is negligible).

[0193] In the blast furnace example, the inspection uses tuyeres to show the furnace fire to the viewer. The fire occurs periodically during operation. In very simplified terms, an image showing the fire can be coded to 1, and an image without the fire can be coded to 0. The resulting time series is {..}~={0,0,1,1,1,0...0,0}~, and the computer continues processing (following each step, including obtaining the rule 421 or converting 523, etc., see Figure 5, or even receiving 513).

[0194] In the second scenario, the non-scalar element is a sequence of sounds. For example, Xn could be an acoustic recording from a microphone sensor for a duration of Δt = 15 minutes. Those skilled in the art can apply appropriate sound processing to achieve 15 * 60 * 20,000 acoustic samples per second, for example, by sampling sounds with a frequency of 20 kHz.

[0195] Adapter 393 then processes these millions of samples into a single scalar. (As illustrated) the scalar can represent, for example, a pitch increase during Δt, a pitch decrease during Δt, a constant pitch during Δt, a pitch increase or decrease during Δt, and so on.

[0196] Adapter 393 can be run by a previously trained, potentially supervised (by a human expert) neural network. For example, Adapter 393 may have been trained using annotated images showing furnace fires (in a blast furnace), annotated sound sequences (obtained from a blast furnace), etc. Adapter 393 can also be trained in an unsupervised manner to enable it to identify the abstract representation of an image. This ensures that the informative content of the image is maximized. This process then reduces the sequence of images to several sequences of scalars.

[0197] Assigning a scalar does not necessarily mean that the measured data is associated with the index Z'. The figure shows that adaptations using differently trained networks can result in different scalars, Xn~ and Xn~~, for the same input Xn. "6" and "7" in the figure are included for illustrative purposes. The results when different scalars are used for the same input are discussed below.

[0198] Relevance Selection Some process parameters are not associated with parameter z. In other words, the entire set of process parameters (x_1 to x_N, with cardinality of N variables) can be divided as follows: • The first subset of H process parameters x_1 to x_H associated with parameter z ("associated parameter set"), and • A second subset of (NH) process parameters not associated with parameter z.

[0199] To digress far from the topic and take music as an example, at least for the listener, it doesn't matter whether the melody is played in "largo" or "allegro"; the melody is the same.

[0200] In an ideal scenario, network 373 would be trained to simply ignore the measured data for the second subset of parameters. In a realistic training scenario, some measured data (from the second subset) may still contribute to the index Z'.

[0201] It should be noted that training is possible without the participation of human experts. However, it is not necessary to introduce such supervision into the training.

[0202] Industrial machine experts can identify relevant parameter sets based on their experience. Only relevant measurement data identified by the experts is sent to the computer. Involving experts in this regard is a one-time effort.

[0203] For example, the power consumption of the furnace does not affect the temperature (the exemplary z-parameter), and the consumption data can be ignored (by not processing the measurement of consumption).

[0204] In addition to manual pre-selection and its alternatives, relevance identification can be performed by computer by selecting measurement data. An example is given below.

[0205] Figure 7 shows the arbitrary selection data selector module 383 (or simply "selector") which performs arbitrary selection of relevant measurement data. The figure shows the selector 383 at the top during operation stage**3, and describes its conditioning in stages**1 and**2 by repeating several method steps from Figure 5.

[0206] During operation, selector 383 receives the multivariate measurement time series 223 (see FIGS. 1 and 3, also denoted as {X1...XM}N) and sends the modified multivariate measurement time series 223 ({{X1...XM}}H) to the transformer 323 (see FIG. 4).

[0207] The modification is a selection (i.e., an active selection of H variables from N variables to {{...}}H with H < N). In other words, this corresponds to the selection of the relevant parameter set (at the measurement data level). The figure illustrates the selection by showing that there are fewer sensor symbols (compared to the machine) in the output of selector 383.

[0208] By executing method 500 (see FIG. 5) with the reduced measurement data set, the accuracy of providing the metric Z' can be improved. This approach may seem counterintuitive, but the selector blocks some of the data to be processed by the network while the accuracy improves. However, the selection arranges the measurement data that the network 373 cannot potentially ignore and may potentially degrade the accuracy. Here again, the unsupervised nature of network training (see FIGS. 2 - 4) is noted.

[0209] Selector 383 can be configured by a learning process that uses the loss function of network 372 / 373 as a value for minimization. This approach is symbolically represented by selector trainer module 380 (or simply "selector trainer").

[0210] Selector trainer 380 communicates with other modules such as network 372 (receiving LOSS during training) and executes the method symbolically represented by the flowchart.

[0211] During the iterations until the intermediate value LOSS—in other words, the loss function—has a minimum value ("minimum loss", YES), the selector trainer 380 modifies the (preliminary) selection ({{...}}H∈{{...}}N) and causes the constituent(s) to perform method 400, which means receiving 411 (of any choice), obtaining rules 421, transforming 452, receiving 462 (of any choice), and training 472 (see Figure 4). Receiving is of any choice because the received data is not modified.

[0212] In other words, the preliminary selection of variables {{...}}H is replaced with other selections until the loss is minimized for the final selection.

[0213] Therefore, although the explanation is simplified, those skilled in the art can implement this method. For example, it is not necessary to repeat the receiving step 462 (because 1{Z1...ZM} is not excluded from the selection).

[0214] When the loss function reaches its minimum, the operations described in Figures 1-5 are performed using H instead of N (i.e., during operation stage 3 as described for the top of the figures).

[0215] In any choice, the operation of selector 383 can be enhanced by selecting several variables in a measurement time series and combining or merging them into the resulting measurement time series. For example, selector 383 can select {...}1, {...}2, and {...}3 (such as temperatures from three locations) and combine them to form a new univariate measurement time series (to become part of {{...}}H).

[0216] Those skilled in the art can perform selection and combination using principal component analysis.

[0217] Essay The modules that perform data modality adaptation (see Figure 6) and relevance selection (see Figure 7) operate independently of each other in stage**3.

[0218] However, to conserve computing resources, it is convenient to prioritize the data selector 383 over the modality adapter 393, at least during operation phase 3. For example, it is not necessary to process the acoustic recording to be scalar (see Figure 6) and then filter out the scalar using selector 383.

[0219] Adapting to a scalar allows the network to process more data, although some adaptations may result in errors. Assigning a scalar allows the adapter to apply networks that have been trained differently. Some results may be related to Z', while others are not. For example, analyzing a sound sequence into categories such as raised / lowered / constant pitch may not be related to Z', but analyzing it into other categories such as vibrating / stable sound may indeed be related to Z'. In this sense, scalars Xn~ and Xn~~ obtained in different ways (see Figure 6) can be treated as relevance selection (see Figure 7).

[0220] Network loss feedback The explanation now returns immediately to Figures 2-3. As already explained, the transformation rules are obtained in the preparation stage**1 so that the transformer provides a feature time series that is invariant to domain differences. Adversarial loss (ADLOSS, see Figure 2) is the criterion.

[0221] To further improve accuracy, the transformation rules can be enhanced by considering network loss (LOSS, see Figure 3). This allows the computer to operate within a loop that includes preparation phase**1 and training phase**2 (which is performed as preliminary training).

[0222] In other words, the steps of obtaining a set of transformation rules for the neural network 372 (step 421) and training 472 are repeated. The transformers 311 / 321 further receive the network loss (LOSS) provided by the network during training. The iteration can be stopped when the network loss is minimized.

[0223] To illustrate an arbitrary approach, Figure 2 shows the receipt of LOSS (loss) with a dashed line. Adversarial loss (ADLOSS) is maximized, while LOSS (loss) is minimized. Those skilled in the art can apply other conventions.

[0224] Usage - Examples Figure 8 shows an example of use where industrial machines 111, 121, and 123 are blast furnaces. There are multiple temperature parameters (symbolized with different values ​​from 200°C to 2000°C) and other process parameters (pressure values, material chemical composition, etc.).

[0225] The domain differences are illustrated in the following two examples, where the size difference is symbolized by furnace 111 being smaller than furnaces 121 and 123. Furnaces 111 and 121 apply different measurement methods to the temperature of the molten material (molten iron temperature). A remote sensor 151 (camera symbol) in furnace 111 yields a time series with samples available at Δt (e.g., 15 minutes). Manual measurements by the operator in furnace 121 yield temperature values ​​available at larger intervals (e.g., 90 minutes), as described in the background of this discussion.

[0226] In this exemplary use case, the virtual sensor 153 acquires an index Z' (for temperature) at Δt. The camera symbol is shown as a small dot, but this virtual sensor represents the computer executing method 500 (see Figure 5). In other words, the figure generally illustrates the use of a computer performing (or executing) method 500 (and being trained according to method 400) to obtain a parameter index Z' representing the parameters of an industrial machine. The use of a computer performing method 500 can be considered equivalent to the use of a virtual sensor.

[0227] Furnaces 121 and 123 may be physically identical, but furnace 123 is a future operational unit, and one network 372 / 373 has completed training. This diagram does not show the operator of furnace 123 taking samples, thus demonstrating that this use case helps minimize human involvement in measurements and allows the operator to focus more on the operation of the furnace. In other words, furnace 121 can be upgraded to furnace 123 with additional virtual sensors.

[0228] By using virtual sensors, it becomes possible to compare existing industrial machines by measuring process parameters for which data would otherwise not be available.

[0229] Use of artificial measurement data as training data The indicator Z' can be used as supplemental historical data and as training data.

[0230] General-purpose computer Figure 9 shows an example of a general-purpose computing device usable with the technology described herein. Figure 9 is a schematic diagram showing an example of a general-purpose computing device 900 and a general-purpose mobile computing device 950 usable with the technology described herein. Computing device 900 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. General-purpose computing device 900 corresponds to computer system 100 in Figure 1. Computing device 950 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, driver assistance systems, or vehicle board computers (e.g., vehicles 401, 402, 403, see Figure 1), and other similar computing devices. For example, computing device 950 can be used as a front-end by a user (e.g., a blast furnace operator) to interact with computing device 900. The components shown herein, their connections and relationships, and their functions are for illustrative purposes only and are not intended to limit the implementation of the inventions described and / or claimed herein.

[0231] The computing device 900 includes a processor 902, memory 904, storage device 906, a high-speed interface 908 connected to memory 904 and a high-speed expansion port 910, and a low-speed interface 912 connected to a low-speed bus 914 and storage device 906. Each of the components 902, 904, 906, 908, 910, and 912 can be interconnected using various buses and mounted on a common motherboard or in other ways as appropriate. The processor 902 can process instructions to be executed within the computing device 900, including instructions stored in memory 904 or storage device 906, to display graphic information for the GUI of an external input / output device such as a display 916 coupled to the high-speed interface 908. In other executions, multiple processors and / or multiple buses may be used as appropriate, along with multiple memories and multiple types of memory. Also, multiple computing devices 900 may be connected to each other to partially provide the necessary operation (e.g., as a server bank, a group of blade servers, or a multiprocessor system).

[0232] Memory 904 stores information within the computing device 900. In one execution, memory 904 is a volatile memory unit (one or more). In another execution, memory 904 is a non-volatile memory unit (one or more). Memory 904 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0233] The storage device 906 can provide a large-capacity storage device to the computing device 900. In one execution, the storage device 906 may be or include computer-readable media such as a floppy disk drive, a hard disk drive, an optical disk drive, or a tape drive, a storage device using flash memory or other similar semiconductor memory, or an array of devices including devices in a storage network or other configuration. A computer program product can be materially embodied in the information carrier. A computer program product may also include instructions that perform one or more of the above-described methods when executed. The information carrier is a computer or machine-readable medium such as memory 904, storage device 906, or the memory of the processor 902.

[0234] The high-speed controller 908 manages bandwidth-intensive operations for the computing device 900, while the low-speed controller 912 manages lower bandwidth-intensive operations. Such function assignments are illustrative only. In one execution, the high-speed controller 908 is coupled to memory 904, a display 916 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 910 that can accept various expansion cards (not shown). In this execution, the low-speed controller 912 is coupled to storage device 906 and the low-speed expansion port 914. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, or network device such as a switch or router, for example, via a network adapter.

[0235] The computing device 900 can run in a number of different forms, as shown in the figure. For example, it can run as a standard server 920, or multiple times within a group of such servers. It can also run as part of a rack server system 924. In addition, it can run in a personal computer such as a laptop computer 922. Alternatively, components from the computing device 900 can be combined with other components in a mobile device (not shown), such as device 950. Each such device may contain one or more computing devices 900, 950, and the overall system may consist of multiple computing devices 900, 950 communicating with each other.

[0236] The computing device 950 includes, among other elements, a processor 952, memory 964, input / output devices such as a display 954, a communication interface 966, and a transceiver 968. Device 950 may also have storage devices such as a microdrive or other devices to provide additional storage. Each component 950, 952, 964, 954, 966, and 968 are interconnected using various buses, and some components may be mounted on a common motherboard or in other ways as appropriate.

[0237] The processor 952 can execute instructions within the computing device 950, including instructions stored in memory 964. The processor may run as a chipset of chips including separate and multiple analog and digital processors. The processor may provide coordination of other components of the device 950, such as control of the user interface, application operation by the device 950, and wireless communication by the device 950.

[0238] The processor 952 may communicate with the user via a display interface 956 and a control interface 958 coupled to the display 954. The display 954 may be, for example, a TFT LCD (thin-film transistor liquid crystal display) or an OLED (organic light-emitting diode) display, or other suitable display technology. The display interface 956 may include suitable circuitry for driving the display 954 to present graphics and other information to the user. The control interface 958 may receive commands from the user and translate them for provision to the processor 952. In addition, an external interface 962 may communicate with the processor 952 to provide near-field communication between device 950 and other devices. The external interface 962 may provide, for example, wired communication in some executions or wireless communication in others, and multiple interfaces may also be used.

[0239] Memory 964 stores information within the computing device 950. Memory 964 can function as one or more computer-readable media, volatile memory units, or non-volatile memory units. Extended memory 984 is also provided and can be connected to device 950 via an expansion interface 982, which may include, for example, a SIMM (Single In-Line Memory Module) card interface. Such extended memory 984 may provide additional storage space for device 950, or may also store applications or other information for device 950. Specifically, extended memory 984 may include instructions for executing or supplementing the processes described above, and may also include secure information. Therefore, for example, extended memory 984 may function as a security module for device 950 and may be programmed with instructions that enable secure use of device 950. In addition, secure applications may be provided via a SIMM card along with additional information, such as placing identification information on the SIMM card in a hack-proof manner.

[0240] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one execution, the computer program product is embedded in the information carrier in a manner that can be perceived as an entity. The computer program product includes instructions that, when executed, perform one or more of the methods described above. The information carrier is a computer or machine-readable medium such as memory 964, extended memory 984, or the memory of processor 952, which can receive information via, for example, transceiver 968 or external interface 962.

[0241] Device 950 can communicate wirelessly via a communication interface 966, which may include digital signal processing circuitry as needed. The communication interface 966 can provide communication under various modes or protocols, including, among others, GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA®, CDMA2000, or GPRS. Such communication may occur, for example, via the radio frequency transceiver 968. In addition, short-range communication may occur using Bluetooth, WiFi, or other such transceivers (not shown). Furthermore, the GPS (Global Positioning System) receiver module 980 may provide the device 950 with additional navigation and location-related radio data, which may be used as appropriate by the application running on the device 950.

[0242] Device 950 can also communicate audibly using an audio codec 960 that can receive voice information from the user and convert it into usable digital information. The audio codec 960 can also generate audible sound for the user, for example, through a speaker in the handset of device 950. Such sound may include sounds from voice telephone calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications running on device 950.

[0243] The computing device 950 can run in a number of different forms, as shown in the figure. For example, it can run as a mobile phone 980. It can also run as part of a smartphone 982, a personal digital assistant, or other similar mobile device.

[0244] The various implementations of the systems and technologies described herein can be realized in digital electronic circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include execution in one or more computer programs that are implementable and / or interpretable in a programmable system that includes a memory system, at least one programmable processor coupled to receive and send data and instructions from and to at least one input device and at least one output device.

[0245] These computer programs (also known as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be executed in a high-level procedural and / or object-oriented programming language and / or assembly / machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and / or device (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, and include machine-readable medium that receives machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0246] To provide user interaction, the systems and technologies described herein can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (e.g., a mouse or trackball) that allows the user to input into the computer. User interaction can also be provided using other types of devices; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and user input can be received in any form, including acoustic, voice, or tactile input.

[0247] The systems and technologies described herein can be run on computing devices that include backend components (e.g., data servers), middleware components (e.g., application servers), or frontend components (e.g., client computers having a graphical user interface or web browser through which a user can interact with the implementation of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the systems can be interconnected by digital data communications (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the internet.

[0248] Computing devices can include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network. The client-server relationship arises from computer programs running on each computer that have a client-server relationship with each other.

[0249] Several embodiments have been described. Nevertheless, it will be understood that various modifications can be made without departing from the essence and scope of the present invention.

[0250] Furthermore, the logic flow depicted in the diagram does not require a specific order or sequence shown to obtain the desired result. In addition, other steps may be provided, or steps may be removed from the described flow, and other components may be added to or removed from the described system. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A computer-based method (400) for training a neural network (373), wherein the network (373) is trained for post-processing of a multivariate measurement time series (223) with measurement data representing instances of specific process parameters (183) of an industrial machine (123), and thereby provides a parameter index (Z') of further process parameters (173, z) for the industrial machine (123), The system receives a first multivariate measurement time series (211, 221) containing measurement history data from a first reference machine (111) and a second multivariate measurement time series (221) containing measurement history data from a physically different second reference machine (121) (411); The transformer module (311 / 321) obtains a set of transformation rules (251 / 252) by processing the first and second multivariate measurement time series (211, 221) so that the first and second multivariate feature time series (231, 241), respectively, are invariant to the domain differences of the first and second reference machines (111, 121), according to the transformation rules (251 / 252) (421); The transformer module (322) that applies the transformation rule (251 / 252) transforms the first multivariate measurement time series (211, 221) into the first multivariate feature time series (232) (452); A univariate measurement time series (272) is received from the first reference machine (112), with measurement data (Z) of further process parameters (173, z); and The neural network (372) is trained (472) using the first multivariate feature time series (232) as input and the univariate measurement time series (272) as output. A method including (400).

2. Obtaining the set of conversion rules (251 / 252) (421) is, Using a set of preliminary transformation rules, the first multivariate measurement time series (211) is transformed into the first multivariate feature time series (231) (431), and the second multivariate measurement time series (221) is transformed into the second multivariate feature time series (241) (431), The discriminator module (361) identifies the origins of the first and second feature time series (231, 241) as the first and second multivariate measurement time series (211, 221) (441), and To change the set of preliminary conversion rules (451), The method (400) according to claim 1, which is performed by a computer repeatedly until the discriminator module (361) can no longer identify the data origin (1, 2).

3. The discriminator module (361) determines that the data origin can no longer be identified by operating until the adversarial loss reaches its maximum value, according to claim 2.

4. The method according to claim 2, wherein the steps of obtaining a set of transformation rules (421) and training a neural network (372) (472) are repeated, during which the transformer module (311 / 321) receives further network loss (LOSS) provided by the neural network (372) being trained, and the iteration stops in accordance with the minimum network loss.

5. The method (400) according to any one of claims 1 to 4, wherein the combination of the transformer module (311 / 321) and the discriminator module (361) operates as a generative adversarial network.

6. The method according to any one of claims 1 to 5 (400), wherein the method steps of receiving a multivariate measurement time series (211, 221) (411), obtaining a set of transformation rules (421), transforming (452), and training a neural network (472) are performed on a multivariate measurement time series in which a data selector module (383) has determined that the variables are associated with further process parameters (173, z).

7. The method (400) of claim 6, wherein the data selector module (383) is trained by a selector trainer module (380) which causes a computer to repeatedly perform the method steps of acquiring, transforming, and training using a criterion for minimizing a loss function during the training (482) of a neural network.

8. The method (400) according to any one of claims 1 to 7, wherein the receipt (411) of first and second multivariate measurement time series (211, 221) having historical data is performed after the data modality of the measurement data has been adapted.

9. The method (400) of claim 8, wherein the adaptation of the data modality is performed by a step selected from: analyzing measurement data on sound samples by categorizing sound samples; and analyzing measurement data on images by categorizing images.

10. The method (400) according to claim 8 or 9, wherein the adaptation is repeated for different goals selected while minimizing the loss function during training (482) of the neural network.

11. The neural network (372) is a recurrent neural network, according to the method (400) of any one of claims 1 to 10.

12. The method (400) according to any one of claims 1 to 11, wherein the measurement data is associated with process parameters, which are physical properties selected from temperature, pressure, chemical composition of materials in the machine, visual appearance of a part of the industrial machine, and sound emitted from the industrial machine.

13. The method (400) according to any one of claims 1 to 12, wherein the industrial machine is selected from a reactor, a metallurgical furnace, a vessel, and an engine.

14. A computer execution method (500) for operating a neural network (373) that has been previously trained by any one of claims 1 to 13, A multivariate measurement time series (223) containing measurement data of an industrial machine under surveillance (123) is received (513); The multivariate measurement time series (223) is converted to a multivariate feature time series (243) (523), and A neural network (373) is operated (533) in a way (500) at each step to provide a parameter index (Z').

15. The method (500) according to claim 14, wherein the operation of the neural network occurs under circumstances selected from sensor failure, data connectivity failure, and minimizing human involvement for measurement.

16. A computer program product that, when loaded into the memory of a computer system and executed by at least one processor of the computer system, causes the computer system to perform one or more steps of the computer execution method described in any of claims 1 to 15.

17. A computer system comprising a plurality of computer-based execution modules that, when implemented by a computer system, perform a plurality of steps of a computer-based execution method described in any one of claims 1 to 15.

18. A method for obtaining a parameter index (Z') representing a parameter (z) of an industrial machine (123), comprising performing the method described in any one of claims 14 to 15.

Images