Method for predicting temporal course of physical target variable by means of machine learning model
By representing sensor data segments as predefined dimensions and utilizing a machine learning model with an attention layer, the prediction problem of irregular sensor data is solved, improving prediction accuracy and computational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2025-11-03
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies struggle to effectively process irregularly sampled multivariate sensor data, resulting in insufficient accuracy of machine learning models in predicting temporal changes in physical target parameters.
Sensor data is divided into segments, and a corresponding sensor data segment representation is determined for each segment. A machine learning model with predefined dimensions is used for prediction. The model includes a transformer model and an attention layer. The attention layer is used to process heterogeneous sensor data to improve prediction accuracy.
It enables efficient and accurate prediction of the temporal changes of physical target parameters under irregular sensor data conditions, reducing computational costs and training time.
Smart Images
Figure CN121996937A_ABST
Abstract
Description
Existing technology
[0001] For various technical (e.g., physical or chemical) processes, it may be worthwhile to predict the time-varying processes of physical parameters based on multivariate time-series data of other physical parameters, and / or to predict anomalies based on multivariate time-series data of multiple physical parameters. For example, it may be worthwhile to predict the state-of-health of a fuel cell or hydrogen loading based on the time-varying processes of current intensity and voltage, or in the case of a drilling rig, to predict which material is being drilled based on the time-varying processes of current intensity and voltage, or to predict anomalies based on the time-varying processes of current intensity and voltage, etc. Typically, for this purpose, a machine learning model can be trained for exactly one use case (e.g., predicting the health status of a fuel cell). Summary of the Invention
[0002] This disclosure relates to a method for predicting the temporal variation of physical target parameters using a machine learning model based on multivariate sensor data, wherein the multivariate sensor data may be irregularly sampled sensor data.
[0003] If sensor data from different sensors are detected, these sensor data may have different sampling rates. Data points may also be missing in some sensor data (e.g., due to measurement errors or because these data points were removed due to excessive uncertainty). The time intervals in which the sensor data are present may also have different durations. Intuitively, it may appear that not every data point in the first sensor data can be bijectively assigned to a data point in the second sensor data, which is different from the first sensor data.
[0004] The method described in this paper enables the prediction of temporal variations in physical target parameters even with irregular sensor data. This is achieved, for example, by dividing the sensor data into segments and then determining a corresponding segment representation for each segment, which has the same predefined dimension across all segments. Therefore, the dimension of the segment representation is independent of the regularity of the data points within the segment (e.g., sampling rate, the presence of data points, etc.).
[0005] The machine learning models described in this paper can also be trained for predictions of corresponding physical target parameters for multiple different tasks, which have at least partially different physical parameters. Thus, physical laws applicable across different tasks can be learned efficiently, for example. Such training is made possible by the method described in this paper's ability to handle irregular, multivariate sensor data.
[0006] Different aspects relate to a method for predicting the temporal variation of physical target parameters using a machine learning model. This method includes: providing multivariate sensor data, the multivariate sensor data being assigned to a time period and having corresponding sensor data for each of a plurality of physical parameters, these corresponding sensor data representing the temporal variation of the physical parameter within the time period, wherein each physical parameter is associated with a corresponding text description, the text description describing the physical parameter (and optionally also describing the measurement environment in which the corresponding sensor data was detected); for each of the plurality of physical parameters: dividing the corresponding sensor data into corresponding plurality (e.g., non-overlapping) sensor data segments; for each of the plurality of sensor data segments: determining... A corresponding sensor data segment representation, which represents the sensor data segment and has a predefined dimension (independent of the number of data points in the sensor data segment), determines the corresponding input elements using the corresponding sensor data segment representation, time-related location information, and a corresponding textual description of the physical parameter, wherein the time-related location information represents the position of the sensor data segment within the time period (e.g., temporally); in response to inputting all input elements and at least one target parameter query into the machine learning model, the machine learning model predicts the temporal variation of the physical target parameter, wherein the target parameter query represents the position of the predicted temporal variation within the time period (e.g., temporally) and the textual description of the physical target parameter.
[0007] The following describes different embodiments.
[0008] Example 1 is a method described above for predicting the time-varying process of physical target parameters using a machine learning model.
[0009] Example 2 is set up according to Example 1, wherein at least one physical parameter has at least two sensor data segments with different numbers of data points.
[0010] Since each sensor data segment is mapped to a corresponding sensor data segment representation with a predefined dimension, all sensor data segment representations have that predefined dimension regardless of the dimension of the sensor data segment. Thus, sensor data segments can have different dimensions (e.g., duration, number of data points (e.g., due to different sampling rates), scalar values, and even no values at all). Intuitively, this method can also predict the temporal variation of target parameters for heterogeneous, multivariate sensor data.
[0011] Example 3 is set up according to Example 1 or 2, where the time-related location information represents the start and end times within the time period.
[0012] Because the method described in this paper enables the implementation of a different number of data points for each sensor data segment, the duration can be described using time-related location information in addition to the start time point (e.g., by the end time point).
[0013] Example 4 is set up according to any one of Examples 1 to 3, wherein the machine learning model has a transformer model whose encoder and / or decoder has an attention layer to which all input elements (i.e., each input element for each physical parameter) are fed.
[0014] Because all input elements (rather than only input elements in the physical parameter dimension or only input elements in the time dimension) are fed to the attention unit, the machine learning model is able to consider more complex correlations (e.g., based on previous training), thereby improving prediction accuracy. This also allows the use of heterogeneous sensor data elements, such as scalar values combined with time series and / or missing values.
[0015] Example 5 is established according to any one of Examples 1 to 4, wherein a corresponding sensor data segment representation for a sensor data segment is determined by means of a (multi-head) attention unit having a learned, sensor data segment-specific parameter vector as a query and having the sensor data segment as a key and as a value; and / or wherein a corresponding input element is determined using the corresponding sensor data segment representation, the corresponding position representation, and the corresponding text representation of the physical parameters, wherein the position representation is determined by means of a (multi-head) attention unit having a learned, position-specific parameter vector as a query and having time-related position information as a key and as a value.
[0016] Example 6 is set up according to any one of Examples 1 to 5, wherein the machine learning model has a transformer model, and one or more attention layers in the encoder and / or decoder of the transformer model have (multi-head) attention units to which the target parameter query is fed.
[0017] Therefore, for example, no trained free parameters are needed as input, allowing the machine learning model to determine predictions with reduced computational cost. Furthermore, no such free parameters need to be trained during training, thus reducing the computational cost of training (and consequently the time required for it). Since the target parameter query has a textual description of the physical target parameters, the accuracy of the prediction is significantly improved.
[0018] Example 7 is a method for controlling a technical (e.g., physical or chemical) process, the method comprising: predicting the time-varying process of a physical target parameter according to one of Examples 1 to 6, using provided multivariable sensor data; and controlling the technical process in consideration of the prediction.
[0019] Example 8 is a control device configured to implement the method described according to Example 7.
[0020] Example 9 is a system having: an apparatus configured to perform a technical process; one or more sensors for detecting multivariable sensor data; and a control apparatus for controlling the technical process as described in Example 8.
[0021] Example 10 is a data processing unit configured to implement the method according to any one of Examples 1 to 6.
[0022] Example 11 is a computer program having instructions that, when executed by a processor, cause the processor to perform the method according to any one of Examples 1 to 7.
[0023] Example 12 is a computer-readable medium that stores instructions that, when executed by a processor, cause the processor to perform the method according to any one of Examples 1 to 7. Attached Figure Description
[0024] In the accompanying drawings, similar reference numerals generally refer to the same portions in all different views. The drawings are not necessarily to scale, whereby the emphasis is generally placed on the representation of the principles of the invention. In the following description, different aspects are described with reference to the following drawings.
[0025] Figure 1 A flowchart illustrating methods for predicting the time-varying process of physical target parameters based on different aspects; Figure 2 An exemplary system is shown, on which the method can be implemented; Figure 3 The illustration shows the time-varying process of the detected physical parameter and the time-varying process of the physical parameter to be predicted; Figure 4 This illustrates the determination of input elements based on different aspects; Figure 5 This demonstrates predictions of the temporal variation of physical target parameters using machine learning models from different perspectives. Figure 6 This illustrates attention layers with single-layer attention based on different aspects; and Figure 7 This illustrates an exemplary query-based forwarding approach used for two layers of attention. Detailed Implementation
[0026] The following detailed description relates to the accompanying drawings, which illustrate specific details and aspects of this disclosure for illustrative purposes, in which the invention may be practiced. Other aspects may be used and structural, logical, and electrical modifications may be made without departing from the scope of protection of the invention. Different aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure may be combined with one or more other aspects of this disclosure to form new aspects.
[0027] The different examples are described in more detail below.
[0028] Figure 1 A flowchart is shown for a method 100 for predicting the time-varying process of physical target parameters based on different aspects.
[0029] Method 100 may (in 102) include: providing multivariate sensor data, which is assigned to a time period and has corresponding sensor data for each of a plurality of physical parameters, the corresponding sensor data representing the temporal variation of that physical parameter within that time period. Each physical parameter may be associated with a corresponding text description, which (e.g., textually) describes the physical parameter and the measurement environment in which the corresponding sensor data was detected.
[0030] Method 100 may (in 104) include: for each of a plurality of physical parameters, dividing the corresponding sensor data into a plurality of (e.g., non-overlapping) sensor data segments. Then, method 100 may further include: for each of the plurality of sensor data segments, determining a corresponding sensor data segment representation, which represents the sensor data segment and has predefined dimensions; and determining corresponding input elements using the corresponding sensor data segment representation, time-related location information representing the sensor data segment's position within that time period (e.g., temporally).
[0031] Method 100 may (in 106) include: in response to inputting all input elements and at least one target parameter query into a machine learning model, using the machine learning model to predict the temporal variation of the physical target parameter, the target parameter query representing the position of the temporal variation to be predicted within the time period (e.g., temporally) and a textual description of the physical target parameter.
[0032] This method can be executed by one or more computers having one or more data processing units. The term "data processing unit" can be understood as any type of entity capable of processing data or signals. For example, data or signals can be processed according to at least one (i.e., one or more) specific functions performed by the data processing unit. The data processing unit may include integrated circuits such as analog circuits, digital circuits, logic circuits, microprocessors, microcontrollers, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable gate arrays (FPGAs), or any combination thereof, or constructed from these. Any other means of implementing the corresponding functions described in more detail herein can also be understood as a data processing unit or a logic circuit device. One or more method steps in the method steps described in detail herein can be implemented (e.g., executed) by the data processing unit through one or more specific functions performed by the data processing unit.
[0033] Therefore, this method varies depending on the implementation scheme, especially computer implementation.
[0034] Figure 2 System 200 is illustrated according to various aspects. System 200 may have device 202 configured to perform a technical process. According to various aspects, device 202 may be robotic equipment (hereinafter referred to as "robot"), such as industrial robots in the form of robotic arms for moving, assembling, or processing workpieces, for bin picking, manufacturing robots, maintenance robots, household robots, medical robots, vehicles (e.g., at least partially automated vehicles), household appliances, hand tools (e.g., drilling rigs), production machine tools, personal assistants, access control systems, etc., and any other type of robotic equipment. According to various aspects, the technical process may be a physical or chemical process, such as a manufacturing process (e.g., manufacturing a product or intermediate product), a processing process (e.g., processing a workpiece), a control process (e.g., moving a robotic arm), a setup process (e.g., calibrating measuring instruments), etc.
[0035] System 200 may have a control device 204 configured to control technical processes (e.g., according to one or more control parameters 206). The term "control device" (also referred to as "control apparatus") can be understood as any type of logical implementation unit that may include, for example, circuitry and / or a processor capable of executing software, firmware, or a combination thereof stored in a storage medium, and can—in this example, issue commands to an actuator. The control device may be configured, for example, via program code (e.g., software) to control the operation of system 200.
[0036] Depending on the context, a multivariate time series of sensor data (i.e., multivariate sensor data) can be detected over a time period. Intuitively, the multivariate sensor data 210 (d = 1 to P) can represent the corresponding temporal variation of a physical parameter d within a time period for each of P physical parameters (where P can be any integer greater than or equal to one). In this case, the sensor 208(d) used to detect the sensor data can be, for example, a temperature sensor, a concentration sensor for detecting one or more elements, a pressure sensor, etc. The sensor data of physical parameters can be not only output parameters of the technical process but also input parameters applied to control the technical process according to one or more control parameters 206, such as applied voltage and / or (e.g., current intensity generated by the applied voltage). The sensor data of physical parameters can be detected in-situ or ex-situ. Characteristics of the manufactured product can be detected (as sensor data) after the technical process has been implemented (e.g., ex-situ). Therefore, it is understood that multivariate sensor data can have a time series of physical parameters associated with the technical process in any way.
[0037] Depending on various aspects, control device 204 may be configured to implement machine learning model 212. Machine learning model 212 may be configured to predict, using multivariate sensor data 210 (d = 1 to D), a temporal variation process 214 of (at least one) physical target parameter (e.g., an undetected variation). Control device 204 may be configured to match one or more control parameters 206 (i.e., control the technical process) taking into account the predicted temporal variation process 214 of the physical target parameter. Depending on various aspects, control device 204 may be configured to identify anomalies based on the predicted temporal variation process 214 of the physical target parameter and correspondingly control the technical process (e.g., stop the technical process and output a signal to the user of device 202 notifying them of the anomaly).
[0038] In the following description, different aspects of method 100 are illustrated in more detail with respect to technical system 200.
[0039] Figure 3 The diagram illustrates the detected time variation process 210(d) of the physical parameter d during a time period, the detected time variation process of the physical target parameter d* during a segment of the time period, and the predicted time variation process 214 of the physical target parameter d*.
[0040] In 104, the sensor data for each physical parameter d can be divided into one or more (e.g., a large number) (e.g., disjoint) sensor data segments. Intuitively, the time variation process 210(d) of each physical parameter d can be subdivided into one or more time intervals. In this case, This can specify the number of sensor data segments, which can be greater than or equal to one. Depending on the aspect, the number of sensor data segments... It can have varying numbers of data points. The number of data points can also be referred to as the number of time points, where each time point is associated with a data point. Therefore, each sensor data segment... Can be assigned to time segments The time segment has a start time point and an end time point within the time period of the sensor data. This can also be called time-related location information because it describes the location within a time period. Such time segments can be represented, for example, by multi-dimensional feature vectors. For instance, each physical parameter d can be assigned a time-related location vector, or a time-related location vector can be assigned to each physical parameter d. .
[0041] For clarity, in various respects, the physical parameters among the plurality of physical parameters are referred to as channels or channel dimensions c. Each physical parameter d may be associated with a corresponding text description TB. This text description may (e.g., textually) describe the physical parameter d and the measurement environment in which the associated sensor data was detected. The text description of the physical parameter d described herein may include, for example, the physical parameter itself, a description of its signal, one or more pieces of information about the sensor, by means of which the sensor data was detected.
[0042] In some respects, at least one time period This could belong to the physical target parameter d*. In this case, the data points of the physical target parameter d* in this time interval 214 can be considered as missing values. In one example, multivariate sensor data can be considered as future sensor data, and the prediction of the temporal change of the physical target parameter d* can be a prediction of future changes. In other respects, for example, if a complete signal of the physical target parameter d* is to be generated, there may be no sensor data for the physical target parameter d*. This is also called a virtual sensor. Intuitively, in this case, all data points of the physical target parameter d* can be considered as missing values.
[0043] Depending on the specific aspects, each sensor data segment can be targeted. Determine the corresponding input elements . Figure 4 Showing sensor data segments based on different aspects For input elements The determination.
[0044] The representation of sensor data segments can be determined based on different aspects. This sensor data segment represents the sensor data segment. And this sensor data segment represents a predefined dimension D (i.e. (with sensor data segment) (It is unrelated to the length of time).
[0045] For example, one can use a (multi-head) (standard) attention unit (such as MSA(Q,K,V) in reference [2], which has query Q, key K and value V) to... The sensor data segment representation for the sensor data segment is determined, and the attention unit has a learned, sensor data segment-specific parameter vector. As a query and having sensor data segments As a key and as a value.
[0046] Intuitively, a specific parameter vector of the sensor data segment can be... Used in all sensor data segments In the case of a query, based on this parameter vector, the corresponding sensor data segment is retrieved. Mapped to sensor data segments with the specified dimension D superior.
[0047] Although MSA is sometimes used as a name for “Multi-Head Self Attention”—where the query Q, key K, and value V are the same (i.e., Q = K = V)—it should be understood that MSA is used in this paper for the Multi-Head Standard Attention Unit (abbreviated as: Multi-Head Standard Attention), and Q, K, and V can also be different from each other.
[0048] Such parameter vectors The learning (for training language models) is described, for example, in J. Devlin et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, arXiv:1810.04805, 2019 (hereinafter referred to as reference [1]), where the parameter vector It is called a special classification token, CLS.
[0049] Sensor data segment specific parameter vector It can have a dimension D (i.e., a parameter with a quantity D). Depending on the aspect, the dimension D described in this article can be set by the user.
[0050] Using sensor data segment representation The position to which it belongs indicates and text representation In the case of input element For example, according to It has been confirmed.
[0051] Text representation This can represent the textual description of the physical parameter d. For example, control device 204 can be configured as a text encoder for implementation. The text encoder is configured to map text descriptions (TBs) to text embeddings. Up (i.e.) Text encoder It can be, for example, an encoder that has been trained as a language model. Control device 204 can be configured for use in (e.g., learnable) (dimensional) matrix In the case of embedding text Mapping to text representation Up (i.e.) Intuitively, It is a vector associated with the textual description of the physical parameter d. (For the textual representation) The use of this makes it possible to directly apply existing models to a set of changes to physical (input and / or output) parameters.
[0052] In some respects, location representation It is possible to use the minimum time point of a predefined quantity T according to This determines whether the minimum time point is guaranteed to be available within a segment (e.g., based on the minimum segment length T). In contrast, in other respects, location representation... This can be determined using a (multi-head) attention unit (MSA), which has a learned, location-specific parameter vector. As a query and containing time-related location information As a key and as a value, i.e. In this way, location is represented. This provides additional information, thereby improving the accuracy of the machine learning model 212. For example, specific parameter vectors from the sensor data segment. In that case, a position-specific parameter vector It can have dimension D and can be used for all sensor data segments.
[0053] Figure 5This illustrates the prediction of the time-varying process 214 of the physical target parameter d* using machine learning model 212 from different aspects. .
[0054] Depending on the aspect, the machine learning model 212 may have or may be a transformer model. The transformer model may have an encoder 212-1 and a decoder 212-2. An exemplary transformer model is described in Y. Zhang et al., “Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting”, International Conference on Learning Representations ICLR, 2022, 2019 (referred to as reference [2] in this paper). However, to achieve satisfactory accuracy, the transformer model described in reference [2] requires the use of regular multivariate sensor data (i.e., the same sampling rate, no missing values, etc.), because in reference [2], all segment lengths must be the same and time information is not considered. Furthermore, the transformer model described in reference [2] can only predict time-varying processes of predefined time lengths because the position encoding is learned. In addition, (due to the lack of textual description) adaptation to other physical parameters cannot be achieved. For the sake of brevity, the differences between the converter model described in reference [2] are described in particular below, while for other aspects please refer to reference [2].
[0055] The encoder 212-1 and / or decoder 212-2 described herein may have multiple attention layers. Input element It can be fed to the first attention layer in encoder 212-1 Each attention layer It can have an attention MSA, which is based on Attention layer input elements Mapping to output vector Up. Then, the attention layer. The output vector Attention layers can be determined by following a transformer architecture that incorporates layer-normen, dropout, skip connections, and feed-forward (see reference [2] for example). Output elements The output element is then used by the subsequent attention layer. input elements Intuitively, This can be explained in an attention layer Intermediate results within, and This can be explained in two successive attention layers The result between them.
[0056] Depending on the aspect, in each attention layer of encoder 212-1 and / or decoder 212-2 It can have all the input elements that have been sent. This involves exactly one attention unit (i.e., a single-layer attention), followed by layer normalization, discarding, skipping connections, feedforward, etc. For example, the first attention layer of encoder 212-1... Send input elements In addition to the target parameter query (Q), input elements from encoder 212-1 can be fed into the first attention layer of decoder 212-2. In contrast, reference [2] uses two layers of attention (not only in the encoder but also in the decoder), in which the temporal dimension (with time) is processed in the first attention unit (i.e., the first layer). (Followed by layer normalization, discarding, skipping connections, feedforward, etc.), and the channel dimension is processed in the second attention unit (i.e., the second layer) (again, followed by layer normalization, discarding, skipping connections, feedforward, etc.). Because this paper uses a single-layer attention ( Therefore, more complex correlations between data from different sensors can be considered. For example, in the case of two-layer attention, features that depend on observations in different channels at different time intervals cannot be learned within a single attention layer (and therefore cannot be fully utilized in inference).
[0057] Figure 6 This illustrates attention layers with single-layer attention units based on different aspects. In this case, a function can be used to resolve the 604 error. All (existing) input elements 602 merge (e.g., join). Then, the join can be... It is fed to a single-layer attention 606 as an intermediate result, according to Generate output vector And then, through application layer normalization, discarding, skipping connections, feedforward, etc. (in 607), the output element is produced. 608.
[0058] Because it uses all input elements Single-layer attention to common input ( Therefore, not all sensor data needs to be time-series. For example, sensor data segments It can also involve scalar values. For example, a scalar value can be the value of a global system parameter (such as the initial charge capacity of a battery), for example, as... Figure 6 As shown, sensor data segment And therefore its input elements (in) Figure 6 The example is the input element This is also missing. This is impossible in the case of two layers of attention, because sensor data is required for each time segment (of the same length).
[0059] Then, refer to Figure 5 Based on the converter architecture, in the encoder In L After an attention layer, the sequence generated by encoder 212-1 is embedded and at least one target parameter is queried. Sent to decoder 212-2. Target parameter query. This can represent the time position of the time change process to be predicted within that time period (e.g., indicating the start and end times). And the textual description TB of the physical target parameter d*. For this purpose, for example, a positional representation can be used. and text representation In cases such as according to Target parameter query Depending on the aspect, one or more target parameter queries can be sent to decoder 212-2. Each target parameter query This can specify the corresponding temporal location and physical target parameters of the temporal change process to be predicted. As mentioned earlier, the decoder 212-2 can also have multiple attention layers. Then, decoder 212-2 can output the corresponding prediction of the time-varying process. Therefore, decoder 212-2 can, for example, be configured to determine the corresponding prediction by means of a linear projection of the output elements. (For example, when the number of data points / prediction) The duration corresponds to the time index i*. Depending on the aspect, the decoder 212-2 can implement an attention unit MSA, which will consider the time position. Used as a query and the output elements are used as values and keys to enable arbitrary prediction durations.
[0060] Depending on the aspect, similar to that in reference [2], each attention layer in encoder 212-1 and / or decoder 212-2 A forwarding (routing) mechanism can be implemented. In this forwarding mechanism, each attention unit... Divided into the first sub-unit Second subunit The first subunit according to Output intermediate features And the second subunit according to These intermediate features Used as both a key and a value. However, unlike the forwarding mechanism in reference [2], it does not learn forwarding variables (referred to as in reference [2]). (This is not a query, but a query with one or more target parameters.) Used as the first subunit The query-based forwarding mechanism can be intuitively implemented by encoder 212-1 and / or decoder 212-2. Depending on the specifics, the query-based forwarding mechanism described herein can also be implemented in encoder 212-1.
[0061] To clarify, Figure 7 The forwarding mechanism is illustrated exemplarily for a two-layer attention unit. It should be understood that this is for illustrative purposes only due to the reduced number of input elements (time dimension t in this example), and that decoder 212-2 and encoder 212-1 use a single-layer attention, as described herein.
[0062] The query-based forwarding mechanism combined with single-layer attention reduces the complexity of the machine learning model 212. Consequently, this model can be trained (or may have been trained with reduced computational cost) (because, for example, there is no need to learn forwarding variables). Integrating information about the target parameters into one or more target parameter queries (using textual descriptions and time references) also produces better embeddings, leading to higher accuracy for the machine learning model 212. The time series of sensor data can also encompass relatively long time spans, allowing the reduction in attention complexity to result in improved computational efficiency.
[0063] While the prediction of the temporal variation of physical target parameters is involved in different aspects, it should be understood that anomalies are also predicted using the machine learning models described in this paper. In one example, anomalies can also be identified based on the predicted temporal variation of physical target parameters. Anomalies can be identified, for example, by determining that the predicted temporal variation of the queried physical parameter is the same as the temporal variation of the input physical parameter, and by evaluating the reconstruction error of the input. For example, if the reconstruction error is greater than or equal to a threshold, the input can be identified as an anomaly.
Claims
1. A method (100) for predicting the time-varying process (214) of physical target parameters using a machine learning model (212), the method (100) having: Provide (102) multivariable sensor data, which is assigned to a time period and has corresponding sensor data (210) for each of a plurality of physical parameters, wherein the corresponding sensor data represents the temporal variation of the physical parameter in the time period, wherein, Each physical parameter is associated with a corresponding text description, which describes the physical parameter. For each of the plurality of physical parameters (104): • Divide the corresponding sensor data into multiple corresponding sensor data segments; • For each of the plurality of sensor data segments: o Determine the corresponding sensor data segment representation, which represents the sensor data segment, and the corresponding sensor data segment representation (independent of the number of data points in the sensor data segment) has a predefined dimension. o Determine the corresponding input element by using the corresponding sensor data segment representation, time-related location information, and corresponding text description of the physical parameter, wherein the time-related location information represents the position of the sensor data segment within the time period; In response to inputting all input elements and at least one target parameter query into the machine learning model (212), the machine learning model (212) predicts (106) the time change process (214) of the physical target parameter, wherein the target parameter query represents the position of the time change process (214) to be predicted within the time period and a textual description of the physical target parameter.
2. The method (100) according to claim 1. in, The corresponding plurality of sensor data segments of at least one physical parameter have at least two sensor data segments with different numbers of data points from each other.
3. The method (100) according to claim 1 or 2. in, The time-related location information represents the start and end times within the time period.
4. The method (100) according to any one of claims 1 to 3. in, The machine learning model (212) has a transformer model, and the encoder (212-1) and / or decoder (212-2) of the transformer model have attention layers, to which all input elements are fed.
5. The method (100) according to any one of claims 1 to 4. in, An attention unit is used to determine the corresponding sensor data segment representation for the sensor data segment, the attention unit having a learned, sensor data segment-specific parameter vector as a query and the sensor data segment as both a key and a value; and / or The corresponding input elements are determined using the corresponding sensor data segment representation, the corresponding position representation, and the corresponding text representation of the physical parameters. The position representation is determined by means of an attention unit, which has a learned, position-specific parameter vector as a query and the time-related position information as a key and as a value.
6. The method (100) according to any one of claims 1 to 5. in, The machine learning model (212) has a transformer model, and the transformer model has attention units in one or more attention layers in the encoder (212-1) and / or decoder (212-2), to which the target parameter query is fed.
7. A system (200) comprising: • Equipment (202), said equipment being configured to carry out the technical process; • One or more sensors (208) for detecting the multivariable sensor data (210); and • A control device (204) configured to predict the time-varying process (214) of the physical target parameter according to any one of claims 1 to 6 and to control the technical process in consideration of the prediction.
8. A data processing unit configured to implement the method according to any one of claims 1 to 6.
9. A computer program having instructions that, when executed by a processor, cause the processor to perform the method (100) according to any one of claims 1 to 6.
10. A computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method (100) according to any one of claims 1 to 6.