A method for predicting the time course of a physical target quantity using a machine learning model.
By segmenting sensor data with predefined dimensions and using transformer models with attention layers, the method effectively predicts the time course of physical quantities in irregularly sampled data, improving accuracy and adaptability across diverse applications.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2025-10-31
- Publication Date
- 2026-05-19
AI Technical Summary
Conventional methods struggle with predicting the time course of physical quantities using machine learning models when dealing with irregularly sampled and heterogeneous multivariate sensor data, which often includes missing data points and varying segment durations, limiting their applicability and accuracy.
The method involves dividing sensor data into segments with predefined dimensions, using transformer models with attention layers to process irregular data, and incorporating text descriptions and time-related information for enhanced prediction accuracy, allowing the model to handle diverse physical quantities and missing values.
This approach enables accurate prediction of physical target quantities even with heterogeneous sensor data, reducing computational costs and enhancing model adaptability to different tasks, while also identifying anomalies.
Smart Images
Figure 2026082745000073 
Figure 2026082745000074 
Figure 2026082745000075
Abstract
Description
[Technical Field]
[0001] Conventional technology For various technical (e.g., physical or chemical) processes, it may be desirable to predict the time course of one physical quantity based on multivariate time series data of other different physical quantities, and / or to predict anomalies based on multivariate time series data of multiple physical quantities. For example, it may be desirable to predict the state-of-health or hydrogen refueling amount of a fuel cell based on the time course of current intensity and voltage, or, in the case of a drilling machine, to predict which material will be drilled based on the time course of current intensity and voltage, or to predict anomalies based on the time course of current intensity and voltage. Typically, a machine learning model can be trained for this purpose for exactly one application case (e.g., to predict the state of health of a fuel cell). [Overview of the project] [Problems that the invention aims to solve]
[0002] Disclosure of the invention This disclosure relates to a method for predicting the time course of a physical target quantity using a machine learning model based on multivariate sensor data, wherein the multivariate sensor data may be irregularly sampled sensor data.
[0003] When sensor data from various different sensors is detected, these sensor data may have different sampling rates. Furthermore, many sensor data may have missing data points (for example, due to measurement errors or excessively high uncertainty). Additionally, the time segments in which the sensor data exists may have different durations. Conceptually, it may not always be possible to bijectively map each data point in the first sensor data to a data point in a second sensor data set that differs from the first. [Means for solving the problem]
[0004] The method described herein makes it possible to predict the time course of a physical target quantity even in such cases of irregular sensor data. This is achieved, for example, by dividing the sensor data into sensor data segments, and then, for each sensor data segment, identifying a sensor data segment representation that has the same predefined dimensions for all sensor data segments. Therefore, the dimensions of the sensor data segment representation do not depend on the regularity of the data points in the sensor data segment (e.g., sampling rate, presence or absence of data points, etc.).
[0005] Furthermore, the machine learning models described herein can be trained to predict physical target quantities for multiple different tasks, each having at least partially different physical quantities. This allows for efficient learning of physical laws that are common to multiple different tasks, for example. Such training is possible precisely because the methods described herein enable the processing of irregular multivariate sensor data.
[0006] Various embodiments are methods for predicting the time course of a physical target quantity using a machine learning model, the method provides multivariate sensor data, the multivariate sensor data is associated with a certain period, and for each of the multiple physical quantities, there is sensor data representing the time course of the physical quantity within that period, and each physical quantity is associated with a text description that describes the physical quantity (and optionally further, the measurement environment when each sensor data was detected) (as text), and for each of the multiple physical quantities, the sensor data is divided into multiple (for example, non-overlapping sensor data segments), and each of the multiple sensor data segments is divided into sensor data segments The method comprises: identifying each sensor data segment representation that represents a sensor data segment and has predefined dimensions (independent of the number of data points in the sensor data segment); identifying each input element using each sensor data segment representation, time-related location information representing the (e.g., temporal) location of the sensor data segment within a period, and the respective text descriptions of the physical quantities; and predicting the time evolution of a physical target quantity using a machine learning model in response to all input elements and at least one target quantity query being input to the machine learning model, wherein at least one target quantity query represents the (e.g., temporal) location of the time evolution to be predicted within a period and the text description of the physical target quantity.
[0007] The following shows various different examples.
[0008] Example 1 is a method for predicting the time course of a physical target quantity using a machine learning model as described above.
[0009] Example 2 is configured according to Example 1. In this example, each of the plurality of sensor data segments of at least one physical quantity has at least two sensor data segments having different numbers of data points from each other.
[0010] By mapping each sensor data segment to a respective sensor data segment representation having a predefined dimension, all sensor data segment representations will have this predefined dimension regardless of the dimension of the sensor data segment. As a result, sensor data segments can have different dimensions from each other (e.g., duration, number of data points (e.g., due to differences in sampling rate), scalar values, or even no values at all). Conceptually, this method can predict the temporal evolution of the target quantity even for heterogeneous multivariate sensor data.
[0011] Example 3 is configured according to Example 1 or Example 2. In this example, the time-related position information represents the start time and the end time within a period.
[0012] Since the method described in this specification allows for different numbers of data points for each sensor data segment, in addition to the start time, this duration (e.g., represented by the end time) can also be represented using the time-related position information.
[0013] Example 4 is configured according to any one of Examples 1 to 3. In this example, the machine learning model has a transformer model, and the encoder and / or decoder of the transformer model has an attention layer, to which all input elements (i.e., each input element of each physical quantity) are supplied.
[0014] (Not only the input elements in either the dimension of physical quantities or the time dimension) but all input elements are supplied to the attention unit, so that the machine learning model can consider more complex dependencies (e.g., based on previous training), thereby enhancing the accuracy of predictions. This enables, for example, the use of heterogeneous sensor data elements such as scalar values and / or missing values in combination with time series.
[0015] Example 5 is configured according to any one of Examples 1 to 4. In this example, each sensor data segment representation for the sensor data segment has a learned sensor data segment-specific parameter vector as a query, and is specified using a (multi-head) attention unit having the sensor data segment as a key and a value, and / or each input element is specified using each sensor data segment representation, each position representation, and each text description of physical quantities. The position representation has a learned position-specific parameter vector as a query, and is specified using a (multi-head) attention unit having time-related position information as a key and a value.
[0016] Example 6 is configured according to any one of Examples 1 to 5. In this example, the machine learning model has a transformer model, and one or more attention layers of the transformer model have (multi-head) attention units in the encoder and / or decoder, and a target quantity query is supplied to the attention unit.
[0017] Thereby, for example, the need for trained free parameters as input is eliminated, so that the machine learning model can identify predictions with less computational cost. During training, there is no longer a need to train such free parameters, thereby reducing the computational cost (and thus the duration required therefor) during training. The accuracy of predictions is significantly enhanced by the target quantity query having a text description of the physical target quantity.
[0018] Example 7 is a method for controlling a technical (e.g., physical or chemical) process, which includes using provided multivariate sensor data to predict the time course of a physical target quantity according to any one of Examples 1 to 6, and controlling the technical process in consideration of the prediction.
[0019] Example 8 is a control device configured to carry out the method according to Example 7.
[0020] Example 9 is a system comprising an apparatus configured to perform a technical process, one or more sensors for detecting multivariate sensor data, and a control device according to Example 8 for controlling the technical process.
[0021] Example 10 is a data processing unit configured to implement one of the methods described in Examples 1 through 6.
[0022] Example 11 is a computer program that, when executed by a processor, includes instructions to cause the processor to perform one of the methods described in Examples 1 through 7.
[0023] Example 12 is a computer-readable medium that, when executed by a processor, stores instructions causing the processor to implement one of the methods described in Examples 1 through 7.
[0024] Similar reference numerals in the drawings generally refer to the same part in all different views. The drawings are not necessarily to scale and generally focus on illustrating the principles of the present invention. In the following description, various embodiments will be described with reference to the following drawings. [Brief explanation of the drawing]
[0025] [Figure 1] This is a flowchart of a method for predicting the time course of a physical target quantity under various different conditions. [Figure 2] This figure shows an exemplary system on which this method can be implemented. [Figure 3] This figure shows the time course of an example physical quantity detected and the time course of a physical target quantity that should be predicted. [Figure 4] This diagram shows the identification of input elements in various different ways. [Figure 5] This figure shows the prediction of the time course of a physical target quantity using various different machine learning models. [Figure 6] This figure shows an attention layer having a single-stage attention mechanism in various different forms. [Figure 7] This diagram illustrates query-based forwarding using a two-stage attention mechanism as an example. [Modes for carrying out the invention]
[0026] The following detailed description relates to the accompanying drawings, which are illustrated to illustrate special details and embodiments of the present disclosure that enable the implementation of the present invention. Other different embodiments may be used without departing from the scope of protection of the present invention, and structural, logical, and electrical modifications may be implemented. The various different embodiments of the present disclosure are not necessarily mutually exclusive, for some embodiments of the present disclosure may be combined with one or more other embodiments of the present disclosure to form new embodiments.
[0027] The following sections will provide more detailed explanations of various different examples.
[0028] Figure 1 shows flowcharts of methods 100 for predicting the time course of a physical target quantity under various different configurations.
[0029] Method 100 (in 102) provides multivariate sensor data, which may include the provision of multivariate sensor data associated with a certain period, and for each of the multiple physical quantities, having sensor data representing the time progression of the physical quantity within that period. Each physical quantity may be associated with a text description (for example, as text) that describes the physical quantity and the measurement environment when the respective sensor data was detected.
[0030] Method 100 may include (in 104) dividing each sensor data for each of the multiple physical quantities into each of the multiple (e.g., non-overlapping) sensor data segments. Furthermore, Method 100 may then include, for each of the multiple sensor data segments, identifying each sensor data segment representation that represents the sensor data segment and has predefined dimensions, and identifying each input element using each sensor data segment representation, time-related positional information representing the (e.g., temporal) position of the sensor data segment within a period, and each text description of the physical quantity.
[0031] Method 100 involves using a machine learning model to predict the time course of a physical target quantity in response to all input elements and at least one target quantity query being input to the machine learning model (in 106), wherein the at least one target quantity query may represent the (e.g., temporal) position of the time course to be predicted within a period and a text description of the physical target quantity.
[0032] This method may be carried out by one or more computers equipped with one or more data processing units. The term “data processing unit” may be understood as any type of entity that enables the processing of data or signals. The data or signals may be processed, for example, according to at least one (i.e., one or more) specific functions performed by the data processing unit. The data processing unit may include, or consist of, integrated circuits such as analog circuits, digital circuits, logic circuits, microprocessors, microcontrollers, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable gate arrays (FPGAs), or any combination thereof. Any other methods for implementing each of the functions described in more detail herein may also be understood as data processing units or logic circuit devices. One or more of the individual method steps described herein may be carried out (e.g., implemented) by the data processing unit via one or more specific functions performed by the data processing unit.
[0033] In other words, this method is implemented in various different embodiments, particularly in computer form.
[0034] Figure 2 shows the system 200 in various different embodiments. The system 200 may have an apparatus 202 configured to carry out a technical process. In various embodiments, the apparatus 202 may be a robotic apparatus (abbreviated as: robot) such as an industrial robot in the form of a robotic arm for moving, attaching, or processing a workpiece, or for bin picking parts, a manufacturing robot, a maintenance robot, a household robot, a medical robot, a vehicle (e.g., a vehicle that is at least partially automated), a household appliance, a manual machine (e.g., a drilling machine), a manufacturing machine, a personal assistant, an access control system, etc., and any other type of robotic apparatus. In various embodiments, the technical process may be a physical or chemical process such as a manufacturing process (e.g., manufacturing a product or intermediate product), a processing process (e.g., processing a workpiece), a control process (e.g., moving a robotic arm), a setting process (e.g., calibrating a measuring device), etc.
[0035] System 200 may have a control device 204 configured to control a technical process (for example, according to one or more control parameters 206). The term “control device” (also referred to as “control equipment”) may include, for example, circuits and / or processors capable of executing software, firmware, or a combination thereof stored in a storage medium, and in this example may be understood as any kind of logical implementation unit capable of issuing instructions to an actuator. The control device may be configured, for example, to control the operation of System 200 by program code (e.g., software).
[0036] In various configurations, multivariate time series of sensor data (i.e., multivariate sensor data) can be detected within a certain period. Conceptually, multivariate sensor data 210 (d=1 to P) can represent the time progression of each physical quantity d among P physical quantities (where P may be an integer of 1 or more) within that period. In this case, the sensor 208(d) for detecting the sensor data may be, for example, a temperature sensor, a concentration sensor for detecting one or more components, a pressure sensor, etc. The sensor data of physical quantities may be not only the output quantities of a technical process, but also input quantities applied according to one or more control parameters 206 to control the technical process, such as applied voltage and / or current intensity (e.g., resulting from the applied voltage). The sensor data of physical quantities may be detected in-situ or ex-situ. For example, the characteristics of a manufactured product can be detected (as sensor data) after the implementation of a technical process (e.g., ex-situ). Therefore, it is understood that multivariate sensor data may contain time series of physical quantities that have some relevance to the technical process.
[0037] In various embodiments, the control device 204 may be configured to implement a machine learning model 212. The machine learning model 212 may be configured to use multivariate sensor data 210 (d=1 to D) to predict the time progression 214 (e.g., undetected) of at least one physical target quantity. The control device 204 may be configured to adapt one or more control parameters 206 (i.e., to control the technical process) taking into account the predicted time progression 214 of the physical target quantity. In various embodiments, the control device 204 may be configured to identify anomalies based on the predicted time progression 214 of the physical target quantity and to control the technical process accordingly (e.g., to stop it and output a signal to notify the user of the device 202 about the anomaly).
[0038] In the following sections, various aspects of Method 100 will be described in more detail, using the technical system 200 as an example.
[0039] Figure 3 shows the time progression of the detection of an exemplary physical quantity d within a certain period 210(d), and the physical target quantity d detected within one segment of that period. * The time progression and the physical target quantity d * This shows the predicted time progression 214.
[0040] In 104, the sensor data of each physical quantity d is divided into one or more (e.g., many) (e.g., non-overlapping) sensor data segments.
number
number
number
number
number
Number
Number
[0041] For the sake of explanation, in various different aspects, among a plurality of physical quantities, the physical quantity is referred to as a channel or a channel dimension c. For each physical quantity d, it may be well associated with a respective text description TB. The text description can describe the physical quantity d and the measurement environment when the corresponding sensor data is detected (for example, as text). The text description of the physical quantity d described in this specification may have, for example, the physical quantity itself, the description of the signal of the physical quantity, one or more information regarding the sensor that detected the sensor data, etc.
[0042] In many aspects, at least one time division
Number
[0043] According to various different aspects, each sensor data segment
number
number
[0044] According to various different aspects, sensor data segment
number
number
number
[0045] For example, the sensor data segment representation for a sensor data segment is a learned parameter vector specific to the sensor data segment.
number
number
number
[0046] Conceptually, a parameter vector specific to the sensor data segment.
number
number
number
[0047] While MSA is often used as a name for a "Multi-Head Self Attention" system where the query Q, key K, and value V are identical (i.e., Q=K=V), in this specification, MSA is used for a multi-head standard attention unit (or multi-head standard attention system), and it is understood that Q, K, and V may be different from each other.
[0048] Such a parameter vector e CLSThe learning of (for training language models) is described, for example, in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", arXiv:1810.04805, 2019 by J. Devlin et al. (hereinafter referred to as reference [1]), in which the parameter vector e CLS However, it is referred to as a special classification token called CLS.
[0049] Parameter vectors specific to sensor data segments
number
[0050] Input element Z i,d,0 This is the sensor data segment representation V i,d and the corresponding positional representation
number
number
number
[0051] Text expression
number
number
number
number
number
number
[0052] position representation
number
number
number
number
number
number
number
number
number
[0053] Figure 5 shows the physical target quantity d using the machine learning model 212 in various different configurations. * Time progression 214,x τ*,d* This indicates a prediction.
[0054] In various embodiments, the machine learning model 212 may have a transformer model, or may be a transformer model. The transformer model may have an encoder 212-1 and a decoder 212-2. An exemplary transformer model is described in "Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting", International Conference on Learning Representations ICLR, 2022, 2019 by Y. Zhang et al. (hereinafter referred to as Reference [2]). However, the transformer model described in Reference [2] requires the use of regular multivariate sensor data (i.e., identical sampling rates, no missing values, etc.) to achieve satisfactory accuracy. This is because, in Reference [2], all segment lengths must be the same, and time information is not considered. Furthermore, the transformer model described in Reference [2] can only predict time elapseds of predefined time lengths because the position encoding is learned. Furthermore, it is impossible to adapt it to other physical quantities (since no text description is used). For brevity, the following will explain the differences from the transformer model described in reference [2], and for other aspects, please refer to reference [2].
[0055] The encoder 212-1 and / or decoder 212-2 described herein may have multiple attention layers l. Input element
number
number
number
number
number
number
number
number
number
[0056] In various different embodiments, each attention layer l* in encoder 212-1 and / or decoder 212-2 may have exactly one attention unit (i.e., one-stage attention mechanism) (followed by layer normalization, dropout, skip connection, feedforward, etc.), and all input elements are included in this attention unit.
number
number
number
[0057] Figure 6 shows attention layers l having a single attention unit in various different configurations. In this case, in 604, all (existing) input elements
number
number
number
number
number
[0058] All input elements
number
number
number
[0059] Referring to Figure 5, then, according to the transformer architecture, sequence embedding generated by encoder 212-1 after L attention layers in the encoder.
number
number
number
number
number
number
[0060] In various different embodiments, each attention layer l in encoder 212-1 and / or decoder 212-2 can implement a routing mechanism, similar to that described in reference [2]. In the routing mechanism, each attention unit MSA osa However, the first subunit
number
number
number
number
number
number
number
number
[0061] For illustrative purposes, the transfer mechanism is shown in Figure 7 as an example of a two-stage attention unit. The two-stage attention unit is used solely for illustrative purposes, based on the small number of input elements (in this example, time dimension t), and it should be understood that both encoder 212-1 and decoder 212-2 use a single-stage attention mechanism as described herein.
[0062] By combining a query-based transfer mechanism with a single-stage attention mechanism, the complexity of the machine learning model 212 can be reduced. This allows the machine learning model 212 to be trained (or pre-trained) with less computational cost (for example, because it is not necessary to learn the transfer variables). By incorporating information about the target quantity (using text descriptions and time criteria) into the target quantity query, even better embeddings are generated, which leads to improved accuracy of the machine learning model 212. Furthermore, the time series of sensor data can include relatively long time spans, and therefore, the reduced complexity of the attention mechanism leads to improved computational efficiency.
[0063] While various approaches have been used to predict the time evolution of physical target quantities, it is understood that anomalies can also be predicted using the machine learning models described herein. In one example, anomalies can be identified based on the predicted time evolution of a physical target quantity. For instance, anomalies can be identified by determining that the predicted time evolution of a query's physical quantity is similar to the time evolution of an input physical quantity, and by evaluating the reconstruction error of the input. If the reconstruction error is, for example, above a threshold, the input can be identified as anomaly.
Claims
1. A method (100) for predicting the time course (214) of a physical target quantity using a machine learning model (212), The above method (100) is, (102) To provide multivariate sensor data, wherein the multivariate sensor data is associated with a certain period, and for each of the multiple physical quantities, there is a sensor data (210) representing the time progression of the physical quantity within that period, and each physical quantity is associated with a text description describing the physical quantity (102), For each of the above-mentioned physical quantities (104), ● Dividing each of the aforementioned sensor data into multiple sensor data segments, ●For each of the sensor data segments among the plurality of sensor data segments, ○ Identifying each sensor data segment representation that represents the sensor data segment and has a predefined dimension (independent of the number of data points in the sensor data segment), ○Identifying each input element using the respective sensor data segment representation, time-related positional information representing the position of the sensor data segment within the period, and the respective text descriptions of the physical quantities, In response to all input elements and at least one target quantity query being input to the machine learning model (212), the machine learning model (212) is used to predict the time progression (214) of the physical target quantity (106), wherein the at least one target quantity query represents the position of the time progression (214) to be predicted within the period and a text description of the physical target quantity (106), Method (100), including the method (100).
2. Each of the plurality of sensor data segments of at least one physical quantity has at least two sensor data segments having a different number of data points from each other. The method according to claim 1 (100).
3. The aforementioned time-related location information represents the start and end points within the period. The method according to claim 1 or 2 (100).
4. The aforementioned machine learning model (212) has a transformer model, The encoder (212-1) and / or decoder (212-2) of the transformer model have a attention layer to which all input elements are supplied. The method according to any one of claims 1 to 3 (100).
5. Each of the aforementioned sensor data segment representations for a sensor data segment is identified using an attention unit that has a learned sensor data segment-specific parameter vector as a query and the sensor data segment as a key and value. and / or, Each of the aforementioned input elements is identified using the respective sensor data segment representation, the respective position representation, and the respective text description of the physical quantity, wherein the position representation is identified using an attention unit having a learned position-specific parameter vector as a query and the time-related position information as key and value. The method according to any one of claims 1 to 4 (100).
6. The aforementioned machine learning model (212) has a transformer model, One or more attention layers of the transformer model have attention units within the encoder (212-1) and / or the decoder (212-2), and the target quantity query is supplied to the attention unit. The method according to any one of claims 1 to 5 (100).
7. System (200), ● Apparatus (202) configured to carry out a technical process, ●One or more sensors (208) for detecting multivariate sensor data (210), ● A control device (204) configured to predict the time progression (214) of a physical target quantity in accordance with any one of claims 1 to 6, and to control the technical process taking the prediction into consideration, A system (200) equipped with the following features.
8. A data processing unit configured to carry out the method described in any one of claims 1 to 6.
9. A computer program, which, when executed by a processor, includes instructions for causing the processor to carry out the method (100) according to any one of claims 1 to 6.
10. A computer-readable medium that, when executed by a processor, stores instructions causing the processor to carry out the method (100) according to any one of claims 1 to 6.