How to generate training data for machine learning models
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2026-01-16
- Publication Date
- 2026-07-30
Smart Images
Figure 2026123813000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a method for generating training data for a machine learning model that generates text descriptions with respect to supplied time series.
Background Art
[0002] In various application cases, it is desired to associate text sensor data of a specific modality (time series, image, etc.), that is, a text description of sensor data, in pairs. For example, the CLIP (Contrastive Language Image Pretraining) model associates text and images with each other. Similarly, a machine learning model can be trained to generate text descriptions with respect to a time series (e.g., a temperature time series), so that a sensor that supplies the time series can "communicate" with a user via text. However, one of the biggest challenges in such an approach is to sufficiently and well select training pairs of text and sensor data of each other modality for training the corresponding machine learning model. This becomes an even greater challenge based on the limited availability of data for a specific area in relation to industrial applications. Therefore, an effective approach for generating training data for a machine learning model that generates text descriptions with respect to supplied time series is desired.
Summary of the Invention
Means for Solving the Problems
[0003] According to various embodiments, a method for generating training data for a machine learning model that generates text descriptions with respect to supplied time series is provided. The method includes for each exemplary time series consisting of a set of exemplary time series, · extracting values of time series features from the exemplary time series, and • Filling in the extracted time-series feature values into a text prototype that has gaps for at least some of its time-series features, • By supplying the filled text prototype as a prompt to a large-scale language model, the reformulation (Umformulierung) of the filled text prototype is generated, • To generate training examples for a machine learning model using a time series as training input and a reformulated filled text prototype as the target output for that training input, Includes.
[0004] The method described above allows for the efficient generation of training data for a machine learning model that generates text descriptions for sensor data time series (i.e., a "CLIP-like" model). This is possible for specific types of sensor data and, consequently, for specific application cases. This significantly simplifies the process of text description of time series data. Furthermore, thanks to the performance of the large-scale language model using the training data generated according to the method described above, the machine learning model can be trained to be supplied with high-quality and diverse text. The diversity of the supplied text can also be controlled by giving the large-scale language model some degree of freedom in generating the target output (i.e., reformulation of the filled text archetype). For this purpose, the large-scale language model can be appropriately prompted. This method also allows for the incorporation of domain knowledge (e.g., through the selection of features and text archetypes), which contributes to improving the quality of the text description.
[0005] A virtual sensor can be generated by forming a text description of sensor data supplied from a sensor in a time-series form (i.e., sensor signals in the form of sample sequences). This allows information (particularly in text format) about the elements encoded by the sensor signal to be obtained based on the sensor signal (i.e., indirect measurements can be performed based on the sensor signal used as a direct measurement).
[0006] As described above, trained machine learning models can also be used to identify anomalies in technical systems. This is done as follows: a preceding time series is paired with a text description (e.g., a description of an observed quantity), such as "pass" or "fail" in a technical test. Based on this information, similar (and therefore, in particular, potentially anomaly) time series can be identified by searching for time series that generated text such as "fail" (or a similar description).
[0007] Training data can be used not only to adjust the parameters of a machine learning model based on loss, but also for other phases of the training process for the machine learning model, such as testing, validation, and verification.
[0008] Text descriptions enable or simplify the exploration, classification, and working with time-series data.
[0009] Various examples are shown below.
[0010] Example 1 is a method for generating training data for the machine learning model described above.
[0011] Example 2 is the method of Example 1, wherein the prompt includes instructions to a large-scale language model on how much the filled text prototype should be reformulated.
[0012] This allows for control over the diversity of text descriptions included as target outputs in the generated training examples, and ultimately, the diversity of text descriptions output by the machine learning model. The prompts that LLM uses when attempting to output reformulations (and thus time-series text descriptions), such as prompts regarding text style information, can also be varied in various ways.
[0013] Example 3 is the method of Example 1 or 2, wherein the method described above includes selecting time series features from a set of time series features (e.g., a set of time series features supplied by an extraction tool) based on expert knowledge regarding the application in which the machine learning model should be trained.
[0014] By doing so, expert knowledge (i.e., domain knowledge) related to each application can be easily incorporated into the training of the machine learning model, and it is possible to ensure that the text descriptions supplied by the machine learning model are relevant to each application.
[0015] Example 4 is an example in which the above method is used. For each of the example time series, which consists of a set of example time series, • By feeding the filled text prototype multiple times as a prompt to a large-scale language model, multiple reformulations of the filled text prototype are generated. Using a time series as training input and a reformulated filled text prototype as the target output for that training input, the system generates training examples for a machine learning model from each reformulation. The method is one of any one of Examples 1 to 3, including the above.
[0016] This allows multiple training examples to be obtained from each example time series, and during training, the machine learning model is provided with multiple "correct" text descriptions (i.e., target outputs for each time series).
[0017] Example 5 is the method described in Example 4, where the prompts are different from each other for various reformulations of the text prototype filled for the exemplary time series.
[0018] For example, it is possible to request the LLM to supply text descriptions regarding various styles. This enhances the diversity of the text descriptions supplied by the (trained) machine learning model.
[0019] Example 6 is a method of training a machine learning model that generates text descriptions regarding the supplied time series, generating training data for the machine learning model according to any one of Examples 1 to 5, training the machine learning model using the generated training data, and including the above steps.
[0020] The training is performed, for example, using a contrast loss. In this case, the training data can also be used as test data and / or verification data.
[0021] Example 7 is a data processing system configured to implement the method described in any one of Examples 1 to 6.
[0022] Example 8 is a computer program including instructions for causing a processor to implement the method described in any one of Examples 1 to 6 when executed by the processor.
[0023] Example 9 is a computer-readable medium storing instructions for causing a processor to implement the method described in any one of Examples 1 to 6 when executed by the processor.
[0024] In the drawings, like reference numerals designate, throughout the various figures, like parts in their entirety. The drawings are not necessarily drawn to scale and have been primarily focused on illustrating the basic principles of the present invention as a whole. In the following description, various aspects will be described with reference to the following respective drawings.
Brief Description of the Drawings
[0025] [Figure 1] It is a diagram showing an example of a production line. [Figure 2] It is a flowchart showing the generation of a training example according to an embodiment. [Figure 3] It is a flowchart showing a method for generating training data for a machine learning model that generates a text description with respect to a supplied time series according to an embodiment.
Modes for Carrying Out the Invention
[0026] The following detailed description is related to the accompanying drawings presented to explain the special details and aspects of the present disclosure in which the present invention can be implemented. Without departing from the scope of protection of the present invention, other aspects can be used and structural, logical, and electrical changes can be implemented. The various aspects of the present disclosure are not necessarily mutually exclusive. This is because some aspects of the present disclosure can form new aspects in combination with one or more other aspects of the present disclosure.
[0027] In the following, various examples will be described in more detail.
[0028] Time series are recorded in various technical application fields, for example, in a production line, in order to obtain information regarding a technical system.
[0029] FIG. 1 shows an example of a production line 100.
[0030] The manufacturing line 100 has a plurality of machines 101 through which the product or the starting material for the product passes. These machines have sensors 102 that supply information about the manufacturing process and / or the (intermediate) product, for example, in the form of temperature, pressure, or camera images of the (intermediate) product, for each machine 101.
[0031] Here, each sensor typically detects data at various measurement points, thereby supplying a time series.
[0032] For various applications, including in relation to manufacturing lines and other technological systems (e.g., vehicles), it may be desirable to describe the above (sensor data) time series in text. For example, • Machine monitoring: Time series may represent machine vibration data over time. Text descriptions may indicate patterns of abnormalities suggesting potential mechanical problems or wear by monitoring staff; • Manufacturing line: The time series may indicate the number of units manufactured per hour at the manufacturing plant. The text description may indicate improvements in efficiency, defects, or failures; • Vehicle elements: Time series may record the vehicle's speed, engine RPM, and fuel consumption while it is in motion. Text descriptions may explain the driving pattern, efficiency, and potential maintenance needs to the driver.
[0033] In the example in Figure 1, for example, the data processing device 103 receives sensor data detected by the sensor 102, creates a text description of the (sensor data) time series, and then displays the text description on a display, for example, or transmits it to one or more other data processing devices via a network. According to various embodiments, the data processing device 103 uses a machine learning model 104 that has been appropriately trained to output a text description for an input time series in order to generate a text description of the (sensor data) time series (or in other words, to associate the text description with data of a modality other than text).
[0034] One example of such a machine learning model is the CLIP (Contrastive Language-Image Pretraining) model, which provides text descriptions for images. In the following, machine learning model 104 will receive a time series (i.e., a measurement sequence of a physical quantity, i.e., a sample sequence, e.g., a temperature sample or a voltage sample) as input and will supply a text description of the time series as output, i.e., text that describes the characteristics of the time series as words (e.g., in German or English).
[0035] To train the machine learning model 104 on these tasks, training examples are needed, where each training example includes a time series as an exemplary input (i.e., training input) for the machine learning model 104 and a target output, i.e., text describing the exemplary input. The target output can also be considered a "label". The machine learning model 104 can be trained using such training examples (i.e., such training data). For example, this is a neural network trained using backpropagation of contrast loss (i.e., tuned so that the contrast loss is reduced for a set (batch) of training examples).
[0036] Various embodiments provide approaches for generating training examples for training the above-described machine learning models. This is done by extracting important features from a time series (which may take expert knowledge into account) based on the time series without corresponding text descriptions, and then extracting one or more text descriptions for the time series (as target outputs of the time series used as training input for the machine learning model) based on the extracted features. Thus, training examples that can later be used for training the machine learning model are generated in the form of text time series pairs.
[0037] These methods for generating training data will be explained in more detail below, with reference to Figure 2.
[0038] Figure 2 shows a flowchart 200 for generating a training example according to one embodiment.
[0039] Feature extraction 202 is performed once for each of the multiple sensor data time series 201, and this feature extraction 202 supplies feature values 203 (i.e., values representing specific quantities of features) for each sensor data time series 201. This is done, for example, using a feature extraction tool, such as tsfresh. Typically, not all of the feature values supplied in this case are relevant to their respective uses (and therefore to the text description to be generated) (i.e., not all features in the set of features are relevant).
[0040] Therefore, in 204, feature values are limited to feature values of the relevant feature subset. Expert knowledge 205 (domain knowledge) relating to one or both domains, sensor data, and applications is considered in order to define the relevant subset (i.e., select relevant features). If such expert knowledge is not available, relevant features can be automatically selected for specific tasks, such as classification.
[0041] In other words, the relevant feature values obtained from the limitation are the values of features identified as important for each task (to be processed in each application).
[0042] After the relevant feature values 206 are selected, in 207, the relevant feature values are populated into a predefined archetype 208 that has gaps for the relevant features. Thus, this provides each time series with a pair consisting of a time series and a text description, the latter of which is simply a populated text archetype (i.e., a populated "template"). Thus, the text descriptions here essentially describe the characteristics of each time series (with respect to the relevant features), but when the format is extremely limited, they may be incomplete sentences and have low diversity depending on the situation.
[0043] Therefore, in order to obtain a better target time series output, a filled template is supplied to a large language model (e.g., ChatGPT) at 209 (as a prompt) to create a final text description 210 of the time series. The filled template can then supply a text-based reformulation of the filled prototype, which is then used as the target output of a training example that includes a time series as training input.
[0044] By using a Large-Scale Language Model (LLM) to reformulate a filled archetype, high-quality sentences are provided as target outputs with great diversity (which can be further influenced by the LLM's corresponding prompts (i.e., instructions for the LLM in the prompts)). It is also possible to generate multiple target outputs for different training examples for a single time series (containing all identical time series as training inputs); that is, using an LLM, multiple different descriptions (e.g., sentences) can be generated from the same filled archetype, for example, in different styles (by giving the LLM appropriate instructions as prompts).
[0045] Examples of sets of features from which feature values are extracted: Sum of time series values Median of time series 0.1 minutes About 0.3 minutes Approximately 0.9 minutes Average value of a time series Number of peaks Peak growth rate Peak location
[0046] Examples of related features: Number of peaks Peak growth rate Peak location
[0047] Examples of prototypes (templates): For each peak, "Peak [Peak_Ordnung] is the segment of measurement at [Peak_Segment_Nummer]. The rate of increase of the peak is [Peak_Anstieg]."
[0048] An example of a reformulated archetype (i.e., a target output for training examples): "The time series exhibits two peaks. The first peak is located in the third segment of the measurement. The first peak has a steep rate of increase. The second peak is found in the first segment of the measurement. The time series reaches a maximum value of 48.86 and has a minimum value of 0.55."
[0049] Other features or related features that the template has gaps in, for example, include the maximum value of the time series, symmetric evaluation, autocorrelation coefficient, other quantiles, FFT coefficients, wavelet coefficients, etc. As mentioned above, features may include, for example, those extracted by tsfresh. From these features, related features can be selected using expert knowledge (i.e., domain knowledge).
[0050] The generation of training data and the training of the machine learning model 104 can be performed by the data processing device 103 itself, or at least partially by another data processing device. For example, the first data processing device generates training data, the second data processing device trains the model, and then transmits the model to the data processing device 103 for application.
[0051] In summary, the above method is provided by various embodiments, as shown in Figure 3.
[0052] Figure 3 shows a flowchart 300 illustrating a method for generating training data for a machine learning model that generates text descriptions with respect to a supplied time series, according to one embodiment.
[0053] For each example time series consisting of a set of example time series, In 301, the values of time series features are extracted from the example time series. In 302, the extracted time-series feature values are populated into the text prototype which has gaps for at least some of the time-series features. In 303, by supplying the filled text archetype as a prompt to the large-scale language model, a reformatting of the filled text archetype is generated. In step 304, a training example for a machine learning model is generated using a time series as the training input and a reformulated filled text prototype as the target output for that training input.
[0054] An example time series is, for instance, a time series of sensor data, i.e., a sequence of sensor data values.
[0055] The method shown in Figure 3 can be implemented by one or more computers (i.e., a data processing system) having one or more data processing units. The term “data processing unit” can be understood as any type of entity that enables the processing of data or signals. Data or signals can be processed according to at least one (i.e., one or more) special function performed by the data processing unit. A data processing unit may include, or can be formed from, analog circuits, digital circuits, logic circuits, microprocessors, microcontrollers, central units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), or any combination thereof. Any other method for implementing each of the functions described in more detail herein can also be understood as a data processing unit or a logic circuit device. One or more of the individual method steps described herein can be implemented by a data processing unit and can be implemented (e.g., embodied) by one or more special functions performed by the data processing unit.
[0056] Therefore, the above method is implemented in various embodiments, particularly in computers.
[0057] After training, the machine learning model can be applied to sensor data calculated by at least one sensor.
[0058] For example, a machine learning model is used to control a robotic device after training, by being fed sensor data relating to the robotic device and / or its environment. The term "robotic device" can be understood as relating to any technical system (including the mechanical part whose movement is controlled), such as a computer-controlled machine, vehicle, household appliance, power tool, manufacturing machine, personal assistant, or access control system. For example, a machine learning model is supplied with a textual description of the internal state of the technical system, which the user can then use as a basis for control, for example.
[0059] Various embodiments can receive and use time-series sensor data from various sensors, such as video, radar, LiDAR, ultrasound, motion, and thermal mapping. The sensor data can be measured over a predetermined period or simulated (specifically to generate training examples).
Claims
1. A method for generating training data for a machine learning model (104) that generates text descriptions with respect to a supplied time series, For each example time series (201) consisting of a set of example time series, Extracting time series feature values (206) from the aforementioned example time series (201) (301), (302) Filling in the extracted time-series feature values (206) into a text prototype (208) having gaps for at least a portion of the time-series features, By supplying the filled text prototype as a prompt to a large-scale language model, a reformatting (210) of the filled text prototype is generated (303), Using the time series as training input and the reformulation of the filled text prototype (210) as the target output for the training input, a training example (304) for the machine learning model (104) is generated. A method that includes this.
2. The method according to claim 1, wherein the prompt includes an instruction to the large language model regarding the extent to which the filled text prototype should be reformulated.
3. The aforementioned method, Based on expert knowledge (205) regarding application cases in which the machine learning model (104) should be trained, select the time series features from a set of time series features (for example, a set of time series features supplied by an extraction tool). The method according to claim 1 or 2, including the method described in claim 1 or 2.
4. The aforementioned method, For each of the example time series (201) consisting of the set of example time series, By supplying the aforementioned filled text prototype to the large-scale language model multiple times as a prompt, multiple reformulations (210) of the filled text prototype are generated. Using the time series as training input and the reformulated (210) of the filled text prototype as the target output for the training input, a training example for the machine learning model (104) is generated from each reformulated (210), The method according to any one of claims 1 to 3, including
5. The method according to claim 4, wherein the prompts are different from each other for various reformulations among the reformulations (210) of the text prototype filled with the exemplary time series (201).
6. A method for training a machine learning model (104) that generates text descriptions with respect to a supplied time series, To generate training data for a machine learning model (104) according to any one of claims 1 to 5, The machine learning model (104) is trained using the generated training data, A method that includes this.
7. A data processing system (103) configured to carry out the method described in any one of claims 1 to 6.
8. A computer program, when executed by a processor, includes instructions for causing the processor to perform the method according to any one of claims 1 to 6.
9. A computer-readable medium storing instructions for causing a processor to perform the method according to any one of claims 1 to 6, when executed by the processor.