Method for data annotation and method for creating model data
By integrating measurement data from multiple environmental sensors for automated annotation and projection, combined with neural network training, the problem of high error variability in manual annotation in autonomous driving systems has been solved, achieving more accurate and faster data annotation and multimodal model training, thus improving system performance.
Patent Information
- Application Number
- CN202510994053.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-07-18
- Filing Date
- 2025-07-18
- Publication Date
- 2026-01-20
AI Technical Summary
Existing autonomous driving system training networks require a large amount of manually labeled baseline data, which leads to high error-proneness and low efficiency in labeling, making it difficult to effectively utilize multimodal sensor data.
By integrating measurement data from multiple environmental sensors, automated data annotation and projection are performed. Combined with neural network training, the accuracy and efficiency of annotation are improved, and a multimodal model is constructed.
It achieves more accurate and faster data annotation, reduces annotation error variability, and can effectively utilize multimodal sensor data to train multimodal models, thereby improving the performance of autonomous driving systems.
Smart Images
Figure CN121361472A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present invention relates to a method for data annotation. Furthermore, the present invention also relates to a method for creating model data. BACKGROUND
[0002] The development and manufacture of autonomous driving systems for vehicles increasingly apply machine learning methods, including training of deep neural networks. The range of applications of the driving systems extends from specific tasks, such as the recognition of vehicles in images, to comprehensive concepts that consider the entire processing chain from sensor measurements to the output of steering instructions for the vehicle in a unique common network.
[0003] Autonomous driving systems that pursue higher levels of automation are equipped with complex sensor systems that include a plurality of sensor modalities and a plurality of environmental sensors in each sensor modality, usually with a 360° panoramic view around the vehicle. This can result in up to 20 individual sensors in total per vehicle.
[0004] Today's networks are mainly trained in a supervised mode, which means that these networks require ground truth as input. This ground truth is created by humans by manually annotating the sensor measurement data, for example by drawing two-dimensional or three-dimensional bounding boxes around visually visible environmental objects in the sensor measurement data. SUMMARY
[0005] According to the invention, a method for data annotation is proposed. Thereby a more accurate and faster annotation can be performed. The susceptibility to errors of the annotation can be reduced. The annotated sensor measurement data enables the training of a multi-modal model.
[0006] The method for data annotation according to the invention has the following steps:
[0007] providing first sensor measurement data of a first environmental sensor and second sensor measurement data of a second environmental sensor, the sensor measurement data being for a common environmental scene of a vehicle environment of the vehicle having at least one environmental object,
[0008] creating a first annotation basis, the first annotation basis comprising at least the first and the second sensor measurement data merged in a common first time period,
[0009] annotating the first sensor measurement data of the first time period for marking the environmental object,
[0010] projecting the annotation of the first sensor measurement data into the second sensor measurement data of the first time period,
[0011] checking the labeling of the first sensor measurement data at least taking into account the projection in the second sensor measurement data, and
[0012] outputting the checked labeled first sensor measurement data.
[0013] The vehicle can be a motor-driven vehicle, in particular a motor vehicle or a heavy goods vehicle. The vehicle can be a two-wheeler, in particular a motorcycle or a bicycle.
[0014] The environmental objects can be other vehicles, buildings, traffic infrastructure devices such as traffic lights or traffic signs, roadways, living beings, plants or mobile devices.
[0015] The common environmental scenario is the vehicle environment of the vehicle at a certain position of the vehicle and at a certain point in time.
[0016] The first and second environmental sensors can be assigned to the same or to different sensor modalities. A sensor modality designates a certain sensor type. For example, a camera is assigned to one sensor modality and a radar sensor or a lidar sensor is assigned to another sensor modality. A lidar sensor is also assigned to a sensor modality which is different from the radar sensor. The first environmental sensor can be a lidar sensor, a radar sensor, a camera or an ultrasonic sensor. The second environmental sensor can be a lidar sensor, a radar sensor, a camera or an ultrasonic sensor.
[0017] The sensor measurement data can be present in the form of a point cloud. The point cloud can be two-dimensional or three-dimensional. The sensor measurement data can comprise image data, distance data and / or velocity data about the environmental objects in the vehicle environment.
[0018] The labeling can be a recognition, a marking, an assignment, an arrangement, a contour detection and / or a classification of the environmental objects and / or of object properties of the environmental objects. An example of a classification of an environmental object is an assignment of the environmental object to an object class, for example a vehicle, a pedestrian, a bicycle, a road sign, a traffic light, a lane marking, a road intersection, a plant, a building and others. An object property can be an object position and / or an object shape. The labeling can be a two-dimensional or three-dimensional bounding box. An object property can be further information about an environmental object, for example a vehicle type, a color, a driving direction, a lane, a road type, a parking area, a velocity, an acceleration, a trajectory or similar information. An object property can comprise an event assignment of a certain environmental object or of multiple environmental objects to one or more events, for example a pedestrian crossing, a passing process or a turning process.
[0019] The labeling can be a manual, partially automated or automated labeling. The labeling can also be referred to as a labeling.
[0020] The annotation basis is a data-based, in particular visual, preparation of the merged at least first and second sensor measurement data. The annotation basis can be visually built on the merged at least first and second sensor measurement data. The annotation basis can be built by data fusion of the first and second sensor measurement data.
[0021] The merging of the first and second sensor measurement data can be an insertion of the second sensor measurement data into the first sensor measurement data or vice versa. The insertion can occur before the integration of the first and second sensor measurement data. The integration can be a mutual adjustment of the first and second sensor measurement data, for example a matching to a reference time point. The reference time point can be a measurement time point of the first or second sensor measurement data.
[0022] The merging can be a comparative presentation of the first and second sensor measurement data. For example, the first and second sensor measurement data can be presented above and below or side by side.
[0023] The common time period can be a measurement time point of the first and second sensor measurement data or a time interval having one or more measurement time points of the first and second sensor measurement data. The common time period can be one frame in a time sequence consisting of multiple frames. For example, the time sequence can last 10 s. The common time period can have a duration of less than 1 s, preferably less than 200 ms, preferably 100 ms. The common time period relates to the same time point or the same time interval of the environmental scene detected by the first and second sensor measurement data.
[0024] The projection of the annotation is in particular automated, rather than manually performed.
[0025] The checking can involve a comparison of the annotated first sensor measurement data with the projected annotation in the second sensor measurement data. The checking can involve a change of the annotation of the first sensor measurement data depending on the comparison. The checking can involve a confirmation of the annotation of the first sensor measurement data depending on the comparison. The comparison can be a visual comparison. The comparison can be performed manually.
[0026] The creation of the annotation basis and the annotation of the first sensor measurement data can be performed sequentially in time, simultaneously or in time reversed to each other. By the annotation basis, higher value annotations can be created which contain for example a larger angular range and more semantic information compared to a single sensor measurement data, like the first sensor measurement data.
[0027] In creating the labeling basis, the individual sensor measurement data present at different measurement time points in the first time period can be integrated, preferably by compensating for the ego-motion of the vehicle in the sensor measurement data and fusing the sensor measurement data. For example, the ego-motion in the plurality of point clouds as sensor measurement data can be compensated for, in particular by shifting the point clouds, and the point clouds are fused into one common large point cloud at a predefined reference time point. The reference time point can form the assumed measurement time point of each environmental object labeled within the labeling basis.
[0028] In a preferred embodiment of the application it is advantageous if the first and the second sensor measurement data are present at at least one measurement time point each within the first time period. The first sensor measurement data can comprise a single measurement at one measurement time point within the first time period or a plurality of measurements at various measurement time points within the first time period. The second sensor measurement data can comprise a single measurement at one measurement time point within the first time period or a plurality of measurements at various measurement time points within the first time period. The measurement time points of the measurements of the first and the second sensor measurement data can be identical or different from each other.
[0029] If more than one measurement is made at the corresponding measurement time point of one environmental sensor in the first time period, all measurements or a selected one of the measurements can be used for the labeling basis, wherein the other measurements of this environmental sensor can then be discarded. Alternatively, the measurement data of this environmental sensor involved can be completely discarded for the first time period. Thus, the sensor measurement data of this environmental sensor involved can be excluded from the creation of the labeling basis. Conversely, in a subsequent time period, the sensor measurement data of this environmental sensor can be introduced into the consideration again at the time of creating the labeling basis.
[0030] In a special configuration of the application it is advantageous if, in addition, the output of the labeled second sensor measurement data by projection is made. Before the output of the labeled second sensor measurement data, in particular the labeling of the second sensor measurement data can be changed depending on a check.
[0031] In a special configuration of the application it is advantageous to carry out a creation of a second annotation basis which comprises at least the first and the second sensor measurement data merged in a common second time period which temporally follows the first time period and a further projection of the annotation of the first sensor measurement data of the second time period into the first and / or the second sensor measurement data of the second annotation basis. The further projection of the annotation of the first sensor measurement data of the first time period into the first and / or the second sensor measurement data of the second annotation basis can furthermore be related to the projection of the annotation of the first sensor measurement data into the second sensor measurement data of the first annotation basis.
[0032] In a preferred embodiment of the application it is provided that in addition the annotation of the first sensor measurement data of the first annotation basis is checked taking into account the further projection into the first and / or the second sensor measurement data of the second annotation basis. Then, depending on the check, an output of the first and / or the second sensor measurement data of the second time period which is annotated by the further projection can also be carried out.
[0033] A preferred configuration of the application is advantageous in which a creation of a third annotation basis is carried out which comprises at least the first and the second sensor measurement data merged in a common third time period which temporally follows the second time period and a further annotation of the first sensor measurement data of the third time period, wherein the further projection of the annotation of the first sensor measurement data into the first and / or the second sensor measurement data of the second annotation basis is carried out depending on the annotation of the first sensor measurement data of the first time period and the further annotation of the first sensor measurement data of the third time period. The further projection of the annotation of the first sensor measurement data of the first time period into the first and / or the second sensor measurement data of the second annotation basis can furthermore be related to the projection of the further annotation of the first sensor measurement data into the second sensor measurement data of the third annotation basis.
[0034] In a preferred embodiment of the application it is provided that in addition the annotation of the first sensor measurement data of the first annotation basis is checked taking into account the further annotation of the first sensor measurement data of the third time period. Then, depending on the check, an output of the first sensor measurement data of the third time period which is annotated by the further annotation can also be carried out.
[0035] In a preferred embodiment of the application it is provided that an additional projection of the further annotation of the first sensor measurement data into the second sensor measurement data of the third annotation basis is carried out. Furthermore, an output of the second sensor measurement data of the third time period which is annotated by the additional projection can be carried out.
[0036] In a particular embodiment of the application it is advantageous if the first environmental sensor has a first detection area and the second environmental sensor has a second detection area which at least partially differs from the first detection area, and the first annotation basis reproduces an environmental area which at least extends the first detection area. The first and the second detection area can be spaced apart from each other or partially overlap each other. The second and / or the third annotation basis can also reproduce an environmental area which at least extends the first detection area.
[0037] According to the application, a method for creating model data is also proposed. The neural network can be a convolutional neural network (CNN), which as an artificial neural network is designed specifically for processing input data having a grid structure. The neural network can be a recurrent neural network (RNN), in particular a long short-term memory network (LSTM). The neural network can have a Transformer architecture, in particular in the case of the use of an attention mechanism. The neural network can be built up from a plurality of layers, for example consisting of convolutional layers, pooling layers and fully connected layers for a CNN.
[0038] The model data can be used as training data for training the neural network. The neural network learns from these training data in such a way that it adjusts parameters (weights) in order to minimize the deviation between the predicted output and the actual annotation. During the training of the neural network, a loss is calculated and reduced by means of a gradient descent method.
[0039] The annotated first sensor measurement data can also be test data for testing the neural network, for example for performance evaluation of a trained neural network, and / or can also be validation data for validating the neural network, in particular during the learning.
[0040] The neural network can form a multi-modal model. The multi-modal model can process different input data, in particular input data of environmental sensors of different sensor modalities.
[0041] Furthermore, the application relates to a computing device for implementing the above-described method for annotating data and / or for implementing the above-described method for creating model data.
[0042] Further advantages and advantageous configurations of the application result from the description of the figures and the figures. BRIEF DESCRIPTION OF DRAWINGS
[0043] The application is described in detail below with reference to the accompanying drawings. Specifically, the figures show:
[0044] Figure 1 A method for annotating data in a particular embodiment of the application.
[0045] Figure 2 : a first annotation basis.
[0046] Figures 3a) to 3c) : a method for data annotation in another special embodiment of the invention.
[0047] Figure 4 : sensor measurement data of an environmental sensor. DETAILED DESCRIPTION
[0048] Figure 1 A method for data annotation is shown in a special embodiment of the invention. First, the method 10 for data annotation of sensor measurement data M of an environmental sensor S of a vehicle 12 comprises providing first sensor measurement data Ml of a first environmental sensor S1 and second sensor measurement data M2 of a second environmental sensor S2, which sensor measurement data relate to a common environmental scene 14 of a vehicle environment 20 of the vehicle 12 with an environmental object 16, here for example a tree 18. The first environmental sensor S1 can be a radar sensor and the second environmental sensor S2 can be a lidar sensor. The first environmental sensor S1 has a first detection area 22 and the second environmental sensor S2 has a second detection area 24 which is at least partially different from the first detection area.
[0049] Furthermore, a creation 26 of a first annotation basis Al is carried out, which first annotation basis comprises the first and second sensor measurement data Ml, M2 merged in a common first time period 28. Here, the first time period 28 is for example a time interval of the environmental scene, the measurement time point of the first sensor measurement data Ml and the measurement time point of the second sensor measurement data M2 lie within this time interval. As will be explained further below, these measurement time points can be different from one another. The first annotation basis Al is in particular a visual preparation for a fusion of the first and second sensor measurement data Ml, M2.
[0050] In Figure 2 , such a first annotation basis Al formed by the first and second sensor measurement data Ml, M2 is exemplarily shown. The first annotation basis Al reproduces an environmental area 30 which extends the first detection area 22, here for example a front area of the vehicle. Here, the first detection area 22 covers a left area of the extended environmental area 30 and the second detection area 24 covers a right area of the extended environmental area 30. Here, the first and second detection areas 22, 24 can be partially overlapping.
[0051] The first sensor measurement data M1 in the form of a point cloud 32 indicates a tree at the left lane edge 34, and the second sensor measurement data M2 likewise in the form of a point cloud 32 indicates a person 36 on the carriageway in the right lane area 38. Additionally, below the visual of the merged first and second sensor measurement data M1, M2, the first annotation basis Al comprises a camera image 40 of the common environment scenario.
[0052] Returning to Figure 1 Furthermore, the environment object 16 of the first sensor measurement data M1 of the first time period 28 is manually annotated 42. This annotation 42 is, for example, a recognition, labeling, contour detection and classification of the environment object 16, here of the tree as the only environment object in the first sensor measurement data M1 of the first time period 28. The annotation can be displayed, for example, as a bounding box 44. The dimensions of the bounding box 44 are also determined, for example, by the annotation 42.
[0053] Subsequently, a projection 46 of the annotation 42 of the first sensor measurement data M1 of the first time period 28 into the second sensor measurement data M2 of the first annotation basis Al is made. This projection 46 can be a projected bounding box 48 in the second sensor measurement data M2. Here, the dimensions of the bounding box 44, which are predefined by the annotation 42 of the first sensor measurement data M1, can be retained in the projected bounding box 48.
[0054] Furthermore, a creation 50 of a second annotation basis A2 comprising the merged first and second sensor measurement data M1, M2 in a common second time period 52 following in time on the first time period 28, and a creation 53 of a third annotation basis A3 comprising at least the merged first and second sensor measurement data M1, M2 in a common third time period 54 following in time on the second time period 52, is made.
[0055] Here, the further annotation 55 of the environmental object 16 of the first sensor measurement data M1 of the third time period 54 is preferably performed manually, and the further projection 56 of the annotation 42 of the first sensor measurement data M1 of the first time period 28 into the first and / or second sensor measurement data M1, M2 of the second annotation basis A2 is performed. Thus, the further projection 56 of the annotation 42 of the first sensor measurement data M1 of the first time period 28 into the first and / or second sensor measurement data M1, M2 of the second annotation basis A2 is in particular also related to the annotation 42 of the first sensor measurement data M1 of the first time period 28 and the further annotation 55 of the first sensor measurement data M1 of the third time period 54. Further, the additional projection 57 of the further annotation 55 of the first sensor measurement data M1 into the second sensor measurement data M2 of the third annotation basis A3 is performed.
[0056] The first, second and third time periods 28, 52, 54 reproduce the time sequence 58 in total, and the further projection 56 in the second annotation basis A2 can be an interpolation of the annotation 42 of the first sensor measurement data M1 of the first time period 28 and the further annotation 55 of the first sensor measurement data M1 of the third time period 54. Further, further projections calculated by interpolation can be performed for other time periods between the first and third time periods 28, 54, which are not shown here.
[0057] Finally, the output 60 of the annotated first sensor measurement data M1 is performed according to a check 62 of the annotation 42 of the first sensor measurement data M1, which is performed at least according to the projection 48 in the second sensor measurement data M2 of the first annotation basis A1, and additionally according to the further projection 56 in the first and / or second sensor measurement data M1, M2 of the second annotation basis A2 and the further annotation 55 of the first sensor measurement data M1 of the third time period 54, and preferably also according to the additional projection 57 of the further annotation 55 of the first sensor measurement data M1 into the second sensor measurement data M2 of the third annotation basis A3.
[0058] The check 62 can incorporate a manual visual comparison 64 of the annotations 42, 55 and the projections 46, 56, 57, and if necessary, a change 66 of the annotation 42 of the first sensor measurement data M1 of the first time period 28, for example by shifting 68 the bounding box 44, as shown here.
[0059] Fig. 3 shows a method for data annotation in another special embodiment of the present application. The method for data annotation 10 is explained exemplarily with sensor measurement data M of a plurality of environmental sensors S, here first sensor measurement data Ml of a first environmental sensor S1, second sensor measurement data M2 of a second environmental sensor S2, third sensor measurement data M3 of a third environmental sensor S3, fourth sensor measurement data M4 of a fourth environmental sensor S4 and fifth sensor measurement data M5 of a fifth environmental sensor S5.
[0060] In Figure 3a Fig. 3, the determination of the first time period 28 is shown, in which the measurement time points 70 of the individual environmental sensors S lie. For example, all environmental sensors S have exactly one measurement time point 70 within the first time period 28, except for the third environmental sensor S3. The third environmental sensor S3 has three measurement time points 70 within the first time period 28 and thus three third sensor measurement data M3.
[0061] In Figure 3b Fig. 3, the annotation 42 of the first sensor measurement data Ml of the first environmental sensor S1 as the primary sensor 72 selected with respect to the annotated environmental object is carried out.
[0062] In Figure 3c Fig. 3, the projection 46 of the annotation 42, here the bounding box 44, into the second, fourth and fifth sensor measurement data M2, M4, M5 is carried out. The third sensor measurement data M3 has been discarded due to the multiple measurements within the first time period 28 at a total of three measurement time points 70. Here, the projection 46 of the annotation 42, here the bounding box 44, involves a common reference time point 74, here the measurement time point 70 of the first environmental sensor S1. Since the environmental sensors S measure at different measurement time points 70, a dynamic environmental object, for example a car driving by, is detected at different spatial positions. However, in order to improve the comparability for checking the annotation 42, the manually positioned bounding box 44 is projected onto the reference time point 74 of the sensor measurement data M of the other environmental sensors S, wherein the motion of the dynamic environmental object is taken into account, as shown in Figure 4 Fig. 3.
[0063] Here, when annotating 42 the first sensor measurement data Ml for marking the environmental object, it is preferred that for each environmental object to be marked, a primary sensor 72 can be individually selected from all environmental sensors S and the reference time point 74 is determined based on the measurement time point 70 of the sensor measurement data M of the selected primary sensor 72.
[0064] Figure 4The sensor measurement data of the environment sensor is shown. The sensor measurement data M of another environment sensor, in particular of a laser radar sensor, is displayed, for example, in the form of a point cloud 32. In order to improve the comparability for checking the labeling, however, the manually positioned bounding box of the first sensor measurement data is projected onto the reference time point of the sensor measurement data M of the other environment sensor, in turn forming a bounding box 76 which is offset compared to the bounding box 78 of the other environment sensor at the time of measurement. Here, the movement of the dynamic environment object 16, as here a car which is driving, is taken into account. For this purpose, it is helpful to know the state of motion (speed, acceleration, rotational speed) of the environment object 16 in order to obtain a correct projection. The state of motion can be attached to the bounding box 76, for example, and is also valuable for other applications, for example for evaluating the performance of a sensor fusion system which predicts the state of motion of a vehicle during operation or for evaluating the performance of a radar detection network which should predict the speed of an environment object.
Claims
1. A method (10) for data annotation of sensor measurement data (M) of an environmental sensor (S) of a vehicle (12), the method having the following steps: providing first sensor measurement data (Ml) of a first environmental sensor (SI) and second sensor measurement data (M2) of a second environmental sensor (S2), which sensor measurement data are for a common environmental scene (14) of a vehicle environment (20) of the vehicle (12) with at least one environmental object (16), creating (26) a first annotation basis (Al) which comprises at least the first and second sensor measurement data (Ml, M2) which are merged in a common first time period (28), annotating (42) the first sensor measurement data (Ml) of the first time period (28) for marking the environmental object (16), projecting (46) the annotation (42) of the first sensor measurement data (Ml) into the second sensor measurement data (M2) of the first time period (28), checking (62) the annotation (42) of the first sensor measurement data (Ml) at least taking into account the projection (46) in the second sensor measurement data (M2), and outputting (60) the checked annotated first sensor measurement data (Ml). The first and second sensor measurement data (Ml, M2) exist in the first time period (28) at least at one measurement time point (70) respectively. Furthermore, an output of the second sensor measurement data (M2) which is annotated by the projection (46) is made. A creation (50) of a second annotation basis (A2) is made, which comprises at least the first and second sensor measurement data (Ml, M2) which are merged in a common second time period (52) which temporally follows the first time period (28), and a further projection (56) of the annotation (42) of the first sensor measurement data (Ml) of the second time period (52) into the first and / or second sensor measurement data (Ml, M2) of the second annotation basis (A2) is made. Furthermore, the checking (62) of the annotation (42) of the first sensor measurement data (Ml) of the first time period (28) is made taking into account the further projection (56) in the first and / or second sensor measurement data (Ml, M2) of the second annotation basis (A2). 2. The method (10) for data annotation according to claim 1, characterized in that: 3. The method (10) for data annotation according to claim 1 or 2, characterized in that: 4. The method (10) for data annotation according to any of the preceding claims, characterized in that: 5. The method (10) for data annotation according to claim 4, characterized in that: 6. The method (10) for data annotation according to claim 4 or 5, characterized in that: a third annotation basis (A3) is created (53) which comprises at least the first and second sensor measurement data (M1, M2) merged in a common third time period (54) which follows the second time period (52) in time; and a further annotation (55) of the first sensor measurement data (M1) of the third time period (54) is performed, wherein the further projection (56) of the annotation (42) of the first sensor measurement data (M1) into the first and / or second sensor measurement data (M1, M2) of the second annotation basis (A2) is performed in accordance with the annotation (42) of the first sensor measurement data (M1) of the first time period (28) and the further annotation (55) of the first sensor measurement data (M1) of the third time period (54).
7. The method (10) for data annotation according to claim 6, characterized in that: Additionally, the annotation (42) of the first sensor measurement data (M1) of the first time period (28) is checked (62) in accordance with the further annotation (55) of the first sensor measurement data (M1) of the third time period (54).
8. The method (10) for data annotation according to claim 6 or 7, characterized in that: an additional projection (57) of the further annotation (55) of the first sensor measurement data (M1) into the second sensor measurement data (M2) of the third annotation basis (A3) is performed.
9. The method (10) for data annotation according to any of the preceding claims, characterized in that: The first environmental sensor (S1) has a first detection area (22) and the second environmental sensor (S2) has a second detection area (24) which is at least partially different from the first detection area, and the first annotation basis (A1) reproduces an environmental area (30) which at least extends the first detection area (22).
10. A method for creating model data for a neural network, wherein, The model data comprises at least first sensor measurement data (M1) which is annotated by means of the method (10) for annotating data according to any one of the preceding claims. a third annotation basis (A3) is created (53) which comprises at least the first and second sensor measurement data (M1, M2) merged in a common third time period (54) which follows the second time period (52) in time; and a further annotation (55) of the first sensor measurement data (M1) of the third time period (54) is performed, wherein the further projection (56) of the annotation (42) of the first sensor measurement data (M1) into the first and / or second sensor measurement data (M1, M2) of the second annotation basis (A2) is performed in accordance with the annotation (42) of the first sensor measurement data (M1) of the first time period (28) and the further annotation (55) of the first sensor measurement data (M1) of the third time period (54).