Performance evaluation of object detection systems for vehicles and robots

Abstract

The invention relates to a method for evaluating the performance of an object detection system which monitors the environment of a vehicle and / or a robot in view of the presence and / or appearance of an object on the basis of measurement data obtained by observing the environment with at least one sensor, the method comprises the following steps: identifying at least one object instance identified by evaluation of measurement data with an object detection system as a false positively identified object, which does not correspond to any object actually present in the environment of the vehicle and / or robot; determining a criticality assessment for the false positively recognized object, the criticality assessment based at least in part on how a degree of relevance of the false positively recognized object, if actually present, with respect to a future behavioral plan of the vehicle or robot; the sought performance of the object detection system is determined based at least in part on a criticality assessment of one or more false positively recognized objects.

Description

TECHNICAL FIELD

[0001] The present invention relates to object detection systems which can be used in the environmental monitoring of vehicles and / or robots and which can be used, in particular, for at least partially automated driving of vehicles or robots. BACKGROUND

[0002] Driving a vehicle or robot in an operating site or even in public traffic requires continuous monitoring of the surroundings of the vehicle or robot. The future behavior planning of the vehicle or robot is, in particular, dependent on which objects are present in the respective surroundings. Therefore, the measurement data relating to the observations made of the surroundings using one or more sensors are evaluated with the aid of an object detection system which identifies object instances.

[0003] In training and testing such object detection systems, it is checked with the aid of prior knowledge about the respective observed scene whether all objects present in the scene according to this prior knowledge ("ground truth") are identified completely and correctly. In this way, it is ensured that the presence of each object triggers the desired response. SUMMARY

[0004] The present invention provides a method for evaluating the performance of an object detection system. The object detection system monitors the surroundings of a vehicle and / or robot with regard to the presence and / or occurrence of objects on the basis of measurement data obtained by observing the surroundings with the aid of at least one sensor. The measurement data can comprise, in particular, for example camera images, video images, thermal images, radar data, lidar data, ultrasound data or any combination of these measurement modalities. The object detection system can identify, in particular, for example individual object instances and determine for this purpose any desired properties, such as position, velocity vector, type or spatial dimensions, for example in the form of a "bounding box" enclosing the object.

[0005] Within the scope of the method, at least one object instance identified by the evaluation of the measurement data with the aid of the object detection system is identified as an object which has been falsely positively identified and does not correspond to any object actually present in the surroundings of the vehicle and / or robot. The identification can take place in any suitable manner. In particular, for example, at least one identified object instance is identified as an object which has been falsely positively identified by comparison with objects which, according to prior knowledge, are actually present in the surroundings of the vehicle and / or robot in the situation represented by the measurement data. Thus, for example, an identified object instance is likely to be falsely positively identified if it does not correspond to any of a certain number of expected object instances (Soll-Objektinstanzen) which, according to the prior knowledge, should be present. ​

[0006] Alternatively or in combination therewith, object instances that are identified false positives can also be identified, for example, from measurements from different perspectives and / or from a review (Zusammenschau) of various measurement modalities. For example, if an object is identified at a particular location that protrudes from the carriageway, but at the same time reliable depth information for this location indicates that there is just a flat carriageway surface here, then it is clear that this is an object that is identified false positive. The same applies if, for example, camera images from other perspectives contradict the assumed presence of an object that is identified. For example, if a bus is identified as an object instance, but at the same time the camera has a free view through the bus to the carriageway surface, then the bus cannot actually be present at the location where the bus is assumed to be identified. Thus, the basic physical a priori knowledge that buses are not transparent can be used to unmask (entlarven) object instances as false positives. Similarly, it is possible to check whether an identified object instance is actually physically present using conformity with other physical laws

[0007] Determining a criticality assessment for false-positive-identified objects The criticality assessment is based at least in part on how relevant a false-positive-identified object would be for future behavior planning of the vehicle or robot if it were actually present. Thus, for example, a false-positive-identified pedestrian in front of the vehicle or robot in the direction of travel should be assessed as very critical, because if the pedestrian were actually present, the vehicle or robot would have to brake or swerve immediately to avoid a collision. However, if the pedestrian is not actually present, such a maneuver would be completely unexpected for other road users and could, for example, lead to a rear-end collision. The occupants of a vehicle that brakes or swerves suddenly can also be injured, especially if the vehicle is a bus, train, or other short-distance vehicle, in which, as experience shows, not all passengers can be expected to hold on as required. However, if a pedestrian is identified in the air or at a safe distance from the carriageway, this is non-critical, because the vehicle or robot cannot collide with a pedestrian actually present at the respective location.

[0008] The sought performance of the object detection system is determined at least in part on the basis of the criticality assessment of one or more false-positive-identified objects. This can mean, in particular, for example, that in the performance assessment, false-positive-identified objects lead to a penalty (Malus) depending on their respective criticality assessment.

[0009] ​It has been recognized that the use of false-positive identified objects, in particular in combination with a criticality assessment of each single false-positive identified object, can lead to a more convincing evaluation of the performance of an object detection system. Many such systems are optimized with the goal of mainly avoiding missed false-negative identifications in order to miss a reaction of a vehicle or robot to a real existing object as little as possible. On the other hand, the susceptibility to false-positive identifications is increased. These systems identify object instances in a very low-threshold manner and rather identify one object instance too many than one object instance too few. If all false-positive identified objects are taken into account for the performance evaluation of the object detection system without considering their respective criticality, the low-threshold object identification is "penalized" too intensively, resulting in more missed false-negative identifications. The criticality assessment leads to the fact that only those false-positive identified objects that really cause an interference are "penalized".

[0010] In a particularly advantageous embodiment, a false-positive identified object is assessed as relevant for the future behavior planning of a vehicle or robot in response to the fact that the vehicle or robot can reach a position within a given prognostic horizon at which the false-positive identified object, if actually existing, can also reach. Thus, starting from the assumption that the object actually exists, it is assessed on the basis of this assumed situation whether a collision between the vehicle or robot on the one hand and the object on the other hand is possible by comparing the respective reachable positions. If this is not the case, the vehicle or robot has no reason to change its future behavior because of the existence of the object.

[0011] In this check, in particular the type of the false-positive identified object, which is determined for example by the object detection system and / or an object classifier downstream of the object detection system, can be used to determine the positions that the object can reach. Thus, for example, a pedestrian can only move within a much smaller area than a passenger car within a certain time, whereas the spatial dimensions of a passenger car are much smaller than those of a cargo car, so that the space occupied at any point in time is slightly smaller. On the other hand, the set of reachable positions of a passenger car and a cargo car is roughly the same, since these positions can be derived for example from the current speed and possible accelerations.

[0012] It can be checked, in particular, using any suitable physical motion model of the object, the vehicle or the robot, which positions the object, the vehicle or the robot can reach. To take account of the uncertainty of future motions, a motion model based on differential inclusions can be used, in particular. Such a motion model, in a suitable parameterization, can provide a reliable upper bound estimate of all positions that the object, the vehicle or the robot can reach at maximum within a certain time range. Differential inclusions are a generalization of ordinary differential equations in which the time derivative of the sought function is not related to a single value of the sought function, but rather to a set of values, which in turn depend on the value of the sought function (and optionally also directly on its argument). When using differential inclusions, the state variable of the motion equation, which exists in the form of a differential equation, is generalized to a set of possible values. The result provided is thus a set of possible positions that can be reached at maximum under the constraints of the model on which the differential equation is based. The probability of this occurring is irrelevant here.

[0013] In another particularly advantageous embodiment, only those objects that are falsely identified if they are actually present are taken into account in the determination of the performance of the object detection system for the future behavior planning of the vehicle or the robot. As already mentioned, this achieves that the good performance of the object detection system in avoiding the omission of false negatives is not "compromised" by those falsely identified objects that do not actually interfere with the operation of the vehicle or the robot at all. Thus, the conscious acceptance of false identifications of objects that occur to a certain extent is achieved, so that the object detection system does not develop towards a greater omission rate of false negatives.

[0014] In another particularly advantageous embodiment, the performance of the object detection system is additionally determined on the basis of how well objects that are actually present in the environment of the vehicle and / or the robot according to prior knowledge are identified completely and correctly by the object detection system in the situation embodied by the measurement data. It is thus assessed how well the information that is present in this case according to "ground truth" is assessed correctly. This optimization goal can be weighted in any way with respect to the optimization goal of avoiding particular disturbing false identifications, for example in the form of a cost function that is a weighted sum of the contributions related to the respective optimization goals.

[0015] In another particularly advantageous embodiment, in response to the determined performance of the object detection system meeting a pre-given criterion, the architecture of at least one neural network in the object detection system is changed with the goal of improving the performance of the object detection system. For example, a response can be made to the determined performance of the object detection system falling below a pre-given threshold value or otherwise not meeting expectations. The architecture of the neural network can then be designed, for example, to be more complex or deeper, for example, to improve the processing power.

[0016] In particular, the architecture of the at least one neural network can be changed, for example, in such a way that at least one hyperparameter characterizing the architecture is optimized in view of the goal of improving the performance of the object detection system. Thus, for example, after each change of the hyperparameters, the neural network can be retrained and then it is measured how this affects the performance of the object detection system.

[0017] In another particularly advantageous embodiment, in response to the determined performance of the object detection system meeting a pre-given criterion, at least one neural network in the object detection system is further trained with the goal of improving the performance of the object detection system. Thus, for example, if the determined performance falls below a threshold value or otherwise does not meet expectations, in particular new training examples can be fed to the neural network to make up for a "knowledge gap" that led to the poor performance of the object detection system.

[0018] These additional training examples can in particular be tailored, for example, in view of the fact that cases in which objects are falsely identified in an interfering manner are specifically learned to be better handled. Thus, in particular, for example, measurement data of cases in which more objects that are relevant for the future behavior planning of the vehicle or robot are falsely identified can be weighted more heavily in the further training.

[0019] If the performance of the object detection system is good, thus for example exceeds a predefined threshold or otherwise meets expectations, the object detection system can be put into production use. In response to the determined performance of the object detection system meeting the predefined criteria, measurement data can be fed to the object detection system, which is obtained by observing the environment of the vehicle and / or robot using the at least one sensor. A maneuvering signal can then be determined by using the object instances recognized by the object detection system. The maneuvering signal can then be used to maneuver the vehicle and / or robot. In this way, the likelihood of the vehicle or robot performing actions in response to objects being falsely positively recognized, which are not justified by the situation embodied by the measurement data and are completely unexpected, for example for other road users, is reduced.

[0020] This is particularly important in a further particularly advantageous embodiment, in which the maneuvering signal is selected which causes a braking maneuver and / or an avoidance maneuver of the vehicle and / or robot.

[0021] The method can in particular be fully or partially computer-implemented. The application therefore also relates to a computer program having machine-readable instructions which, when executed on one or more computers and / or computing instances, cause these computers and / or computing instances to carry out one of the methods described. In this sense, a control device for a vehicle and an embedded system for a technical device which are also capable of executing machine-readable instructions are to be regarded as computers. Computing instances can be, for example, virtual machines, containers or serverless execution environments, which can in particular be provided in the cloud.

[0022] The application also relates to a machine-readable data carrier and / or a download product having the computer program. A download product is a digital product which can be transmitted via a data network, i.e. downloaded by a user of a data network, which can for example be sold in an online shop for immediate download.

[0023] Furthermore, one or more computers and / or computing instances can be equipped with the computer program, the machine-readable data carrier or the download product. BRIEF DESCRIPTION OF DRAWINGS

[0024] Further measures for improving the application are explained further below in connection with the description of preferred embodiments of the application with the aid of the drawings. Therein:

[0025] Figure 1 An embodiment of a method 100 for evaluating the performance 7 of an object detection system 4 is shown;

[0026] Figure 2 An object instance 5 is shown exemplarily identified as a falsely positively recognized object 5#;

[0027] Figure 3 An exemplary evaluation of the performance 7 of the object detection system 4 is shown, which is based not only on actually existing object instances 5 but also on objects 5# that are falsely positively identified. DETAILED DESCRIPTION

[0028] Figure 1 is a schematic flow chart of an embodiment of a method 100 for evaluating the performance 7 of an object detection system 4. The object detection system 4 is designed for monitoring an environment 2 of a vehicle and / or robot 1 with respect to the presence and / or occurrence of objects based on measurement data 3 obtained by observing the environment 2 with at least one sensor.

[0029] In step 110, at least one object instance 5 that is identified by the evaluation of the measurement data 3 with the object detection system 4 is identified as a falsely positively identified object 5#. The falsely positively identified object does not correspond to any object that actually exists in the environment 2 of the vehicle and / or robot 1, that is, it does not actually exist in the environment 2 of the vehicle and / or robot 1.

[0030] According to block 111, the at least one identified object instance 5 is identified as a falsely positively identified object 5# by comparison with objects that actually exist in the environment 2 of the vehicle and / or robot 1 according to prior knowledge 5* in the light of the circumstances embodied by the measurement data 3.

[0031] Alternatively or in combination therewith, a fusion of measurement data 3 from multiple perspectives and / or measurement modalities can be employed, for example, to reveal the identified object instance 5 as a falsely positively identified object 5#. This will be explained in more detail in connection with Figure 2

[0032] In step 120, a criticality assessment 6 is determined for the falsely positively identified object 5#. The criticality assessment 6 is based at least in part on how relevant the falsely positively identified object 5# would be for a future behavior planning of the vehicle or robot 1 if it actually existed. In this case, the higher the relevance of the object 5# is, the more critical its false positive identification tends to be.

[0033] According to block 121, the falsely positively identified object 5# is assessed as relevant for a future behavior planning of the vehicle or robot 1 in response to the vehicle or robot 1 being able to reach a position la, 5a within a pre-given prediction range that the falsely positively identified object 5# can reach if it actually existed. Thus, for example, it can be determined:

[0034] • on the one hand those positions la that the vehicle or robot 1 can reach within the prediction range, and​

[0035] • The other aspect determines those positions 5a which can be reached by the falsely positively identified objects 5# in case they actually exist.

[0036] If there is an intersection between the set of positions 1a and the set of positions 5a, the falsely positively identified objects 5# are relevant for the future behavior planning of the vehicle or robot 1.

[0037] In particular, e.g. according to block 121a, the type of the falsely positively identified objects 5# determined by the object detection system 4 and / or an object classifier downstream of the object detection system 4 can be used to determine the positions 5a which can be reached by the object 5#.

[0038] In step 130, the sought performance 7 of the object detection system 4 is determined based at least partly on the criticality assessment 6 of the one or more falsely positively identified objects 5#.

[0039] According to block 131, only those falsely positively identified objects 5# are taken into account in the determination of the performance 7 of the object detection system 4 which are relevant for the future behavior planning of the vehicle or robot 1 if they actually exist.

[0040] According to block 132, the performance 7 of the object detection system 4 can additionally be determined based on how well objects actually existing in the environment 2 of the vehicle and / or robot 1 according to prior knowledge 5* are identified completely and correctly by the object detection system 4 in the situation embodied by the measurement data 3. Thus, e.g. a cost function can be established to assess the performance 7, wherein not only the behavior on actually existing objects but also the behavior on falsely positively identified objects 5# is included.

[0041] In step 140, it can be checked whether the determined performance 7 of the object detection system 4 meets a pre-given criterion, i.e. e.g. whether it falls below a pre-given threshold or otherwise does not meet expectations. If this is the case (true value 1), in step 150 the architecture of at least one neural network in the object detection system 4 can be changed with the goal to improve the performance 7 of the object detection system 4. To this end, in particular, at least one hyperparameter characterizing the architecture can be optimized, e.g. according to block 151, in view of the goal to improve the performance 7 of the object detection system 4.

[0042] Alternatively or in combination therewith, in a step 160, at least one neural network in the object detection system 4 can be further trained with the goal to improve the performance 7 of the object detection system 4. For this purpose, in particular, for example, in accordance with a block 161, training examples of the measurement data 3 are weighted more highly, in which cases there are more objects that are relevant for a future behavior planning of the vehicle or robot 1 that are falsely identified.

[0043] In a step 170, it can be checked whether the determined performance 7 of the object detection system 4 meets further criteria, i.e. for example, whether a pre-given threshold is exceeded or a pre-given expectation is otherwise met. If this is the case (true value 1), the object detection system 4 can be put into production use.

[0044] Then, for example, in a step 180, measurement data 3 can be fed to the object detection system 4, wherein the measurement data are obtained by observing the environment 2 of the vehicle and / or robot 1 using at least one sensor. In a step 190, a maneuver signal 190a can be determined by using the object instances 5 identified by the object detection system 4. The maneuver signal 190a can then be used to maneuver the vehicle 1 or robot 1.

[0045] Here, in particular, for example, in accordance with a block 191, a maneuver signal 190a can be selected that causes a braking maneuver and / or an evasive maneuver of the vehicle and / or robot 1. Such a maneuver can have a particularly disruptive effect when it is performed on a falsely identified object 5#, i.e. without a really good reason.

[0046] Figure 2 Two ways of being able to identify an identified object instance 5 as a falsely identified object 5# are shown. In a first way, in accordance with a block 171, the object detection system 4 is trained in such a way that it learns to recognize the situation in which the object instance 5 is a falsely identified object 5#. Figure 2 In the scenario 10 shown in Fig. 1, a vehicle 1 has a camera system as a first sensor 9 and a radar sensor as a second sensor 9' for observing the surroundings 2 of the vehicle 1. The vehicle 1 is driving on a road 11 leading to a house 12. On this road 11, a pedestrian pictogram 13 is drawn to alert the vehicle driver to pay particular attention to pedestrians. In the situation shown in Fig. 1, the evaluation of the measurement data 3 recorded by the camera system 9 falsely identifies this pedestrian pictogram 13 as a pedestrian 13' standing on the road. Figure 2

[0047] ​The radar sensor 9', however, recognizes that its radar beam 9a reaches the house 12 unhindered and is reflected there. This is not possible if the pedestrian 13' is indeed standing upright in the beam path. Therefore, a review of the measurement data 3 of the camera system 9 and the measurement data of the radar sensor 9' provides the information that the recognized upright pedestrian 13' is a falsely recognized object 5# as object instance 5.

[0048] If prior knowledge 5* in the form of a list of expected object instances 5 that should be present in the scene 10 is known for this scene 10, it is also possible to determine which recognized object instances 5 are falsely recognized objects 5# by comparing them to this list. In the example shown in Figure 2 the list contains only the street 11 and the house 12, but not the upright pedestrian 13'. It is therefore clear that the upright pedestrian 13' is a falsely recognized object 5#.

[0049] Figure 3 It is shown how the performance 7 of the object recognition system 4 is evaluated not only on the basis of objects 5 that actually physically exist, but also on the basis of falsely recognized objects 5#.

[0050] First, the object instances 5 recognized by the object recognition system 4 are summarized with the measurement data 3 present here as an image into a combination 3+5. In the example shown in Figure 3 the combination 3+5 is an image to which a bounding box is superimposed for each recognized object instance 5. On the basis of the prior knowledge 5* about those object instances 5 that physically exist, it is now possible to check to what extent the object instances listed there are recognized completely and correctly. For this purpose, for example, the "intersection over union" (IoU) between the expected object instances 5 according to the prior knowledge 5* and the actually recognized object instances 5 can be calculated. This result can be included as a component in the calculation of the performance 7 of the object recognition system 4.

[0051] The IoU measure, however, is only suitable for objects for which there is a correspondence in the prior knowledge 5*. For falsely recognized objects 5#, no such correspondence exists, so that they are not "penalized" in the IoU measure. Therefore, according to the method presented here, falsely recognized objects 5# are identified in a different way. It is then determined which positions la can be reached by the vehicle or robot 1 within a pre-given time horizon. At the same time, it is also determined which positions 5a can be reached by the falsely recognized objects 5# if they do exist within the same time horizon.

[0052] Figure 3The locations 1a and 5a are drawn in the form of a two-dimensional map, which shows a top view of the observed scene with coordinates x and y.

[0053] exist Figure 3 In the example shown, for the first false positive object 5#, it is determined that the possible location 5a of the object does not overlap with the possible location 1a of vehicle 1. The false positive object 5# is evaluated as irrelevant to the future behavior planning of vehicle 1 and is therefore evaluated as low criticality (criticality 6 = ↓).

[0054] The second false positive object 5# is accessible to a set of locations 5a, which overlap at least in the peripheral region with locations 1a accessible to vehicle 1. This false positive object 5# is assessed as relevant to the future behavior planning of vehicle 1 and is therefore assessed as critical (criticality 6 = ↑).

[0055] The third false positive, object 5#, is a pedestrian whose hypothetical reachable location 5a largely overlaps with the reachable location 1a of vehicle 1. Therefore, this pedestrian is assessed as highly relevant to the future behavior planning of vehicle 1 and is thus assessed as highly critical (criticality 6 = ↑↑).

[0056] exist Figure 3 In the example shown, the criticality 6 (5#) of each false positive object 5# is more precisely quantified in the form of Time to Collision (TTC) until a collision may occur. This criticality 6 (5#) is incorporated into the performance evaluation 7 of the object detection system 4.

Claims

1. A method (100) for evaluating a performance (7) of an object detection system (4) that monitors an environment (2) of a vehicle and / or robot (1) in terms of the presence and / or occurrence of objects based on measurement data (3) obtained by observing the environment (2) with at least one sensor, the method comprising the following steps: • at least one object instance (5) identified by an evaluation of the measurement data (3) with the object detection system (4) is identified (110) as a falsely positively identified object (5#) that does not correspond to any object actually present in the environment (2) of the vehicle and / or robot (1); • a criticality assessment (6) is determined (120) for the falsely positively identified object (5#) that is based at least in part on how relevant the falsely positively identified object (5#) would be for a future behavior planning of the vehicle or robot (1) if it were actually present; • the sought performance (7) of the object detection system (4) is determined (130) based at least in part on the criticality assessment (6) of one or more falsely positively identified objects (5#).

2. The method (100) of claim 1, wherein The falsely positively identified object (5#) is assessed (121) as relevant for a future behavior planning of the vehicle or robot (1) in response to the vehicle or robot (1) being able to reach a location (la, 5a) within a pre-given prediction range that the falsely positively identified object (5#) would also be able to reach if it were actually present.

3. The method (100) of claim 2, wherein A type of the falsely positively identified object (5#) determined by the object detection system (4) and / or an object classifier downstream of the object detection system (4) is used (121a) to determine a location (5a) that the object (5#) would likely reach.

4. The method (100) according to any one of claims 1 to 3, wherein Only those falsely positively identified objects (5#) that would be relevant for a future behavior planning of the vehicle or robot (1) if they were actually present are taken into account (131) in the determination of the performance (7) of the object detection system (4).

5. The method (100) according to any one of claims 1 to 4, wherein, The performance (7) of the object detection system (4) is additionally determined (132) based on how well objects actually present in the environment (2) of the vehicle and / or robot (1) according to prior knowledge (5*) are identified completely and correctly by the object detection system (4) in light of what is embodied by the measurement data (3).

6. The method (100) according to any one of claims 1 to 5, wherein In response to the determined performance (7) of the object detection system (4) meeting (140) a pre-given criterion, an architecture of at least one neural network in the object detection system (4) is changed (150) with the goal of improving the performance (7) of the object detection system (4).

7. The method (100) of claim 6, wherein The architecture of at least one neural network is changed in such a way that at least one hyperparameter characterizing the architecture is optimized (151) in view of the goal of improving the performance (7) of the object detection system (4).

8. The method (100) according to any one of claims 1 to 7, wherein In response to the determined performance (7) of the object detection system (4) meeting (140) a pre-given criterion, at least one neural network in the object detection system (4) is further trained (160) with the goal to improve the performance (7) of the object detection system (4).

9. The method (100) of claim 8, wherein In the further training, measurement data (3) in which more object instances relevant for a future behavior planning of the vehicle or robot (1) are falsely identified are more highly weighted (161).

10. The method (100) according to any one of claims 1 to 9, wherein At least one identified object instance (5) is identified (111) as a falsely identified object (5#) by comparison with objects actually existing in the environment (2) of the vehicle and / or robot (1) according to prior knowledge (5*) in the situation embodied by the measurement data (3).

11. The method (100) according to any one of claims 1 to 10, wherein In response to the determined performance (7) of the object detection system (4) meeting a pre-given criterion (170), • measurement data (3) are fed (180) to the object detection system (4), wherein the measurement data are obtained by observing the environment (2) of the vehicle and / or robot (1) with at least one sensor; • a maneuvering signal (190a) is determined (190) by using object instances (5) identified by the object detection system (4); and • the vehicle and / or robot (1) is maneuvered (200) with the maneuvering signal (190a).

12. The method (100) of claim 11, wherein a maneuvering signal (190a) is selected (191) that causes a braking maneuver and / or an evasive maneuver of the vehicle and / or robot (1).

13. A computer program having machine-readable instructions which, when executed on one or more computers and / or computing instances, cause the computers and / or computing instances to perform the method (100) according to any one of claims 1 to 12.

14. A machine-readable data carrier and / or a download product having the computer program according to claim 13.

15. One or more computers and / or computing instances having the computer program according to claim 13, and / or having the machine-readable data carrier and / or the download product according to claim 14.

Images