Urban Human-Machine Interaction Testing Environment

Abstract

The invention relates to a method for operating a test bed for vehicles using a simulation means and a motion capture system, comprising the following work steps: using the simulation means to generate a virtual test environment with at least one virtual creature and at least one virtual vehicle, one of the virtual creatures being a virtual representation of a real creature and one of the virtual vehicles being a virtual representation of a vehicle equipped with a driving assistance system, and additionally at least a part of the vehicle acts as a real test subject on the test bed, the driving assistance system running on the basis of the virtual test environment, in particular being stimulated; based on the generated virtual environment, stimulating the real creature in the motion capture system with stimuli; capturing motion data using the motion capture system, the motion data describing a temporal sequence of poses of at least one part of the anatomical structure of the real creature; and recording the captured motion data.

Description

[Technical field]

[0001] The present invention relates to a method for operating a test bed for a vehicle using a simulation means and a motion capture system, a method for operating a test bed, a system for operating a test bed, a computer program and a computer program product. [Background technology]

[0002] An autonomous or semi-autonomous vehicle is equipped with an array of sensors and algorithms that convert the signals from the sensors into an image of the surroundings.

[0003] In addition to systems which are used especially for safe driving, such as ABS (anti-lock braking system) and ESP (electronic stability program), numerous driver assistance systems are provided in the passenger car and commercial vehicle sector.

[0004] Driving assistance systems that are already used to increase active road safety include parking assistance systems and ACC (Adaptive Cruise Control), which adapts the desired speed selected by the driver to the distance to the vehicle ahead. Further examples of such driving assistance systems are, in addition to ACC, ACC Stop & Go systems, which automatically continue driving the vehicle in case of a traffic jam or when the vehicle is stationary, lane keeping or lane assistance systems, which automatically keep the vehicle in its lane, and pre-crash systems, which prepare or initiate braking in case of a possible collision, for example to remove kinetic energy from the vehicle, and, if necessary, initiate further measures if a collision is unavoidable.

[0005] These driver assistance systems increase safety on the roads by warning the driver in critical situations and initiating autonomous interventions to prevent or minimize accidents, for example by activating emergency braking functions, while also improving driving comfort with functions such as automated parking, automatic lane keeping and automatic distance control.

[0006] The safety and comfort advantages of a driving assistance system can only be perceived positively by the vehicle occupants if the assistance provided by the driving assistance system is performed safely, reliably and as comfortably as possible.

[0007] Furthermore, all driver assistance systems, depending on their function, must be able to carry out the scenarios that arise in traffic with maximum safety for their own vehicle and without endangering other vehicles or other traffic participants.

[0008] In this regard, the degree of automation of each vehicle is classified into so-called automation levels 1 to 5 (see, for example, the standard SAE J3016). The present invention relates in particular to vehicles equipped with driver assistance systems with automation levels 3 to 5, which are generally considered to be highly automated (3 and 4) or autonomous driving (5).

[0009] The challenges in testing such systems are manifold. In particular, it is necessary to find a balance between the test burden and the test coverage. The main challenge in testing ADAS / AD functions (ADAS - Advanced Driver Assistance Systems, AD - Automated Driving) is to demonstrate that the functionality of the driver assistance systems is guaranteed in all possible situations, especially in critical driving situations. Such critical driving situations carry a certain degree of risk, since a non-reaction or an incorrect reaction of the respective driver assistance system can lead to an accident.

[0010] Up to now, tests to test the behavior of driver assistance systems on humans have been carried out using dummies. Typically, dummies are images of a human being, usually male, that are average in terms of anatomical scale and proportions. Dummies are not only expensive but also difficult to handle, so that only one-off and not particularly realistic results can be obtained regarding the behavior of the driver assistance system.

[0011] From the prior art it is known to use real test drive data of a real test fleet of vehicles and to extract scenarios from the recorded data in order to validate and verify driver assistance systems.

[0012] Furthermore, Patent Document 1 discloses a method for testing a vehicle and its algorithms in a situation involving pedestrians. [Prior art documents] [Patent documents]

[0013] [Patent Document 1] GB Patent Application Publication No. 2563400 Summary of the Invention [Problem to be solved by the invention]

[0014] It is an object of the present invention to provide an improved method for operating a test bed, an improved method for operating a test bed using simulation means, as well as corresponding systems, computer programs and computer program products, in particular to achieve a realistic representation of living creatures and their motion patterns. [Means for solving the problem]

[0015] The problem is solved by the independent claims. Advantageous embodiments are claimed in the dependent claims.

[0016] A first aspect of the invention relates to a method for operating a test bench for a vehicle using a simulation means and a motion capture system, said method comprising the following working steps:

[0017] A work step of creating a virtual test environment with at least one virtual creature and at least one virtual vehicle by means of a simulation means, one of the virtual creatures being a virtual representation of a real creature and one of the virtual vehicles being a virtual representation of a vehicle equipped with a driving assistance system, and additionally at least a part of the vehicle acting as a real test object on a test stand, the driving assistance system being activated and in particular stimulated based on the virtual test environment. The real vehicle may in particular at least partly act as a test object on the test stand.

[0018] The method further includes stimulating a real creature in a motion capture system with stimuli based on the created virtual environment and capturing motion data using the motion capture system, the motion data describing a temporal sequence of poses of at least one portion of an anatomical structure of the real creature.

[0019] Additionally, the method may include recording the captured motion data.In particular, the method may include operating a test stand with a virtual test environment.

[0020] A second aspect of the invention relates to a method for operating a test bench using simulation means, in particular according to the first aspect of the invention, comprising:

[0021] The simulation means is used to create a virtual test environment with at least one virtual creature and at least one virtual vehicle, one of the virtual creatures being a virtual representation of a real creature and one of the virtual vehicles being a virtual representation of a vehicle with a driving assistance system. One of the vehicles, which is a virtual representation of the vehicle with the driving assistance system (also called self-object or self-vehicle), at least partially acts as a test subject on the test bench. In other words, at least a part of the vehicle acts as a real test subject on the test bench. The driving assistance system is activated based on the virtual test environment and is particularly stimulated.

[0022] Additionally, the method includes capturing motion data, particularly with a motion capture system, the motion data describing or representing a temporal sequence of poses of at least one part of the anatomy of a real living being.

[0023] The method further includes, inter alia, recording a scenario generated by a response of the driver assistance system to the captured motion data, and the response of the driver assistance system to the captured motion data and the virtual creature is taken into account in forming the virtual test environment.

[0024] Preferably, the method further comprises generating test scenarios for testing a driver assistance system for the vehicle.

[0025] A third aspect of the invention relates to a system for operating a test stand for a vehicle, the system being particularly configured and / or included and provided for carrying out a method, in particular for carrying out a method according to the above aspect.

[0026] The system preferably comprises a simulation means configured for creating a virtual test environment with at least one virtual creature and at least one virtual vehicle and for operating, in particular stimulating, a driving assistance system based on the virtual test environment, one of the virtual creatures being a representation of a real creature and one of the virtual vehicles being a virtual representation of a vehicle equipped with the driving assistance system, and additionally at least a part of the vehicle being operated as a real test subject on a test stand.

[0027] Additionally, the system preferably comprises a motion capture system for capturing motion data, the motion data describing a temporal sequence of poses of at least one part of the anatomy of a real living being.

[0028] The system preferably further comprises a stimulation means configured to stimulate a real creature in the motion capture system based on the created virtual environment by the stimulation.Furthermore, the system preferably comprises a storage means for recording the captured motion data.

[0029] A fourth aspect of the invention relates to a system for operating a test bed, in particular according to the system of the third aspect, comprising simulation means arranged to create a virtual test environment with at least one virtual creature and at least one virtual vehicle, one of the virtual creatures being a virtual representation of a real creature and one of the virtual vehicles being furthermore a virtual representation of a vehicle equipped with a driver assistance system.

[0030] The system is further configured to additionally operate the vehicle at least partially as a real test subject on a test stand and to activate, in particular stimulate, a driving assistance system based on the virtual test environment. Furthermore, the system comprises means, in particular a motion capture system or interface, configured to capture motion data describing a temporal sequence of poses of at least one part of the anatomical structure of a real living being.

[0031] Furthermore, the system has a storage means for recording scenarios generated by the reaction of the driving assistance system to the captured motion data, and the reaction of the driving assistance system to the captured motion data and the virtual creatures is taken into account when forming the virtual test environment.

[0032] A system and / or means in the sense of the present invention may be configured as hardware and / or software, in particular comprising at least one, preferably digital, processing unit, in particular a microprocessor unit (CPU), a graphics card (GPU) etc., connected by data or signals to a storage system and / or a bus system, and / or one or more programs or program modules. The processing unit may be configured to process instructions implemented as a program stored in the storage system, to detect input signals from the data bus and / or to send output signals to the data bus. The storage system may comprise one or more, in particular different storage media, in particular optical, magnetic, solid state and / or other non-volatile media. The program may be of such a nature that it is capable of embodying or executing the methods described herein, such that the processing unit executes the steps of such methods and thus in particular is capable of operating or monitoring a device.

[0033] A fifth aspect of the invention relates to a computer program or computer program product comprising instructions, in particular stored on a computer readable and / or non-volatile storage medium, which when executed by one or more computers or in particular a system for operating a test bench for vehicles, cause the computer or the system to perform a method for operating a test bench for vehicles using simulation means and a motion capture system, in particular according to the embodiments described above.

[0034] In one embodiment, the computer program product comprises, and may in particular be, a storage medium, in particular a computer-readable and / or non-volatile storage medium, for storing a program or instructions or on which the program or instructions are stored. In one embodiment, the execution of said program or instructions by a system or control device, in particular a computer or an assembly of computers, causes the system or control device, in particular a computer, to perform one or more of the methods or steps thereof described herein, or the program or instructions are configured for this purpose.

[0035] A scenario in the sense of the present invention is preferably formed from a time sequence of, in particular, static scenes, where the scenes show, for example, the spatial arrangement of at least one object, in particular at least one virtual creature, in particular the arrangement of traffic participants, and / or the arrangement of, in particular, immobile virtual objects, virtual creatures, in particular virtual traffic participants, relative to an ego object. In particular, a scenario can include driving situations in which the driving assistance system at least partially controls a vehicle, referred to as ego vehicle, which is equipped with the driving assistance system, and for example autonomously executes at least one vehicle function of the ego vehicle.

[0036] Motion data of at least one part of the anatomy of a real creature in the sense of the present invention is understood to mean that preferably at least a minimal part of the body part of the real creature that can be moved by joints and / or muscles is represented by motion data, which motion data describes the temporal sequence of this minimal part, in particular this makes it possible to record at least substantially all possible motions of the real creature, represented by motion data and represented in the virtual test environment by a virtual representation of the real creature.

[0037] A driving situation in the sense of the present invention preferably describes a situation that should be taken into account for the selection of an appropriate behavior pattern of the driving assistance system at a certain time. Therefore, the driving situation is preferably subjective in that it represents the viewpoint of the own vehicle. The driving situation further preferably includes the conditions, possibilities and influencing factors related to the action. The driving situation is further preferably derived from the scene by a process of information selection, for example based on mission-specific temporal, but also on persistent goals and values.

[0038] Driving behavior in the sense of the present invention is preferably the behavior of the driving assistance system by its actions and reactions in the vehicle's environment.

[0039] Quality in the sense of the present invention preferably characterizes the scenario that is simulated.Quality is preferably understood to be the quality or nature of the scenario that is simulated with respect to its suitability for testing driver assistance systems.In this case, more critical scenarios preferably have higher quality.Preferably, the riskiness of the driving situation resulting from each scenario for the tested driver assistance system is the measure of the quality of the scenario.

[0040] A pose in the sense of the present invention is the spatial position, in particular a combination of position and orientation, of a part of the anatomical structure of an object, in particular of a living being. In particular, the pose may relate to separately movable anatomical parts of a living being, which pose may be captured using a motion capture system, in particular in the overall context of the pose of the living being. The capture may be performed in particular using a stereo camera, infrared tracking, image recognition or comparable systems, in particular using motion capture systems and methods for motion capture in three-dimensional volumes.

[0041] A motion capture system in the sense of the present invention can capture motion with or without markers, in particular active or passive markers, in particular through pattern recognition, silhouette tracking and / or the like. The motion capture system may in particular be connected to the test stand, in particular in wireless or wired data communication. The motion capture system may be locally separate from the test stand.

[0042] The invention is based on the idea of ​​a realistic virtual representation of a real creature in motion on a test bed of a vehicle with a driver assistance system. By recording the captured motion data a database with the captured motion data can be created. This recorded motion data can be accessed to generate test scenarios, in particular scenarios. The recorded motion data can be combined and / or supplemented with other motion data. The recorded motion data can in particular form a motion atlas, which is used as a basis for simulating the motion of a virtual creature or avatar.

[0043] Furthermore, the present invention can be used to create a test environment that allows testing or optimizing a driving assistance system that is intended for interaction with living beings. Furthermore, during the interaction of a real vehicle with a real living being, situations that are particularly dangerous for at least the living being can be depicted. In other words, it is possible to create situations that may be dangerous for the life of the living being, particularly by repeatedly making small changes to the motion data, preferably generated or calculated by a computer, so that the driving assistance system can be optimized and / or trained. This can improve the quality of the virtual test environment, particularly the scenarios, particularly with respect to the living being.

[0044] In advantageous embodiments, the captured motion data may be associated with stimuli for a real creature within the motion capture system, and the stimuli associated with the captured motion data may be stored as well. In further advantageous embodiments, the captured motion data and the response of a driver assistance system to the captured motion data may be taken into account when creating the virtual test environment.

[0045] In a further advantageous embodiment, the recorded scenario may be associated with the captured motion data, and the resulting motion data pertaining to the scenario may likewise be stored.

[0046] In further advantageous embodiments, the captured motion data may be taken into account in an iterative manner when creating the virtual test environment.

[0047] This allows the motion data to be recorded only once and / or to be repeated with the motion data, particularly in a virtual test environment that may be larger than the captureable volume of the motion capture system, particularly where the pose sequence is outside of the area in which the motion capture system can capture motion. Advantageously, this allows the motion capture system to be designed to be smaller than the virtual test environment, yet still represent the motion data of a real creature by a virtual simulation (representation) of the real creature in the virtual test environment.

[0048] By combining the captured motion data with at least one, in particular visual, tactile and / or auditory, stimulus, the motion atlas can store the reactions and / or interactions of the real creature, in particular with the self-object, depending on the situation. These can in particular include the creature's direct interactions with the self-object, preferably touching, pushing, etc., which can stimulate the real creature in the motion capture system, in particular via a corresponding interface, such as for example a tactile glove (in English "feedback glove"). Furthermore, the captured motion data can include the distance of the self-object relative to the virtual creature in the virtual test environment, the position of the objects in the virtual test environment and / or their time derivatives, in particular their velocity or acceleration, data on parts of the self-object or data on the totality of the parts contained in the self-object.

[0049] The repetition of the captured motion data in the virtual test environment may in particular relate to motion data representing a sequence of poses of at least one part of the anatomical structure of a real living organism. The repetition of the captured motion data allows to generate a changed stimulus for the driving assistance system, in particular through the repetition of a temporal sequence of poses at different or the same location in the virtual test environment. Advantageously, this allows the driving assistance system to optimize its response to a particularly homogeneous stimulus that occurs repetitively and / or at different locations in the virtual test environment. In particular, this allows to repeat a pose, in particular a pose that is generally perceived as abnormal, or a temporal sequence of a particularly abnormal pose, preferably as a stimulus for the driving assistance system. The repetition of the captured motion data in the virtual test environment allows to (better) train the driving assistance system compared to non-repeatable motion data, in particular by using machine learning methods.

[0050] In this case, the variation space of possible scenarios is generally constituted by many dimensions, for example by different road properties, the behavior of other traffic participants, weather, etc. A further dimension can be constituted by motion data. It is particularly important for testing driver assistance systems to extract from this almost infinite multidimensional parameter space parameter sets relating to particularly critical scenarios that may lead to particularly abnormal or dangerous driving situations. This can be determined in particular by combining the motion data with at least one stimulus.

[0051] In a further advantageous embodiment, the test object can be operated as Hardware-in-the-Loop, in particular as Vehicle-in-the-Loop.

[0052] This allows the hardware under test to react to scenarios in the virtual test environment and, on the other hand, the virtual test environment to react to the hardware under test. Advantageously, the "loop" can operate in real time, which allows the hardware under test to be specifically tested and / or optimized under real time conditions.

[0053] In a further advantageous embodiment, the capture may be coupled with at least one, in particular visual, tactile and / or auditory, stimulus to the creature in the motion capture system from the virtual test environment.

[0054] The motion capture system may be particularly configured such that one or more stimuli, particularly one stimulus or multiple different stimuli simultaneously, may be presented to or recorded by a real creature within the motion capture system. These may be directional, particularly in the case of auditory stimuli, such as auditory stimuli from one or more directions that are localizable to the real creature, or auditory stimuli that are generally unlocalizable, particularly low frequency stimuli.

[0055] This allows the motion data to be clustered, in particular with their links, whereby advantageously, in particular test scenarios, in particular scenarios with clustered motion data, can be modified in order to generate variations of the virtual test environments.

[0056] In a further advantageous embodiment, the repetitive consideration of the captured motion data, in particular the repetition of the captured motion data, may comprise a modification of at least one part of the anatomical structure. Alternatively or additionally, the repetition of the motion data may comprise a modification of the temporal sequence of the pose of at least one part of the anatomical structure.

[0057] This allows the virtual creature to be adapted to various features, in particular of at least one part of the anatomy, which may occur biologically in a real creature comparable to the real creature. Furthermore, by varying the temporal sequence of the poses, it is possible to highlight parts of the motion, in particular to accelerate or decelerate them. Advantageously, this allows the driving assistance system to be specifically trained and / or optimized, in particular to various features of the motion, in particular to the temporal sequence of the poses.

[0058] In an advantageous embodiment, at least one portion of the anatomical structure may be altered based on an empirical quantile of the portion of the anatomical structure.

[0059] This allows adapting the scenario, in particular to the characteristics of at least one part of the anatomy, according to empirical quantiles of real creatures, in particular according to corresponding percentiles of the at least one part of the anatomy, which allows in particular to generate a large number of virtual creatures, in particular for a scenario and / or for a large number of scenarios, in particular by capturing motion data.

[0060] In a further advantageous embodiment, the time sequence of the virtual test environment, when iterating over the consideration, in particular the motion data, can be faster or slower than the time sequence of the motion data or faster or slower than real time, which makes it possible, in particular when the time sequence is slower than real time, to carry out, for example, computationally complex simulations, in particular finite element simulations for structural optimization, computational fluid dynamics simulations for shape optimization or thermodynamics simulations for system optimization.

[0061] This allows, in particular, a better understanding of the scenario in the test environment and / or an improved tuning or configuration of the driver assistance system.Furthermore, this allows, in particular, mechanical and / or structural design / changes to the self-object associated with the creature to be tried out and / or tested, in particular by simulation.Furthermore, the changeable temporal sequence of the virtual test environment allows testing a large number of scenarios, in particular variations of a large number of scenarios, in particular with a change in the temporal sequence of the pose of the virtual creature, in particular when the temporal sequence is faster than real time.

[0062] In a further advantageous embodiment, the iterative consideration, in particular the iteration, may further include iterative consideration, in particular the iteration, of second captured motion data different from the first captured motion data in forming the virtual test environment.

[0063] This allows, in particular, the introduction of repetitions of motion sequences with different components into a scenario, in particular, the repetition of a gesture with a different configuration from another gesture, in particular, allowing a driver assistance system to be specifically optimized in response to contrast with different temporal sequences of poses, rather than being trained or optimized for a particular temporal sequence of poses.

[0064] In a further advantageous embodiment, the method may furthermore comprise, inter alia, the following working step: determining transition data from the first captured motion data to the second captured motion data, the transition data describing a temporal and / or spatial transition from the first captured motion data to the second captured motion data.

[0065] The method may further include: reproducing in time, in the virtual testing environment, the transition data between the first captured motion data and the second captured motion data.

[0066] This allows, in particular, motion data which by themselves represent non-continuous temporal sequences of poses to be combined and depicted and / or displayed in the virtual test environment. In particular, this allows combining motion data which are not relevantly captured and / or recorded in a motion capture system. Advantageously, this allows combining motion data of different real creatures with one another, in particular forming from a combination of two or more motion data the overall motion of a virtual creature in the virtual test environment, in particular with respect to a virtual representation of an observer and / or vehicle in the virtual test environment, in particular with respect to a self-object acting at least partially as a test subject on the test stand.

[0067] In a further advantageous embodiment, the method includes combining the first captured motion data and the second captured motion data in a randomized manner.

[0068] This allows, in particular, the driver assistance system to be optimized and / or trained for motions, in particular anomalous motions, that are randomly combined, in particular from the first captured motion data and the second captured motion data. Alternatively, the first captured motion data and the second captured motion data can be combined based on a combination of the first captured motion data and the second captured motion data, in particular the combination of the first captured motion data and the second captured motion data can be based on machine learning. In particular, machine learning allows combining the first motion data and the second motion data in a way that corresponds to the natural motion of a real creature. Advantageously, this allows combining, in particular, motion data captured at different locations using a motion capture system, in particular assigned to different real creatures, but in particular creatures belonging to the same species. In particular, the method can include that the captured motion data of the different real creatures can be adapted in such a way that the motion data corresponds to anatomical given conditions of the virtual creature, in particular such that the captured motion data can be represented in the virtual test environment as motion data of the virtual creature. This can be done in particular by a machine learning process.

[0069] In a further advantageous embodiment, the method may further comprise allowing a test engineer to initiate a repetition of the captured motion data, thereby enabling the test engineer to intervene in the virtual test environment and / or to individually modify, adapt and / or manipulate the scenario, which may be in particular temporal and / or spatial terms.

[0070] A test engineer in the sense of the present invention is preferably an engineer who can move and / or intervene in a virtual test environment by means of a virtual reality, an augmented reality or a mixed reality system, in particular the test engineer can place, remove and / or manipulate objects in the scenario, in particular in time and in space.

[0071] In a further advantageous embodiment of the method, the scenario recording may comprise scenario parameters selected from the following group depending on the type of driving assistance system being tested: vehicle speed, in particular initial speed, vehicle trajectory, lighting conditions, weather conditions, road surface conditions, the number and position of static and / or dynamic objects, in particular virtual creatures, and more in particular their relationship to the vehicle, the speed and direction of movement of dynamic objects, in particular motion data of virtual creatures, the state of signal equipment, in particular luminous signal equipment, traffic signs, lane elevation, width and / or drivability, lane extension, number of lanes, critical infrastructure such as, for example, road works obstructing visibility.

[0072] Features and advantages discussed above in relation to the first aspect of the invention apply to the other aspects of the invention as appropriate, and vice versa.

[0073] Further characteristics and advantages will become apparent from the following description with reference to the figures, in which: FIG. [Brief description of the drawings]

[0074] [Figure 1a] FIG. 2 is a block diagram of a first embodiment of a method for operating a test stand. [Diagram 2] FIG. 4 is a block diagram of a second embodiment of a method for operating a test stand. [Diagram 3] FIG. 1 illustrates an example of a scenario in a virtual testing environment. [Figure 4] FIG. 1 illustrates an embodiment of a system for operating a test stand. [Figure 5a] FIG. 1 illustrates an embodiment of a system for operating a test stand and various components of the test stand. [Figure 5b]FIG. 1 illustrates an embodiment of a system for operating a test stand and various components of the test stand. [Figure 5c] FIG. 1 illustrates an embodiment of a system for operating a test stand and various components of the test stand. [Figure 5d] FIG. 1 illustrates an embodiment of a system for operating a test stand and various components of the test stand. [Figure 5e] FIG. 1 illustrates an embodiment of a system for operating a test stand and various components of the test stand. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0075] FIG. 1 is a block diagram illustrating one embodiment of a method 100 for operating a test bench 1 for a vehicle.

[0076] Activity step 101 relates to creating a virtual test environment, activity step 102 to stimulating a real organism, activity step 103 to capturing motion data, and activity step 104 to recording the captured motion data.

[0077] Method 100 may be repeated according to the illustrated embodiment, particularly to generate new and / or modified test scenarios for testing driver assistance systems.

[0078] The method 100 comprises, inter alia, an operational step 102 of stimulating a real creature based on the created virtual environment, an operational step 103 of capturing motion data with a motion capture system, and an operational step 104 of recording, in particular storing, the captured motion data, which is combined with at least one stimulus from the virtual test environment for the real creature 2 in the motion capture system 12.

[0079] The working steps 105 of determining transition data, 106 of reproducing the transition data and 107 of iteratively considering the motion data, in particular with the transition data, are indicated by dashed lines in Fig. 1. In particular, as shown, the method can be repeated in parts or at least substantially in its entirety. In particular, different scenarios for testing and / or optimizing the driver assistance system can be generated in this way and the test bed can be operated accordingly. The working steps indicated by dashed lines are optional.

[0080] FIG. 2 is a block diagram of a further method 200 for operating a test stand.

[0081] In operational step 201 a virtual test environment is created. In operational step 202 motion data is captured, in particular using a motion capture system, and in operational step 203 a scenario is recorded. Operational steps 204, 205, 206 correspond to operational steps 105, 106, 107 of method 100 and are shown with dashed lines because they are optional, in which transition data are determined, the transition data are reproduced and the motion data are iteratively taken into account.

[0082] The method 200 according to Fig. 2 differs from the method 100 according to Fig. 1 mainly in that scenario data characterizing a scenario are recorded. Using the interface 22, in particular a user interface such as a motion capture system, motion data is captured and taken into account, together with the reactions of the driver assistance systems, in forming the virtual test environment.

[0083] Preferably, the method 200 may also be performed based on the method 100, whereby the motion data captured in the method 100 is transmitted.

[0084] FIG. 3 illustrates an exemplary virtual test environment that may be created by the methods 100, 200 for operating a test stand.

[0085] A virtual vehicle 3' controlled by a driving assistance system is driving in the right lane, next to which further vehicles 5b, 5c, 5d are parked, which make the virtual pedestrian 2' undetectable or difficult to detect for the sensors of the driving assistance system.

[0086] In addition to the virtual pedestrian 2' and the parked vehicles 5b, 5c, 5d, there is another vehicle 5a traveling in another lane around the vehicle 3' controlled by the driving assistance system (also called the own vehicle) toward the vehicle 3' controlled by the driving assistance system.

[0087] Behind this further vehicle 5a a motorcycle 4 is travelling. It cannot be explained from Fig. 3 whether this motorcycle is visible in the surroundings of the virtual vehicle 3' controlled by the driving assistance system. In the illustrated test environment the motorcycle 4 is about to overtake a further vehicle 5a in the other lane. At the same time in the illustrated scenario a virtual pedestrian 2' is crossing the road.

[0088] Depending on how the driving assistance system reacts or behaves in the test environment, i.e., what driving behavior the driving assistance system exhibits in the test environment of the virtual vehicle 3', dangerous or less dangerous scenarios can be obtained.

[0089] FIG. 4 shows an embodiment of a system 10 for operating a test bench 1 with a virtual test environment.

[0090] The system 10 preferably comprises simulation means 11 for creating a virtual test environment with at least one virtual creature 2' and at least one virtual vehicle 3'.

[0091] In order to enable the virtual creature 2' (a pedestrian in the illustrated example) to be controllable by the first user 2 as a traffic participant 2', the system 10 may further have a first user interface 13 and, preferably, a second user interface 12.

[0092] At least one first user interface 13 may be used to output the virtual environment of the virtual walker 2' to the first user 2. The user interface 13 may be an optical user interface, in particular a stimulation means such as a head-mounted device or a screen, and / or an audio interface such as a loudspeaker, possibly a device capable of influencing the respective user's sense of balance.

[0093] The second user interface 12 is preferably configured to detect inputs from each user 2, preferably a motion capture system 12 capable of detecting the pose and motion of the user 2 via various sensors and, for example, a treadmill.

[0094] Additionally, the system 10 preferably includes storage means 14 for recording the captured motion data.

[0095] Furthermore, the system 10 preferably comprises a data store 15 for providing scenario data characterizing the scenario in which the virtual pedestrian 2' is located.

[0096] The simulation means 11 are preferably configured to simulate a virtual test environment for the virtual vehicle 3' based on the scenario data. Furthermore, the simulation means 11 are preferably also configured to render said environment.

[0097] Finally, an interface 6 of the test bench 1 is configured to output the virtual test environment to a driving assistance system of the vehicle 3. If the driving assistance system comprises an optical camera K, such an interface 6 may be a screen, as shown in FIG.

[0098] The simulation means 11 calculates a response signal S' based on the detection signal S and the simulation environment, and the response signal S' is output to the camera K of the driving assistance system. In this way, the function of the driving assistance system can be tested. The response signal S' can also be output to the radar of the driving assistance system via a radar stimulator. Further environments can be simulated for the lidar, ultrasonic or infrared camera.

[0099] Depending on which components of the driver assistance system are to be tested, the simulated virtual test environment may output signals by emulation to the driver assistance system sensors K, as shown in Figure 4. Alternatively, signals may be generated that are directly input to the driver assistance system data processing unit or that are processed only by the driver assistance system software.

[0100] Preferably, the storage means 14 and the simulation means 11 are part of a data processing unit.

[0101] Figures 5a to 5e show an embodiment of the system 10 with a motion capture system 12, in particular shown in a working step 103, 202 of capturing motion data, in which the motion of a pedestrian 2 is captured by a sensor.

[0102] In FIG. 5a, as a real creature 2, a pedestrian is provided so that its motion, and in particular the motion of its anatomical parts, can be recorded.

[0103] The depicted human is on a treadmill, which may be used to simulate motion sequences, particularly as the human moves forward, and may also capture and / or record temporal sequences, particularly poses, of at least one part of the anatomy.

[0104] This temporal sequence of poses can be captured and / or recorded as motion data. This motion data can then be transmitted to a virtual creature 2', in particular an avatar, as shown in Fig. 5b. The virtual creature 2' performs the same, or at least substantially the same, motions as the real creature 2. In other words, the captured motion data of the real creature 2 is transmitted to a virtual creature 2', in particular an avatar, such that the avatar performs the same motions as the real creature 2.

[0105] In the operational step 101, 201 of creating a virtual test environment, a virtual creature 2' is embedded in the test environment such that the virtual creature 2', in particular an avatar, exhibits at least approximately the same temporal sequence of poses in the virtual test environment.

[0106] 5c shows an exemplary virtual test environment including a pedestrian crossing, two lanes with a bus in the opposite lane and further vehicles behind it, and a virtual pedestrian 2' at the start of the crossing and crossing the lane of the ego vehicle. Furthermore, in the illustrated virtual test environment, objects that are already known or recognized by the driving assistance system are framed.

[0107] The field of view of the virtual walker 2' is correlated with the field of view of the real human 2 in the motion capture system 12, and this field of view may be shown to the human 2 using, in particular, augmented reality glasses, virtual reality glasses or mixed reality glasses (not shown in FIG. 5c).

[0108] The real creature 2 is allowed to react to situations or scenarios in the virtual test environment, and this reaction is represented in the virtual test environment by captured motion data of the real creature 2 using the virtual creature 2'.

[0109] In Fig. 5d, the image of a virtual test environment for sensors, in particular camera sensors, of a test object 3 on a test stand 1 using a screen 6 is shown. In the test environment shown there, a virtual creature 2' can be recognized on the road.

[0110] As an example, as shown in Fig. 5e, the test environment can be presented to a test object 3, so that a driving assistance system associated with the test object 3 can capture the virtual creature 2' with its sensors. Fig. 5e shows a part of a real vehicle as the test object 3 on a test stand 1 and an image 6 of the virtual test environment from the point of view of a virtual vehicle 3' placed in front of the vehicle 3.

[0111] It should be noted that these embodiments are merely examples that are not intended to limit the scope of protection, the scope of application and the structure in any way, but rather, the above description provides a guide for a person skilled in the art to implement at least one embodiment, and various modifications can be made, particularly with regard to the function and arrangement of the described components, without departing from the scope of protection resulting from the claims and equivalent combinations of features. [Explanation of symbols]

[0112] 1 Test bench 2. Real Creatures 2' Virtual Creatures 3. Vehicle 3' Virtual vehicle 4 Motorcycles 5a, 5b, 5c, 5d More Vehicles 6. Interface 10. System 11 Simulation Methods 12 Motion capture system 13 Stimulation means 14 Memory means 15 Data storage devices 22 Interface 100, 200 ways 101, 201 Steps to create a virtual test environment 102 Steps to Stimulate Real Creatures 103, 202 Steps for capturing motion data 104 Steps for recording captured motion data 105, 204 Work steps for determining transition data 106, 205 Steps to reproduce transition data 107, 206 A work step in which motion data is repeatedly considered using transition data 203 Recording a scenario K Optical Camera S Detection signal S' Response signal

Claims

1. A method (100) for operating a test stand (1) for a vehicle using a simulation means (11) and a motion capture system (12), comprising the following work steps: a work step (101) of creating a virtual test environment with at least one virtual creature (2') and at least one virtual vehicle (3') using the simulation means (11), one of the virtual creatures (2') being a virtual representation of a real creature (2) and one of the virtual vehicles (3') being a virtual representation of a vehicle (3) equipped with a driving assistance system, and additionally at least a part of the vehicle (3) acting as a real test subject on the test stand (1), the driving assistance system being activated or stimulated based on the virtual test environment; an operational step (102) of stimulating the real creature (2) in the motion capture system (12) with stimuli based on the created virtual test environment; - capturing (103) motion data using said motion capture system (12), said motion data describing a temporal sequence of poses of at least one part of the anatomy of said real creature (2); an operational step (104) of recording the captured motion data; The method (100) comprises:

2. 10. The method of claim 1, wherein the captured motion data is associated with stimuli for the real creature in the motion capture system, and the stimuli associated with the captured motion data are also stored.

3. 3. The method (100) of claim 1 or 2, wherein the captured motion data and the reaction of the driver assistance system to the captured motion data are taken into account when creating the virtual test environment.

4. A method (200) for operating a test stand (1) for a vehicle, comprising the following work steps: a work step (201) of creating a virtual test environment with at least one virtual creature (2') and at least one virtual vehicle (3') using a simulation means (21), one of the virtual creatures (2') being a virtual representation of a real creature (2) and one of the virtual vehicles (3') being a virtual representation of a vehicle (3) equipped with a driving assistance system, at least a part of the vehicle (3) operating as a real test subject on the test stand (1), the driving assistance system being activated or stimulated based on the virtual test environment; an operational step (202) of capturing motion data, said motion data describing a temporal sequence of poses of at least one part of the anatomical structure of said real creature (2); an operation step (203) of recording scenarios generated by the reaction of the driver assistance system to the recorded motion data, wherein the recorded motion data and the reaction of the driver assistance system to the recorded motion data are taken into account when forming the virtual test environment; The method (200) comprises:

5. The method (200) of claim 4, wherein the motion data is captured using a motion capture system (22).

6. 5. The method (200) of claim 4, wherein the recorded scenario is combined with the captured motion data, and motion data associated with the generated scenario is also stored.

7. The method (100) of claim 3, wherein the captured motion data is considered iteratively in forming the virtual test environment.

8. A method (200) according to any one of claims 4 to 6, wherein the captured motion data is taken into account repeatedly when forming the virtual test environment.

9. A method (100, 200) according to any one of claims 1, 2, 4 to 6, wherein the test object (3) is operated as hardware-in-the-loop, in particular as vehicle-in-the-loop.

10. 7. The method (100, 200) of any one of claims 1, 2, 4 to 6, wherein the repeated consideration of the captured motion data includes modifying at least one part of the anatomical structure and / or modifying the temporal sequence of poses of at least one part of the anatomical structure.

11. The method (100, 200) of claim 10, wherein the modification of at least one portion of the anatomical structure is performed based on an empirical quantile of the portion of the anatomical structure.

12. A method (100) as described in claim 7, wherein the time sequence of the virtual test environment is faster or slower than the time sequence of the motion data when the motion data is repeatedly considered, or faster or slower than real time.

13. A method (200) as described in claim 8, wherein the temporal sequence of the virtual test environment is faster or slower than the temporal sequence of the motion data when the motion data is repeatedly considered, or faster or slower than real time.

14. Further work steps:

7. The method (100, 200) of any one of claims 1, 2, 4 to 6, comprising an operational step (107, 206) of iteratively considering second captured motion data different from the first captured motion data when forming the virtual test environment.

15. Further work steps: determining transition data from the first captured motion data to the second captured motion data, the transition data describing a temporal and / or spatial transition from the first captured motion data to the second captured motion data (105, 204); and 15. The method (100, 200) of claim 14, further comprising the step (106, 205) of temporally recreating transition data between the first captured motion data and the second captured motion data when creating the virtual test environment.

16. The method of claim 15, wherein the first captured motion data and the second captured motion data are combined in a randomized manner. Or, 15. The method (100, 200) of claim 14, wherein the first captured motion data and the second captured motion data are combined based on a combination of the first captured motion data and the second captured motion data.

17. The method (100, 200) of claim 16, wherein the combination of the first captured motion data and the second captured motion data is based on machine learning.

18. The method (100, 200) of claim 14, wherein a test engineer can initiate the repetition of the captured motion data.

19. The following work steps: a work step (201) of creating a virtual test environment comprising at least one virtual creature (2') and at least one virtual vehicle (3') using a simulation means (21), one of the virtual creatures (2') being a virtual representation of a real creature (2), one of the virtual vehicles (3') being a virtual representation of a vehicle (3) equipped with a driving assistance system, at least a part of the vehicle (3) operating as a real test subject on a test stand (1), the driving assistance system operating based on the virtual test environment; an operational step (202) of capturing motion data using a motion capture system (12), said motion data describing a temporal sequence of poses of at least one part of the anatomical structure of said real creature (2); an operational step (203) of recording scenarios generated by the reaction of the driver assistance system to the captured motion data, wherein the captured motion data and the reaction of the driver assistance system to the captured motion data are taken into account when creating the virtual test environment; The method (100) of claim 4, further comprising:

20. A computer program stored on a computer-readable non-transitory storage medium, the computer program including instructions that, when executed by one or more computers, cause the computers to perform a method (100, 200) according to any one of claims 1, 2, 4 to 6.

21. A simulation means (11) configured to create a virtual test environment with at least one virtual creature (2') and at least one virtual vehicle (3'), one of the virtual creatures (2') being a virtual representation of a real creature (2) and one of the virtual vehicles (3') being a virtual representation of a vehicle equipped with a driver assistance system, and additionally at least a part of the vehicle (3) acting as a real test subject on a test stand (1), the simulation means (11) being for operating or stimulating the driver assistance system based on the virtual test environment. a motion capture system (12) for capturing motion data, the motion data describing a temporal sequence of poses of at least one part of the anatomical structure of the real creature (2); - stimulation means (13) configured to stimulate the real creature (2) in the motion capture system (12) based on the created virtual test environment with stimuli; and A system (10) for operating a test bench (1) for a vehicle, comprising storage means (14) for recording said captured motion data.

22. a simulation means (11) configured to create a virtual test environment with at least one virtual creature (2') and at least one virtual vehicle (3'), one of the virtual creatures (2) being a virtual representation of a real creature and one of the virtual vehicles (3') being a virtual representation of a vehicle (3) equipped with a driver assistance system, the vehicle (3) additionally acting at least partially as a real test subject on a test stand, the simulation means (11) being for operating or stimulating the driver assistance system based on the virtual test environment; - means (12), or a motion capture system or interface, configured to capture motion data, said motion data describing a temporal sequence of poses of at least one part of the anatomy of said real creature (2); and a storage means (14) for recording a scenario generated by the reaction of the driving assistance system to the captured motion data, wherein the reaction of the driving assistance system to the captured motion data and the virtual creature (2') is taken into account when generating the virtual test environment; A system (10) for operating a test stand (1) for a vehicle, comprising:

Images