Virtual training method for neural network for actuating technical device

Abstract

To provide a method for training a neural network for actuating a technical device in a virtual training environment.SOLUTION: In a computing system 2, a method includes establishing a first data link between a neural network 8 and a first simulation 14a of a technical device and training the neural network by actuating the first simulation. Once a first training goal has been achieved, the first data link is broken, and a second data link is established between the neural network and a second simulation of the technical device to train the neural network by actuating the second simulation 14b. The second simulation is configured to be more realistic than the first simulation and requires more mathematical operations than the first simulation. The second simulation trains the neural network.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] In recent years, neural networks have brought about great progress in the development of autonomous systems that can independently perform complex movement patterns or monitoring tasks in a changing environment. However, a problem in this regard is the large amount of training data required to adequately train a neural network, especially when the neural network is intended for safety-critical tasks where a false recognition can have catastrophic consequences. Examples include self-driving cars intended for road traffic, smart surveillance cameras that recognize drowning people in lakes, or rescue robots that search for injured people from areas or buildings. Not only do such systems require a particularly large amount of different training data in order to optimally eliminate malfunctions caused by too little or too one-sided training, but the training data must also contain a large amount of data depicting catastrophic or critical scenarios that the neural network should be trained to recognize and process reliably. Such scenarios, by their nature, occur rarely, and therefore the corresponding data collection is also rare, which means that collecting sufficient training data represents a huge task.

[0002] One known solution to this problem from the prior art is to generate synthetic training data using computer simulation. In this case, scenarios for training the neural network are reconstructed in a virtual environment and training data are generated based on these reconstructed scenarios in a form understandable to the neural network. In this way, the virtual scenarios can be easily modified as desired. However, since synthetic data generally does not perfectly reproduce real-world data, there is a risk in principle that the neural network will become accustomed to the synthetic training data and will not be sufficiently trained for real-world use.

[0003] The scientific paper “Synthetic Data for Deep Learning” by Sergey I. Nikolenko (2019) contains a comprehensive overview of the relevant prior art. It contains various examples of neural networks successfully trained in virtual environments. However, these academic works only express that in principle it is possible to do so. This does not change the fact that the more realistic the synthetic training data, the more reliable the neural networks trained using those synthetic data will be. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to training a neural network for operating a technical device with synthetic training data. In this case, if the technical device is a device subject to strict safety requirements, for example a highly automated vehicle, synthetic training data with a high degree of realism must be used for training in order to ensure that the capabilities of the neural network learned in a virtual environment can be transferred to the real world. It is therefore necessary to provide a virtual simulation of a technical device that gives the neural network an almost perfect and outstanding illusion of operating the technical device in the real world. However, the realism of a simulation model is inversely proportional to its execution speed, since the more technical components of the device and physical phenomena and interactions are taken into account in the simulation, the more realistically the simulation model of the technical device behaves. The demand for the highest possible realism is therefore in conflict with the demand for the fastest possible execution speed of the simulation in order to complete the training within a reasonable time frame.

[0005] Against this background, the object of the present invention is to speed up the training of neural networks for operating technical devices in a virtual training environment.

[0006] DE 102018220865 A1 proposes a training method that starts with a purely virtual training environment and gradually replaces it with a real-world training environment in order to gradually accustom the neural network to the more complex real world in later training phases. This method makes it possible to use a relatively simple simulation, but later training phases rely on real-world training data, which has the associated drawbacks mentioned above. [Means for solving the problem]

[0007] The present invention is a method for training a neural network for operating a technical device and a virtual training environment for carrying out the method. According to the invention, a first data link is established between the neural network to be trained and a first simulation of the technical device. For this purpose, the first simulation is equipped with a first data interface, by means of which the neural network can read state data from the first simulation and transfer control data to the first simulation.

[0008] Preferably, the first data interface reproduces a real-world data interface of the technical device, and the neural network is intended to use said data interface to operate the real-world technical device once training of said data interface has been completed and verified. Status data should be interpreted as data that provides the neural network information about the current state of the technical device and that is required by the neural network to operate the technical device. Control data should be interpreted as data that is suitable for changing the current state of the technical device towards a target state of the technical device specified by the neural network. In case the technical device is a land-based vehicle, for example, examples of possible status data are the distance from a line marking the edge of the road, the speed of the vehicle, the distance from a vehicle in front, gear engagement, the roll angle of the vehicle, or a description of the surroundings of the vehicle, and examples of possible control data are the steering angle, the opening of the throttle valve, or the braking force.

[0009] In addition, a first training goal is set for operating the first simulation, the neural network is trained by the first simulation operated by the neural network, and the training progress of the neural network against the first training goal is periodically checked.

[0010] According to the invention, training of the neural network using the first simulation continues until the neural network achieves a first training goal. Once the first training goal is achieved, the first data link is disconnected and a second data link is established between the neural network and a virtual second simulation. The second simulation is configured to be more realistic than the first simulation and, due to the higher realism of the second simulation, requires more mathematical operations per simulation cycle than the first simulation. The second data link is established by a second data interface of the second simulation for reading state data from the second simulation and for transferring control data to the second simulation.

[0011] A simulation cycle refers to the complete execution of one simulation step, specifically one time step, of a simulation.

[0012] Thus, according to the invention, in an initial training phase, a relatively simple but alternatively fast simulation model of the technical device is used, and as soon as the neural network has sufficiently mastered the operation of the simple simulation model, as indicated by the achievement of the training objectives, the simple simulation model is replaced by a more complex and slower but alternatively more realistic simulation model, and training continues with the second simulation operated by the neural network.

[0013] The assumption underlying the present invention is that the simulation model does not need to be very realistic in the initial training phase, since the neural network is still working on learning the basics at this point. To exemplarily explain this, it is assumed that a neural network should be trained to steer an autonomous road vehicle based on the method according to the present invention. An untrained neural network would first need to understand what its task actually is (keeping the vehicle between the right edge marking and the center line) and how to master this task in basic terms (a negative steering angle makes the vehicle drive left, and a positive steering angle makes the vehicle drive right).

[0014] To learn these fundamentals, a very simple simulation of the vehicle is sufficient. As soon as the neural network has learned these fundamentals sufficiently, the first simulation is replaced by a second simulation of the vehicle, which takes into account more complex phenomena, such as roll angles, instability in curves, driving comfort, skidding situations or crosswinds. The concept of the invention is a special application of the so-called "transfer learning", where the pre-learned capabilities of the neural network are used as an advantage to learn another capability (in this case, to run a more realistic simulation). In training scenarios where very complex and realistic simulations must be used for training for safety reasons, the invention results in significant time savings.

[0015] Advantageously, a second training goal for running the second simulation is set, and the neural network is trained by running the second simulation until the second training goal is achieved. For this purpose, during the running of said second simulation, the training progress of the neural network is checked against the second training goal. The second training goal may be identical to the first training goal or may be different from the first training goal. Once the second training goal is achieved, the training can be considered as completed and a physical data link can be established between the neural network and the technical device in the real world in order to verify the training of the neural network in the real world. However, it goes without saying that it is also possible to continue the training in the virtual training environment before the verification in the real world by running a third simulation and any number of further simulations of the technical device, in which case the realism of each simulation of the technical device is in each case higher than the preceding simulation. Terminating a simulation run upon achievement of an associated training objective and transitioning to a subsequent associated simulation run is in each case performed in a manner similar to that described with respect to the first and second simulation examples.

[0016] The first and second data interfaces are preferably configured identically, so that the first data link can be replaced by the second data link instantly and without modifying the data exchange operated by the neural network.

[0017] Simulating the technical device advantageously includes generating simulated sensor data that contain information about the virtual environment of the technical device. Naturally, the nature of these sensor data depends on the technical device and the task that the neural network is trained to accomplish. By way of example, the sensor data may be an object list generated based on simulated moving or stationary objects in the virtual environment, lane lines, polygon-based reproductions of surfaces in the environment, or raw data from an image generating sensor, in particular from a camera, lidar, radar or echolocation. The virtual sensor data is transferred to the neural network, which is trained to evaluate the sensor data and to take it into account when operating the associated simulation and ultimately the technical device.

[0018] Needless to say, the simulated sensor data must be generated in a format understandable by the neural network, in particular in the same format in which real-world sensor data is also available in the technical device. In the prior art, solutions are available for the proper simulation of sensor data based on virtual environments, see for example the scientific paper “Development of Full Speed ​​Range ACC with SiVIC, a virtual platform for ADAS Prototyping, test and evaluation” Dominique Gruyer et al., IEEE Intelligent Vehicle Symposium (2013).

[0019] In the extension phase of the invention, modifications to the virtual environment can be made at the same time to increase the complexity of the simulation of the technical device. These modifications can be purely geometric modifications to ensure that the neural network learns to generalize and not simply memorize patterns of behavior in a particular virtual environment. However, these modifications can also be extensions to the virtual environment that have a fundamental effect on the behavior of the simulation of the technical device, making it more realistic and possibly more demanding for the neural network.

[0020] The technical device is preferably a robot. A robot should be interpreted as any device that is endowed with the ability to perform independent complex movement patterns to perform a given action by computer activation, in particular also by activation by a neural network. In particular, the technical device may be an at least partially automated aircraft, ship or aircraft, a robotic arm, a robot for positioning or attaching materials and / or objects, a robot for cleaning surfaces, a robot for inspecting spaces or surfaces, a robot for applying chemicals, such as cleaning agents, disinfectants, paints, varnishes or coatings, a robot for performing medical procedures, or any combination of the aforementioned robot categories.

[0021] The training goal may be configured differently depending on the type of technical equipment and the task that the neural network has to learn. For example, the training goal may be a predefined section to travel on a virtual training course or on a virtual test route, a predefined number of movement patterns to be performed without any collisions or within a predefined range of movement, a predefined period during which the neural network runs the first simulation properly without any undesired events, or a predefined threshold to be reached for a reward function on the basis of which the neural network is trained. The reward function is periodically used to train the neural network in order to evaluate during training whether the output of the neural network leads to an improvement over the previous training phase. The reward function is defined in such a way that the result of this reward function makes it possible to indicate how well the neural network has mastered its specific task. The configurations of neurons and synapses that have led to an improvement are kept as the basis for further training steps, while the configurations that have led to impairments are rejected.

[0022] There are different options for configuring the second simulation to be more realistic than the first simulation: for example, the second simulation can take into account more mechanical and / or electrical components of the technical device, can take into account more mechanical degrees of freedom, can simulate physical phenomena and / or laws more realistically or with greater accuracy, can take into account a greater number of physical forces and / or interactions, or can have a smaller simulation step size than the first simulation.

[0023] The drawings and the following description of the drawings explain the invention in more detail on the basis of specific examples. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a schematic diagram of a virtual training environment. [Diagram 2]13 is a flowchart of a training operation. [Diagram 3] FIG. 1 illustrates a virtual environment for simulation of highly automated vehicles. [Figure 4] FIG. 1 illustrates a sensor simulation for simulating lane recognition in a highly automated vehicle. [Diagram 5] FIG. 1 illustrates a first simulation of a highly automated vehicle. [Figure 6] FIG. 1 illustrates a second simulation of a highly automated vehicle. [Figure 7] FIG. 13 illustrates a third simulation of a highly automated vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The drawing in FIG. 1 shows a computer system 2 on which a virtual training environment 4 and a development environment 6 for a neural network 8 are set up. A neural network 8 intended for operating a technical device and to be trained for its tasks is embedded in the development environment 6. The development environment 6 is logically separated from the training environment 4 and comprises, in addition to the neural network 8, a first development routine 10 for implementing a reward function T and a second development routine 12 configured to monitor the function value of the reward function T and to modify the configuration of the neural network 8 based on this function value, i.e. to modify the values ​​stored in the neurons and synapses of the neural network 8 to optimize the function value of T in a predefined manner. The function value of T depends on a matrix x, the entries in which are supplied by the virtual training environment 4.

[0026] Development environments such as the illustrated development environment 6 are known in the art and are commercially available. Examples include PyTorch and Tensorflow.

[0027] As an example, the virtual training environment 4 includes three simulations of technical equipment, namely a first simulation 14a, a second simulation 14b and a third simulation 14c. The first simulation 14a includes a first data interface 16a for the neural network 8 to operate the first simulation 14a. While training on the first simulation 14a, the neural network 8 reads status data from the first simulation 14a and transfers control data to the first simulation 14a. Similarly, the second simulation 14b includes a second data interface 16b and the third simulation 14c includes a third data interface 16c. Both the second data interface 16b and the third data interface 16c are functionally identical to the first data interface 16a. As a result, the three simulations 14a, 14b, 14c can be instantly swapped with one another and the neural network 8 cannot directly identify which of the three simulations 14a, 14b, 14c it is running.

[0028] The degree of realism of these simulations of the technical devices increases step by step, i.e. the second simulation 14b is configured to be more realistic than the first simulation 14a and the third simulation 14c is configured to be more realistic than the second simulation 14b. As a result of the higher degree of realism of each of these simulations, the simulation cycle of the second simulation 14b requires more mathematical operations to be performed by the computer system 2 than the simulation cycle of the first simulation 14a, which makes the second simulation 14b slower than the first simulation 14a, and the simulation cycle of the third simulation 14c requires more mathematical operations to be performed by the computer system 2 than the simulation cycle of the second simulation 14b, which makes the third simulation 14c even slower than the second simulation 14b.

[0029] The virtual training environment 4 includes a first virtual multiplexer 20 for establishing a data link between one of the three simulations 14a, 14b, 14c and a programming interface 18 of the virtual training environment 4. The development environment 6 is connected to the training environment 4 by the programming interface 18, whereby, as a result of the positioning of the first virtual multiplexer 20, a first data link can be established between the neural network 8 and the first simulation 14a, or a second data link can be established between the neural network 8 and the second simulation 14b, or a third data link can be established between the neural network 8 and the third simulation 14c.

[0030] As well as the simulations 14a, 14b, 14c of the technical device, the virtual training environment 4 also includes a virtual environment 22 of the technical device, the second virtual multiplexer 21 being configured to establish a data link between this virtual environment 22 and one of the simulations 14a, 14b, 14c. The virtual environment 22 is a reproduction of a typical environment of the technical device, and one of the three simulations 14a, 14b, 14c is integrated into the virtual environment 22 based on the data link established by the second virtual multiplexer 21 so that the training environment 4 can take into account the interaction between the associated simulation 14a, 14b, 14c and the virtual environment 22, simulating a typical interaction of the technical device with a real-world environment. The first simulation 14a, the second simulation 14b and the third simulation 14c each also include at least one sensor simulation 30, which simulates real-world sensors of a technical device, generates simulated sensor data 34 and transmits this simulated sensor data to the neural network 8, from which the neural network 8 can retrieve information about the virtual environment 22.

[0031] The training function 24 is configured to operate the first virtual multiplexer 20 and the second virtual multiplexer 22 such that both multiplexers are connected at any one time to the same simulation 14a, 14b, 14c of the technical device, so that at any one time there is a continuous and indirect data link between the virtual environment 22 and the neural network 8 via the first simulation 14a, the second simulation 14b or the third simulation 14c. In addition, the training function 24 is configured to read a first training target for the first simulation 14a, a second training target for the second simulation 14b and a third training target for the third simulation 14c from the training target memory 26. An operator computer 28 containing configuration software is connected to the computer system 2, so that the first, second and third training targets can be set by the user and stored in the training target memory 26 by the operator interface of the configuration software.

[0032] The diagram of FIG. 2 shows the method steps performed by the training function 24 to train the neural network 8. At the start, the training function 24 connects the first simulation 14a to the neural network 8 by activating the first virtual multiplexer 20, thereby establishing a first data link. The training function 24 then connects the first simulation 14a to the virtual environment 22 by activating the second virtual multiplexer 21. Thus, the training function establishes a continuous data link between the neural network 8 and the virtual environment 22, mediated by the first simulation 14a, and the neural network 8 can run the first simulation 14a in the virtual environment 22. At this point, the training function 24 starts training the neural network 8 with the first simulation 14a being run by the neural network 8, and during training, it continuously checks the training progress of the neural network 8 against the first training goal and continues training until the first training goal is achieved.

[0033] Due to the fast execution speed of the first simulation 14a, the first training objective is achieved in a relatively short time. Advantageously, the configuration of the first simulation 14a and the first training objective are selected such that, once the first training objective is achieved, the neural network 8 learns the fundamentals of operating a technical device.

[0034] Once the first training objective is achieved, the training function 24 connects the neural network 8 to the second simulation 14b by activating the first virtual multiplexer 20. In this way, the training function 24 cuts off the first data link and replaces it by a second data link between the neural network 8 and the second simulation 14b. The training function 24 then connects the second simulation 14b to the virtual environment by activating the second virtual multiplexer 21. The second simulation can now be incorporated in the virtual environment 22 in the same way and run by the neural network 8 in the same way that the first simulation 14a was run before. Similarly, the training function 24 starts and monitors the training and continues the training until the second training objective is achieved. Also similarly, a third training phase is performed, including the third simulation 14c, until the third training objective is achieved. Once the third training objective is achieved, the training function 24 ends the training.

[0035] The final third simulation 14c is preferably a highly complex and realistic simulation of a technical device, and the third training objectives and the configuration of the third simulation are preferably selected such that, once the final third training objective is achieved, the neural network 8 learns how to operate the technical device safely even under rare exceptional circumstances.

[0036] Once training has been completed in the virtual training environment 4, a data link can be established between the neural network and the technical device in order to operate the technical device in a real-world environment via the neural network 8, either for the purpose of validating the training carried out in the virtual training environment 4 or for the purpose of continuing the training in a real-world environment.

[0037] The diagram in Fig. 3 shows a virtual training environment for training a neural network for operating a highly automated road vehicle. The virtual environment 22 reproduces a typical environment of a highly automated road vehicle and for this purpose includes a number of stationary and moving 3D objects O1,...,O8 such as vehicles, vegetation, traffic signs and buildings, and a virtual test route 32 reproducing a road. In the illustrated example, the technical equipment is a road vehicle and a first simulation 14a of the road vehicle is integrated into the virtual environment 22 as a virtual road user, whereby the first simulation 14a moves in the virtual environment 22 in the same way as a real-world road vehicle in a real-world road traffic scenario.

[0038] The first simulation 14a (and also the second simulation 14b and the third simulation 14c) comprises a sensor simulation 30, which is assigned a sensor position in the virtual environment 22 (in the illustrated case in front of the simulated road vehicle 14a). By way of example, the sensor simulation 30 simulates a smart camera for recognizing lanes. In the virtual environment 22, the sensor simulation 30 is assigned a field of view FV. Lanes are visible to the sensor simulation 30 as long as they are within the field of view FV and are not hidden from the viewpoint of the sensor simulation 30 by 3D objects O1,...,O8 or the topography of the virtual environment 22.

[0039] The neural network 8 should be trained to steer the road vehicle 14a laterally in the virtual environment by manipulating the steering angle. Other aspects of operating the road vehicle 14a, such as acceleration, braking, and distance control, are performed by routines of the first simulation 14a or the second simulation 14b or the third simulation 14c and are not influenced by the neural network 8.

[0040] The diagram of Fig. 4 shows by way of example the functionality of the sensor simulation 30. As input, the sensor simulation 30 receives a description of the virtual environment 22 from the virtual training environment 4, for example in the form of an object list, generates from this description simulated sensor data 34 and transmits this simulated sensor data 34 to the neural network 8 by means of the programming interface 18. In the example shown, the sensor data 34 consists of rows of spots, each row of spots representing a lane segment visible to the smart camera, each spot being represented by three spatial coordinates. The columns of spots specify a range of movement for a simulated road vehicle 14a in the virtual environment 22. A first training goal of the neural network 8 for the first simulation 14a is to operate the first simulation 14a such that the road vehicle simulated by the first simulation 14a travels along a given section, for example a 5 kilometre section, on the virtual test route 32 without colliding, while on this section, with any spot from one of the rows of spots in the simulated sensor data 34. For this purpose, the training function 24 is configured to monitor the route travelled by the road vehicle simulated by the first simulation 14a on the virtual test route 32 in order to recognise collisions between the road vehicle and road boundaries.

[0041] The rows of spots in the sensor data 34 are an example of so-called ground truth sensor data. The rows of spots in the sensor data 34 are particularly easy to simulate sensor data that become available after a smart evaluation of raw data from an image generating sensor and based on the virtual environment 22. In principle, however, the simulated sensor data 34 may be structured in any form depending on the type of sensor data 34 that the neural network 8 is to be trained to process, for example as an object list or as raw data from a camera, a thermal imaging camera, a radar, a lidar or an echolocation. The simulated sensor data 34 may originate from a real-world camera directed at a screen on which an image of the virtual environment 22 is rendered in real time from the viewpoint of the sensor simulation 30. Those skilled in the art can find an overview of the different options for sensor simulation and the breakpoints to be implemented for them in the scientific paper “Full spectrum camera simulation for reliable virtual development and validation of ADAS and automated driving applications” Rene Molenaar et al., 2015 IEEE Intelligent Vehicles Symposium (IV), using the example of camera simulation.

[0042] The diagram of FIG. 5 shows a possible configuration of a first simulation 14a of a road vehicle. As an input from the neural network 8, the first simulation receives the steering angle θ and assigns the change in direction Δφ of the road vehicle on the two-dimensional plane of the virtual test route 32 to the steering angle θ based on a simple linear function by multiplying the constant steering sensitivity s and the simulation step size Δt. Based on the change in direction Δφ, the first simulation 14b calculates the new heading in the following simulation step. The first simulation 14a is thus a highly simplified model of a road vehicle, in which the road vehicle is a solid body without any rotational degrees of freedom and moves at any time parallel to the ground on a complete circular arc, the curvature of which is linearly predefined by the steering angle θ. Based on the first simulation 14a, the neural network 8 learns the fundamentals of steering a road vehicle.

[0043] Once the first training goal is achieved, the first simulation 14a is replaced by a second simulation 14b, as described above on the basis of Fig. 1, which is shown in the diagram of Fig. 6. Exactly like the first simulation 14a, the second simulation receives the steering angle θ as input from the neural network 8 and ascertains on the basis of this steering angle θ a directional change Δφ that is visible to the neural network 8 thanks to the sensor data 34 transmitted to the neural network 8, the composition of the sensor data 34 remaining unchanged compared to the first simulation 14a. The second data interface 16b, which the neural network 8 uses to read out state data (in this case the sensor data 34) from the second simulation 14b and to transfer control data (in this case the steering angle θ) to the second simulation 14b, is therefore identical to the first data interface 16a of the first simulation. The neural network is therefore not able to directly identify that the first simulation 14a has been replaced by the second simulation 14b. However, the second simulation 14b is still very simplified compared to reality, but is configured to be more realistic than the first simulation 14b. The second simulation 14b is a so-called kinematic bicycle model, which is a basic model known to those skilled in the art for simple simulation of four-wheeled land-based vehicles. The kinematic bicycle model combines two wheels each located at one level into one wheel such that the two wheels are arranged on a common longitudinal axis, whereby the front wheel is assigned a tilt angle depending on the steering angle θ, from which the directional change Δφ is calculated in conjunction with the fixed rear wheel. The kinematic bicycle model takes into account more degrees of freedom than the first simulation 14a, i.e. the ability of the front wheel to rotate, and more physical phenomena, such as the extent of the vehicle in space or the presence of the center of mass of the vehicle.Thus, more mathematical operations have to be processed in the case of the simulation cycle of the second simulation 14b than in the case of the simulation cycle of the first simulation 14a. Overall, therefore, the training of the neural network 8 is slower compared to the training on the first simulation 14a. However, the second simulation 14b still ignores many phenomena, such as the loss of tire adhesion and the vehicle degrees of freedom. The second simulation 14b is therefore fast enough to achieve rapid training success also in the second training phase.

[0044] The second simulation 14b is assigned a second training goal, which may be identical to the first training goal. Once the training goal is achieved, the second simulation 14b is replaced by the third simulation 14c, as described above, which is shown in the drawing of FIG. 7. The third simulation 14c is a complex simulation that reproduces the technical and mechanical behavior of a real-world road vehicle in high detail. The third simulation 14c includes a complete simulation of the vehicle's drive train and all suspensions, a simulation of the engine with realistic acceleration and braking behavior, a realistic tire model, the yaw and roll motion of the vehicle when the body is placed on the road, loss of contact of some or all tires with the road, and aerodynamic forces acting on the vehicle. The third data interface 16c of the third simulation 14c is also identical to the second data interface 16b and the first data interface 16a.

[0045] Due to the high level of detail in the third simulation 14c, training with the third simulation 14c takes significantly longer than training with the second simulation 14b and the first simulation 14a, but the performance of the third simulation 14c closely resembles the actual performance of a road vehicle. As a result, once the third training objective is achieved, the neural network has reached a high level of training and can be empirically operated on real-world road vehicles on real-world test routes.

Claims

1. A computer-implemented method for training a neural network for operating a technical device, said method comprising the following method steps: - establishing a first data link between said neural network and a virtual first simulation of said technical device by means of a first data interface of said first simulation for reading status data from said first simulation and for transferring control data to said first simulation; setting a first training objective for operating the first simulation; training the neural network with the first simulation run by the neural network and checking the training progress of the neural network against the first training objective; - cutting off the first data link once the first training objective has been achieved and then establishing a second data link between the neural network and a virtual second simulation of the technical equipment by means of a second data interface of the second simulation for reading status data from the second simulation and for transferring control data to the second simulation, the second simulation being configured to be more realistic than the first simulation and requiring more mathematical operations per simulation cycle than the first simulation due to the greater realism of the second simulation; training the neural network with the second simulation run by the neural network; The method includes:

2. The first data interface and the second data interface are configured identically. The method of claim 1.

3. The method comprises the following method steps: - generating simulated sensor data comprising information about a virtual environment of said technical device; transferring the simulated sensor data to the neural network; training the neural network to evaluate the sensor data and to take the sensor data into account when running the first simulation and / or the second simulation; Including, The method of claim 1.

4. Said technical devices are robots, in particular those of the following categories: vehicle, Robot arm, a robot for positioning or attaching materials and / or objects; robots for cleaning surfaces, Robots for inspecting spaces or surfaces, Robots for applying chemicals, e.g. cleaning agents, disinfectants, paints, varnishes or coatings, Robots for performing medical procedures; A robot that belongs to at least one of The method of claim 1.

5. The first training objectives are classified into the following categories of training objectives: A section travelling on a virtual training course or a virtual test route; A plurality of movement patterns performed without any collision or within a predetermined range of motion; a period during which the neural network properly operates the first simulation without any undesired events occurring; The threshold to be reached for the reward function, belongs to at least one of The method of claim 4.

6. The second simulation comprises: taking into account more mechanical and / or electrical components of the technical device than in the first simulation; and / or taking into account more mechanical degrees of freedom than the first simulation; and / or simulating physical phenomena and / or laws more realistically and / or accurately than said first simulation; and / or taking into account a larger number of physical forces and / or interactions than in the first simulation; and / or having a smaller simulation step size than the first simulation; The method of claim 1.

7. The method comprises the following method steps: setting a second training objective for operating the second simulation; checking the training progress of the neural network against the second training objectives while the second simulation is running; - after the second training objective has been achieved, establishing a physical data link between the neural network and the technical device; operating said technical device by means of said neural network; Including, The method of claim 1.

8. A virtual training environment for training a neural network for operating a technical device, said virtual training environment comprising: a first simulation of said technical device, said first simulation having a first data interface for reading status data from said first simulation and for transferring control data to said first simulation; a programming interface for establishing a first data link between the neural network and the first simulation; a training function configured to set training goals for the first simulation to be run by the neural network and to check the training progress of the neural network against the training goals; In a virtual training environment, including the virtual training environment includes a second simulation of the technical device; the second simulation is configured to be more realistic than the first simulation, and due to the greater realism of the second simulation, requires more mathematical operations per simulation cycle than the first simulation; the second simulation includes a second data interface for reading state data from the second simulation and for transferring control data to the second simulation; The training function includes: disconnecting the first data link when the training objective is achieved; replacing the first data link with a second data link between the neural network and the second simulation; training said neural network (8) by said second simulation (14b) run by said neural network (8); The present invention is characterized in that it is configured as follows: Virtual training environment.

9. The first data interface and the second data interface are configured identically.

9. The virtual training environment of claim 8.

10. said virtual training environment comprising a virtual environment of said technical device and a sensor simulation; The virtual training environment includes: generating simulated sensor data by said sensor simulation, said simulated sensor data including information about said virtual environment; transmitting the simulated sensor data to the neural network via the programming interface; It is configured as follows:

9. The virtual training environment of claim 8.

11. Said technical devices are robots, in particular those of the following categories: vehicle, Robot arm, a robot for positioning or attaching materials and / or objects; robots for cleaning surfaces, Robots for inspecting spaces or surfaces, Robots for applying chemicals, e.g. cleaning agents, disinfectants, paints, varnishes or coatings, Robots for performing medical procedures; A robot that belongs to at least one of 9. The virtual training environment of claim 8.

12. The training objectives are classified into the following categories of training objectives: A section travelling on a virtual training course or a virtual test route; A plurality of movement patterns performed without any collision or within a predetermined range of motion; a period during which the neural network properly operates the first simulation without any undesired events occurring; The threshold to be reached for the reward function, belongs to at least one of 12. The virtual training environment of claim 11.

13. The second simulation comprises: taking into account more mechanical and / or electrical components of the technical device than in the first simulation; and / or taking into account more mechanical degrees of freedom than the first simulation; and / or simulating physical phenomena and / or laws more realistically and / or accurately than said first simulation; and / or taking into account a larger number of physical forces and / or interactions than in the first simulation; and / or 10. The virtual training environment of claim 8, having a smaller simulation step size than the first simulation.

Images