Method for training a teleoperator, computer program and teleoperation system

Abstract

One aspect of the invention relates to a method for training a teleoperator (11) to carry out a teleoperated driving maneuver of a vehicle (12), comprising the following steps: - identifying a teleoperator (11) on a teleoperation system (1); - determining the degree of competence of the teleoperator (11) with regard to the suitability to carry out a teleoperated driving maneuver with a teleoperation system (1); - depending on the specific level of competence, training the teleoperator (11) on the teleoperation system (1).

Description

[0001] Aspects of the invention relate to methods for training a teleoperator, a computer program product and teleoperation systems.

[0002] Methods are known in which a vehicle, in particular a motor vehicle, is controlled by a human remote controller, in particular a teleoperator, positioned externally to the vehicle. The teleoperator is typically located in a fixed remote control center, and thus in a building in which one or more display units, in particular screens, are also located. On these screens, the remote controller can identify the vehicle to be remotely controlled and / or the surroundings of this remotely controlled vehicle, or this is displayed on the screens.

[0003] Such procedures differ significantly from those in which a driver is positioned outside of his or her own vehicle in close proximity and must be there in order to, for example, perform driving maneuvers with the vehicle he or she has left using a portable remote control held in one hand. For example, parking maneuvers are performed here. However, such procedures require, and are only permitted in this way, that the driver is positioned in close proximity to the vehicle with such a portable remote control and must also be able to see the vehicle in order to control such driving maneuvers.

[0004] This is not the case with significantly different procedures, in which the above-mentioned scenario is carried out with a central remote controller, i.e., a teleoperator, located remotely from the vehicle. The remote controller in the remote control center is positioned in such a way that they cannot directly see the vehicle or the vehicle's surroundings. Therefore, with such a procedure, it is not necessary, and indeed may not even be possible, for the remote controller to be positioned within a specified close range, for example, a maximum of 5 meters, of the vehicle to be remotely controlled.

[0005] Therefore, with such procedures it is more difficult for a remote control operator in a remote control center to be able to maneuver the vehicle safely during corresponding movements.

[0006] The object of the present invention is to provide a method for remotely performing a driving maneuver with a human remote controller or teleoperator, who cannot directly see the vehicle being remotely controlled. Furthermore, it is an object to create an electronic remote control system or a teleoperation system.

[0007] This object is achieved by methods, a computer program product and electronic teleoperation systems according to the above claims.

[0008] One aspect of the invention relates to a method for training a teleoperator to carry out a teleoperated driving maneuver of a vehicle, in particular comprising the following steps: - Identifying a teleoperator on a teleoperation system; - Determining, in particular with the teleoperation system, the level of competence of the teleoperator with regard to the suitability to carry out a teleoperated driving manoeuvre using a teleoperation system; - depending on the specific level of competence, carrying out training of the teleoperator on the teleoperation system, in particular using an electronic system, such as a training system, in particular the teleoperation system.

[0009] This method now makes it possible to train a human teleoperator on a real teleoperation system. Thus, the real teleoperation system itself is advantageously used for this training, which is also intended for a subsequent, actual teleoperation process, i.e., a remote-controlled maneuver by a teleoperator. This allows an existing real teleoperation system to be used multifunctionally, namely for training purposes and, in principle, for performing a real remote-controlled driving maneuver.

[0010] A particularly advantageous feature of the method is that the system organizes the training of the teleoperator independently and automatically. By preferably identifying the teleoperator on the real teleoperation system, which is also used to carry out a real teleoperation maneuver or a teleoperated driving maneuver as intended, the system also always knows exactly which level of competence this teleoperator currently has. A particularly advantageous feature of the method is that it does not provide a blanket training session or a blanket specific training session, but rather that the system then also checks and recognizes the current level of competence of the teleoperator and only then decides whether training is necessary depending on this. In particular for this purpose, the teleoperation system preferably has an electronic training system.In particular, this is now a component of the teleoperation system. The teleoperation system itself is now individually configured and can offer and conduct the corresponding training itself. A particularly advantageous feature in this context is the possibility of this teleoperation system being able to automate the training at least partially, or even completely.

[0011] This offers significant advantages over conventional approaches. In conventional teleoperation systems, a teleoperator is trained in the teleoperation system by another human teleoperator. On the one hand, this requires additional personnel, and on the other hand, the time of the training teleoperator is used for this training, meaning that this training teleoperator cannot be used for other purposes, such as performing teleoperated driving maneuvers themselves. On the other hand, with a human teleoperator providing training, the training depends on their commitment and their own competence. This allows for a wide variety of training outcomes for the teleoperator being trained.

[0012] Furthermore, it is also known that a teleoperator being trained receives a printed user manual, which they then have to familiarize themselves with and train themselves. This, too, has significant disadvantages, as it is impossible to accurately verify the degree of commitment the teleoperator being trained has to this user manual and the knowledge they have acquired through it.

[0013] Therefore, the proposed method is advantageous in many respects, as this system-based training process provides the information to be trained very precisely, and the teleoperators being trained achieve the same training outcome, which is also verifiable. Furthermore, this integration of the training into an existing real-world teleoperation system can expand the functionality of such a system. Furthermore, the need for training personnel can be reduced.

[0014] Identification also particularly concerns authentication. Identification is possible by logging into the system with user data. Additionally, biometric data of the teleoperator can also be recognized and considered for identification purposes.

[0015] Preferably, the procedure involves carrying out this check right at the beginning, when a teleoperator wishes to use a real teleoperation system, to determine the level of competence he or she has and whether, and in particular to what extent, training is required.

[0016] Such a method is particularly advantageous in enabling a high level of security when operating a teleoperation system. This is because it can also be provided that, depending on the current level of competence of a teleoperator, the system only releases specific teleoperation maneuvers for this teleoperator that correspond to their level of competence. On the other hand, it is also possible that, again particularly depending on the current level of competence, a decision is made as to whether training is to be provided immediately or can be carried out at a later date. This can also depend on which teleoperation maneuvers the teleoperator should or must subsequently perform on this teleoperation system.

[0017] In one embodiment, the training is specified on the system side by a training module. This can, for example, be a pure software module that is implemented in the teleoperation system. However, the training module can also have hardware components. In one embodiment, the training process is carried out automatically on the teleoperation system by this at least one training module. This enables highly needs-based training for a teleoperator. Individual training configurations can therefore also be generated automatically. This enables a particularly high level of training. In particular, training can then be very needs-based on the teleoperator being trained, in particular starting from their current level of competence and moving towards a desired and achievable level of competence.

[0018] In one embodiment, upon initial identification of a teleoperator on a real teleoperation system, initial training for the teleoperator on this teleoperation system is conducted. This prevents a teleoperator from performing a real teleoperated driving maneuver without a level of competence, or possibly with a fundamentally insufficient level of competence, that allows operation of a teleoperation system. This embodiment also prevents a fundamentally untrained teleoperator from performing a teleoperation maneuver with the teleoperation system. This also ensures a high level of safety. Furthermore, a teleoperator can also trust that they will not be left alone with regard to operating the teleoperation system and can automatically trust that they will receive early and sufficient automated training.This means that a teleoperator can achieve a calmer and more secure approach when working on the teleoperation system, even in view of their own nervousness, and this is also known to the teleoperator themselves.

[0019] In one embodiment, at least one of the following topics is taught during this initial training: - general explanation of the functioning of a teleoperation system and its regulations; - general explanation of the functioning of the teleoperation system to which the teleoperator is currently identified; - Explanation of virtual representations, in particular operating elements and / or functional representations and / or symbols, on at least one screen of the teleoperation system, which are displayed as part of a teleoperated driving manoeuvre and describe the functions and / or states of the vehicle to be manoeuvred and / or of the teleoperation system; - Performing actions until the teleoperation system is in an active state to perform a teleoperated driving maneuver.

[0020] This basic information makes it easier for a teleoperator to access the teleoperation system. This helps to overcome any inhibitions about using the teleoperation system. Furthermore, this information and training in these topics enable a better and faster understanding of the teleoperation system. The general introduction to teleoperated maneuvering is therefore specifically designed for a teleoperator, so that a logical understanding of the system and its properties also forms a solid foundation for further training and practice in other aspects.

[0021] In this context, it is particularly advantageous that displays that are or can be displayed on at least one screen of the real teleoperation system are presented and explained during this training. This can be done acoustically and / or visually. This ensures particularly thorough intuitive comprehensibility for the teleoperator being trained. This also enables highly strategic and well-structured training, which efficiently leads to the success of the training. The information being taught can thus be perceived and understood by the teleoperator in a more structured manner, resulting in a particularly high learning effect.

[0022] Extreme situations, such as emergency situations, can be presented in a variety of ways, which is particularly advantageous in this context. This also allows highly customized training content on these topics to be presented, ensuring that it is both easily understood and comprehensively understood by the teleoperator being trained.

[0023] It is also advantageous to learn the basic starting and commissioning of the teleoperation system. To avoid possible misuse, one embodiment provides that several specific actions, particularly in a specific sequence, must be performed by a teleoperator before they can perform a teleoperated driving maneuver with the teleoperation system. Starting and commissioning the teleoperation system alone requires training to avoid incorrect inputs and to ensure that the full functionality of the teleoperation system is available. To alleviate the teleoperator's nervousness about possibly failing to commission the teleoperation system, training on this aspect is not a trivial matter but is very helpful in many ways.

[0024] In a further embodiment, at least one of the following additional aspects is trained during the initial training: - Latency times between optical and / or acoustic information between the real vehicle environment and the display on the screen of the teleoperation system; - Latency times between an operation of a teleoperation instrument of the teleoperation system and the active implementation of the vehicle function linked to this operation.

[0025] These aspects are extremely beneficial in helping the teleoperator understand these circumstances. Since during a teleoperated driving maneuver the teleoperator is not sitting in the vehicle itself, which they are supposed to maneuver, but rather outside and at a great distance from it, and they do not have direct visual contact with the vehicle at all, but only see it on at least one screen, such latency times can occur due to system factors. This gives the human teleoperator a different feeling when maneuvering the vehicle than if they were actually sitting in the vehicle and maneuvering the vehicle. Such constellations in particular lead to great tension and nervousness in a teleoperator and / or result in the teleoperator not thinking about these aspects at all, thus causing safety-critical driving maneuvers.By training these aspects, especially those that are tailored to the needs, advantages can be achieved both in terms of more relaxed operation of the teleoperation system by a trained teleoperator and in terms of the safety of a teleoperated driving maneuver.

[0026] Here, too, it is possible for the system, in particular the training system of the teleoperation system, to compile and deliver a predefined training course or one individually configured for the teleoperator currently being trained. This allows a wide variety of scenarios to be generated through this training with regard to the recognition and learning of such latency periods. It is particularly advantageous in this context that, depending on the level of competence of the teleoperator being trained, the training of such aspects can also be compiled automatically by the system, thus enabling very needs-based training for the teleoperator. In this context, "needs-based" also means preferably adapted to the level of competence of the teleoperator being trained and / or adapted to their character.This allows highly individual and intelligent training scenarios, particularly regarding specific topics, to be created and offered automatically, which not only develop the teleoperator being trained in terms of expertise but also in terms of character, for example, so that they can assess specific situations more correctly and / or handle them more calmly.

[0027] In one embodiment, at least during the initial training, training videos are displayed on at least one screen of the teleoperation system. This allows, for example, specific reference behaviors and / or reference maneuvers to be presented. This can be done, for example, using real, recorded teleoperated driving maneuvers. However, it is also possible for such scenarios to be simulated or animated and displayed on the screen.

[0028] Furthermore, at least during the initial training, it is also possible for interaction to take place between the teleoperator being trained and the system, in particular the training system of the teleoperation system. In particular, an action-reaction scenario can be carried out in this context. This allows even more customized training scenarios to be created and even more needs-based training for a teleoperator to be created. In this context, the system can then also recognize which training content has already been recognized and at which level during the training, and where, if necessary, additional time and / or more comprehensive training content must be offered to the teleoperator being trained.

[0029] In the action-reaction scenario, both acoustic and optical interactions can be offered between the system and the teleoperator being trained.

[0030] It is therefore also possible for the system to automatically communicate the quality of the teleoperator's training behavior during the action-reaction scenario visually and / or acoustically. This allows the system to provide the teleoperator with almost immediate feedback, improving the intensity and quality of the training, as well as the teleoperator's understanding.

[0031] For example, it is also possible, not only in this context, to display a superimposed representation of a reference maneuver with a training maneuver currently being performed by the teleoperator being trained on at least one screen. This can be done statically or dynamically.

[0032] It is also possible, for example, for recorded teleoperated driving maneuvers to be replayed and displayed on the screen during training. For example, a complete observation of a previously performed and recorded teleoperated driving maneuver can then be observed. This can then also be displayed from different perspectives during training. This allows the perspective of the teleoperator at their teleoperation station to be shown. In addition to or instead of this, the perspective of someone sitting in the vehicle can also be displayed.

[0033] As already explained above, various control elements, for example a complete control panel, can be displayed on a screen of the teleoperation system during training. This can, on the one hand, train the understanding of such a user interface. This can, on the one hand, increase the understanding of the numerous elements, if any, and also lead to an understanding of the associated functions. During training, it is possible for statically displayed written explanations to be shown on the screen that relate to the respective control element or control panel. These static explanations can, for example, also be shown as an overlay on the screen. It is also possible for these written representations to be switched on and off. This can, for example, occur depending on certain time intervals and / or an expressed understanding of the teleoperator being trained.

[0034] In one embodiment, at least one training drive can be conducted during the initial training, thus representing a teleoperated training maneuver. During this training drive, the teleoperator being trained performs a teleoperated training driving maneuver with a real training vehicle equipped with the teleoperation system. It is also possible for this training driving maneuver to be animated or simulated and displayed accordingly on at least one screen.

[0035] This training maneuver can preferably be recorded and / or evaluated by the system. It is possible for the system to provide quality information to the teleoperator in parallel with the training maneuver or afterward. This can be done acoustically and / or visually. It is also possible for the system to automatically transmit specific explanations to the teleoperator in this context, in order to explain to the teleoperator any good or bad actions during the training maneuver.

[0036] In one embodiment, at least during initial training, additional explanations, particularly those associated with the visual representation, are displayed on at least one screen of the teleoperation system. It is also possible for an avatar to be displayed on the screen, providing additional visual and / or acoustic explanations.

[0037] During initial training, at least in one embodiment, a reference maneuver is displayed and explained as a teleoperated driving maneuver on the screen of the teleoperation system. Regarding the explanation, it should also be noted that acoustic and / or visual information is output by the system regarding specific actions for operating a teleoperation instrument.

[0038] In one embodiment, the training is automatically selected and / or automatically configured by the system depending on the type of teleoperation system on which the training is conducted, and / or depending on the teleoperation system on which the teleoperator to be trained is to perform teleoperated driving maneuvers after the training, and / or depending on the level of competence of the teleoperator to be trained with regard to their current knowledge and / or abilities to perform teleoperated driving maneuvers, and / or depending on the environment in which teleoperated driving maneuvers are to be performed by the teleoperator to be trained in the future, and / or depending on the type of vehicle that the teleoperator to be trained is to maneuver teleoperated in the future. These possible parameters, in order to be able to provide not only a general, predetermined training but also training tailored to the situation, are not to be understood as exhaustive.However, these can be used to create a wide variety of training courses, ensuring a particularly high level of quality and enabling particularly efficient and understandable training for individual teleoperators. This allows for highly tailored training for each individual teleoperator.

[0039] In general, a teleoperated driving maneuver can be, for example, a parking maneuver of a vehicle. For example, this could be parking in or out of a parking space. A vehicle can be a passenger car, a truck, or a bus. It is also possible for the vehicle to be a combination. In this case, it has a towing vehicle and at least one trailer.

[0040] An environment or a teleoperation environment in which the teleoperated driving maneuver is to take place can, for example, be a parking garage or other parking space. However, an environment can also be designed in a variety of other ways, such as a car wash. It is also possible for an environment to be a service workshop, for example. Here, too, the examples listed are not exhaustive. Nevertheless, they show diverse and specific scenarios in which teleoperated driving maneuvers are possible and advantageous. Since they pose different difficulties with regard to the requirements posed by the environment and / or the vehicle, it is very advantageous to be able to provide appropriately trained teleoperators.Especially for such constellations, a wide variety of training scenarios can be created, which then subsequently allow for the safe and successful execution of such diverse and multifaceted teleoperated driving maneuvers. Furthermore, the various described environments and / or the various vehicles can lead to different permutations and configurations, which in turn result in individual aspects of the respective driving maneuvers. For example, these could be individual bottlenecks and / or specific turning circles and radii of the vehicles and / or a variety of single- or multi-train driving maneuvers. In this context, it is also important to be able to offer appropriate training courses that cover precisely these aspects.This also means that any individual actuation of a steering device of the vehicle by the teleoperator and / or the actuation of an element to accelerate or decelerate the vehicle must be known.

[0041] In one embodiment, it is provided that the teleoperator to be trained takes an exam during the training. This exam can preferably be generated online and can be fully specified by the system. This means that both the timing and, if applicable, the scope of the exam are specified by the system. Such a specification can be general, but can also be individually configured. In one embodiment, the teleoperator to be trained is automatically grouped into one of at least two different classes of teleoperator, at least taking into account the result of the exam. This provides a further advantage in achieving a high level of security for the teleoperators and knowing their respective competence reliably and on the system side. This also allows for more efficient deployment planning of teleoperators in a teleoperation center.Depending on a classification, the system can then automatically recognize which teleoperated driving maneuvers are to be carried out in the future, which suitable teleoperators are available for this purpose, and how these can be deployed as needed.

[0042] Depending on the classification, the system may authorize or block specific teleoperated maneuvers for the trained teleoperator. This can also prevent unauthorized actions by a teleoperator. It can also prevent the execution of a teleoperated maneuver for which the teleoperator is not, or not sufficiently, trained and qualified.

[0043] Examinations can consist of a single examination at the end of the training course. It is also possible for several, particularly smaller, examinations to be held during the training course. These can then also be created and administered by the system as needed. For example, they can include quizzes on the topics and aspects mentioned above. The content and the number of topics and content to be quizzed can also be individually compiled for an examination, which is also preferably done automatically by the system. It is also possible for an intermediate examination to be held during the training course and a large final examination to be held at the end of the training course. Such an examination can contain questions on the topics and aspects covered. In addition to or instead of this, it can also include a teleoperated training trip as mentioned above.

[0044] It is also possible for the teleoperation system to have a training simulator. At least part of the training can be conducted using this. The exam can also be partially conducted using this training simulator. In the exemplary embodiment, it is also possible for vehicle-specific views, such as interior views, to be displayed on the screen of the teleoperation system during training. This means that vehicle-specific training can also be carried out. Specific vehicles that the teleoperator being trained is to maneuver during teleoperated driving maneuvers after the training can therefore be presented even more efficiently. For example, a section of a dashboard of a specific vehicle can be displayed as an interior view. This improves the teleoperator being trained's understanding of the vehicle to be maneuvered.This is because a wide variety of control elements and / or systems, as well as assistance systems, can vary greatly between vehicles and / or in quality. This also makes it possible for such individual scenarios to be explained during such training. In particular, these vehicle-specific views are then also taught in a vehicle-specific manner. In particular, vehicle-specific assistance systems, such as a vehicle's parking assistance systems, can be trained in addition to or instead of these. This can significantly improve the teleoperator's understanding of what the respective vehicle can do and what can be adjusted and displayed. This also leads to greater safety when performing a teleoperated driving maneuver. It is also possible for the vehicle status of the vehicle to be maneuvered to be displayed on at least one screen during the training.This also allows for a better understanding of an individual vehicle that is to be maneuvered teleoperated. This allows for immediate and clear recognition and understanding of critical driving conditions of the vehicle, which may signal a vehicle condition precisely through this visual and / or acoustic information, during a subsequent teleoperated driving maneuver actually performed by the teleoperator.

[0045] In one embodiment, depending on the level of competence achieved by the trained teleoperator during the training, in particular at least during the initial training, follow-up training or extended training, such as an upgrade training, is carried out. This means that during the follow-up training, at least parts of the completed training are taught again and / or in a different way. During the extended training, additional new aspects can then be taught based on the at least one training course already completed in the past. In particular, it is possible for the time of the follow-up training and / or the extended training and / or the scope of the follow-up training or extended training to be determined by the system, in particular to be determined automatically depending on at least one decision criterion.This also provides a further advantage of the process, as it allows for automated, highly individualized responses to the teleoperator's current situation. This intelligently allows for a teleoperator's further training to be tailored to their needs, both in terms of time and scope.

[0046] In one embodiment, at least one of the following is considered as a decision criterion: - Number and / or type of teleoperated driving maneuvers performed by the teleoperator since the last training; - Quality of the teleoperated driving maneuvers performed by the teleoperator since the last training; - extent and / or type of training last provided; - current level of competence of the teleoperator with regard to his current knowledge and / or ability to carry out teleoperated driving maneuvers.

[0047] It is therefore also possible to obtain certification for teleoperators. This ensures that the teleoperators are particularly highly qualified, which in turn ensures a high level of security for the services provided.

[0048] A further aspect of the invention relates to a computer-implemented method for training a teleoperator to carry out a teleoperated driving maneuver of a vehicle, in particular with a real teleoperation system, comprising in particular the following steps: - Providing identification information of a teleoperator, with which the teleoperator is identified on a teleoperation system, to a computing device; - Determining the degree of competence of the teleoperator with regard to the suitability to carry out a teleoperated driving maneuver with a teleoperation system using the computing device; - Depending on the specific level of competence, selecting a training course for the teleoperator using the computing device for training the teleoperator on the teleoperation system on which the teleoperator is identified.

[0049] Advantageous embodiments of the above-mentioned first aspect concerning the method are envisaged as advantageous embodiments of the computer-implemented method. In particular, the required information is provided to the computing device, which then processes it accordingly to generate the individual results. This applies both to the individual compilation of training content and, if necessary, to the individual design of an examination.

[0050] A further aspect of the invention relates to a computer program product comprising instructions which, when the program is executed by a computer, in particular a computing device, cause the computer to carry out the method at least according to the above-mentioned aspect relating to the computer-implemented method or an advantageous embodiment thereof.

[0051] A further aspect of the invention relates to an electronic teleoperation system for performing a real teleoperated driving maneuver, which has at least one screen and a computing device. The teleoperation system has at least one training module with which a teleoperator can be trained in his or her level of competence for performing a real teleoperated driving maneuver using the teleoperation system itself. The training module can be a software module. However, it can also have hardware components. It can be part of a training system. This, in turn, is preferably part of the teleoperation system. With this teleoperation system, it is therefore possible for it to be intended both for performing real teleoperated driving maneuvers and simultaneously be used for training purposes.

[0052] A further aspect of the invention relates to an electronic teleoperation system having at least one screen and at least one computing device. This teleoperation system preferably has a training system, wherein the teleoperation system is designed to carry out a method according to the above-mentioned aspect or an advantageous embodiment thereof. In particular, the method is carried out using this teleoperation system.

[0053] Embodiments of the invention are explained in more detail below with reference to schematic drawings. They show: Fig. 1 a schematic representation of an embodiment of a teleoperation system according to the invention; Fig. 2 an exemplary representation of training information of a training of a teleoperator on a screen of the teleoperation system; and Fig. 3 shows a further exemplary representation of training information of a training of a teleoperator on the at least one screen of the teleoperation system.

[0054] In the figures, identical or functionally identical elements are provided with the same reference symbols.

[0055] In Fig. 1 shows a schematic representation of an embodiment of an electronic teleoperation system 1. The teleoperation system 1 is designed to offer teleoperated driving maneuvers of a vehicle carried out by a teleoperator. The teleoperation system 1 is designed, for example, in a teleoperation center 2. The teleoperation system 1 has at least one computing device 3. The computing device 3 can have one or more microprocessors. It can also have one or more memory units. The teleoperation system 1 also has at least one screen. In the exemplary embodiment, several, in particular three, screens 4, 5 and 6 are provided. In addition, the teleoperation system 1 has at least one teleoperation instrument. This can be, for example, a keyboard 7.Further possible examples of teleoperation instruments include a steering wheel and / or a joystick, as symbolically indicated by reference numerals 8 and 9. It is also possible for the teleoperation system 1 to have at least one element 10 that can be actuated by a foot. This can be, for example, a pedal, in particular an accelerator pedal or a brake pedal.

[0056] These components of the teleoperation system 1 are corresponding aids for a human teleoperator 11. The latter is positioned in the teleoperation center 2 when using the teleoperation system 1.

[0057] The teleoperation system 1 is particularly designed for wireless communication with at least one real vehicle 12 via a wireless communication interface 13. Thus, the vehicle 12 can be remotely maneuvered by means of the teleoperation system 1 and the teleoperator 11. In particular, the teleoperator 11 has no direct visual contact with the vehicle 12, at least in phases during the teleoperated driving maneuver, in particular the entire time.

[0058] The vehicle 12 is arranged in a teleoperation environment 14. Such a teleoperation environment 14 can, for example, be a parking area, such as a parking garage or the like. Such a teleoperation environment 14 is also possible in the context of a car wash, a service workshop, or the like. In particular, teleoperated maneuvering is thus provided for driving maneuvers that involve a maneuvering operation. In particular, speeds of the vehicle 12 during this teleoperated driving maneuver are thus provided that are less than or equal to 40 km / h, in particular less than 30 km / h.

[0059] In the exemplary embodiment, the teleoperation system 1 also comprises an electronic training system 15. This allows a teleoperator 11 to be trained with the real teleoperation system 1, which they use to perform real teleoperated driving maneuvers. The training system 15 can comprise at least one computing device. This can be the computing device 3. Preferably, the training system 15 comprises at least one training module. This can be a software module.

[0060] Thus, the teleoperation system 1 is also simultaneously and multifunctionally designed to train a teleoperator. This allows a teleoperator 11 to train and learn. This takes place, in particular, precisely on the real teleoperation system 1, which is then also available and intended to be used for the real teleoperated driving maneuvers to be performed.

[0061] In particular, it is intended that the training and thus the education of the teleoperator 11 is carried out at least partially, in particular fully automatically.

[0062] Thus, in one embodiment, it may be provided that training is deliberately requested and desired by the teleoperator 11. This can be the case, for example, by appropriate input into the teleoperation system 1.

[0063] However, it is also possible for training to be suggested and carried out by the system if the teleoperator 11 wishes to use the teleoperation system 1. In particular, it is then provided that the teleoperator 11 identifies himself to the teleoperation system 1. This can be done, for example, by logging into the system with his user data. This means that the system automatically knows which human teleoperator 11 currently wishes to use the teleoperation system 1. In particular, the teleoperation system 1 also stores the level of competence of a teleoperator 11. This can be done, for example, through linked data between the teleoperator 11 and the assigned level of competence.If a teleoperator 11 has not yet been registered in the teleoperation system 1 and identifies themselves for the first time, the system also recognizes this, and in one exemplary embodiment, the teleoperator 11 must then first undergo training, in particular initial training. This is specified by the system, for example, the training system 15 or the teleoperation system 1. The initial training can be a standardized training that is the same for all teleoperators to be trained. However, it is also possible for such initial training to be individually configured by the system. For example, the scope and content of the initial training can be individually created and configured by the system.This may, for example, depend on which teleoperator 11 is to be trained and / or which vehicles are to be maneuvered teleoperated in the teleoperation environment 14 and / or which specific teleoperation environment 14 exists.

[0064] Thus, after identifying the teleoperator 11, their level of competence is determined. Depending on this determined level of competence, the system automatically decides whether training is required to be able to use the teleoperation system 1 in the future, and if so, which training should be provided. At least during the initial training, topics and aspects can be taught, as already explained in detail above.

[0065] Training content can be displayed and presented acoustically and / or visually by the system. Both training videos and action-reaction scenarios can be implemented for this purpose. It is possible for the system to provide feedback to the teleoperator 11 being trained during the training. This can, on the one hand, improve the level of attention and / or understanding of the training content. It is also possible for the teleoperation system 1 to have at least one camera 16. This camera can record the teleoperator 11 at least temporarily during the training.Based on this information, the system can also assess, particularly by evaluating the image and video information, the extent to which the teleoperator 11 is attentively attending the training and / or whether, based on their behavior, they recognize that they lack, lack sufficient, or have a very good understanding of the training content presented. This allows the system to recognize and decide, even during the training, how the planned training can be adapted if necessary to optimize the training outcome for the teleoperator 11.

[0066] It is also possible that, at least during the initial training, a training drive is conducted in which the teleoperator 11 being trained performs or remotely controls a teleoperated training driving maneuver with a training vehicle equipped with the teleoperation system 1. The training vehicle can be a real vehicle, which is present, for example, on a training site or in the teleoperation environment 14. However, it is also possible that this training maneuver, and thus also the training vehicle if applicable, is generated by simulation and displayed in an animated manner on at least one of the screens 4 to 6.

[0067] Thus, in one embodiment, it is provided that when the teleoperator 11 registers or logs in to the teleoperation system 1, an automatic check is carried out to determine whether the teleoperator 11 is suitable to operate this teleoperation system 1 and, if so, with what maximum level of competence this is possible. Depending on the current individual handling of the teleoperation system 1 by the teleoperator 11, the system 11 can recognize and decide whether training of the teleoperator 11 is necessary or not, or to what extent training is necessary. In particular, the system can then also assess whether this training should be carried out immediately after the registration of the teleoperator 11 or whether this training can take place at a later time.

[0068] It is also possible that, depending on the current level of competence of the teleoperator 11, who, for example, has already completed initial training in the past, further training or advanced training is required or recommended. This may be the case, for example, if the software and / or hardware of the teleoperation system 1 has changed since the last training of the teleoperator 11. In addition to or instead of this, an evaluation of the teleoperated driving maneuvers performed after the last training can be taken into account when deciding whether further training of the teleoperator 11 is required, for example based on further training or advanced training.This can be the number of teleoperated driving maneuvers performed by the teleoperator 11 after the last training session and / or the assessed and stored quality of these teleoperated driving maneuvers performed by the teleoperator 11 and / or the respective difficulty level of the teleoperated driving maneuvers performed by the teleoperator 11. These influencing parameters, which allow the system to assess what further training is required and whether such further training is necessary, are not to be understood as exhaustive. Numerous other decision parameters or decision criteria can also be considered.

[0069] In the exemplary embodiment, it is also possible for the training of a teleoperator 11 to be accompanied by an intermediate examination and / or a final examination. This can also be assessed and decided automatically by the system. The type and scope of such an intermediate examination and / or a final examination can also be specified in a general manner in one exemplary embodiment or can be individually determined and configured by the system.

[0070] The teleoperation system 1 preferably also comprises a computer program product or a computer program with which a method explained above can be carried out. In particular, the training of a teleoperator 11 to perform a teleoperated driving maneuver of a vehicle can also be carried out using a computer-implemented method. In particular, a computing device 3 can be used to decide whether training is required and, if so, which training is required and / or which training needs to be individually configured.

[0071] It is also possible during a training session that, according to the schematic representation in Fig. 2 a user interface 17 is displayed on at least one screen, for example the screen 5. For this purpose, symbols can be displayed that are assigned to specific functions by means of which the vehicle 12 can be maneuvered teleoperated. For example, this can be a control element 18 that symbolizes the reverse gear of the vehicle 12. Another element 19 can symbolize, for example, the electronic parking brake of a vehicle 12. Another element 20 can symbolize, for example, the forward driving state of the vehicle 12. Further symbols or elements 21, 22, which represent turn signals of the vehicle 12, and 23, which represents a hazard warning light of the vehicle 12, can also be displayed. These elements, the position, function and number of which are by no means exhaustive but merely exemplary, can also be individually displayed during training.For this purpose, it can be provided during training, as is the case with a further element 24, that a written explanation 25 is designed, at least temporarily, to explain this individual element 24. This explanation can be displayed as a temporary overlay. Furthermore, it is possible for the vehicle 12 to be maneuvered or to be maneuvered teleoperated to be shown in a further element 26. For example, this can be a top view. It is also possible for this view to be changed. Furthermore, a further element 27 can be displayed, which here, for example, triggers an emergency stop of the vehicle 12 when actuated. Further additional elements with corresponding displays 28 are also possible, which can be shown and explained during training.Here, for example, latency times that may arise between the actions of a teleoperator 11 at his teleoperation station and the actual implementation of these commands by the vehicle 12 can also be displayed and explained.

[0072] In Fig. In this context, only elements or symbols are shown in Figure 2, which can be an example of a user interface of the teleoperation system 1. However, numerous other configurations of such a user interface are possible, which can then also be displayed and trained.

[0073] In Fig. 3 shows a schematic representation of an at least pictorial presentation explaining actions that must be performed by a teleoperator 11 in order to be able to use the teleoperation system 1 for a teleoperated driving maneuver. For example, according to a displayed symbol 29, a pedal, for example a brake pedal such as pedal 10, can first be actuated. For this purpose, it can be provided that this pedal can be actuated over an at least predetermined actuation path, which is, for example, less than 50 percent of the total possible actuation path, for example 30 percent of this total actuation path. Furthermore, it can be provided that, according to the illustrated element 30, an operating element of the teleoperation system 1 is to be actuated simultaneously or subsequently, for example, a start button must be pressed or a touch-sensitive control panel must be touched.

[0074] This means that the exemplary scenario in Fig. 3 also enables training on how the teleoperation system 1 can be started and taken over by a teleoperator 11. In this context, it is also possible that additionally on this display according to Fig. 3 an optical progress bar is displayed which signals how many of the required actions have already been successfully completed by the teleoperator 11 in order to start the teleoperation system 1.

[0075] In general, it is therefore also possible for a system-based classification of the teleoperator 11 to occur depending on the current or previously conducted training and the training outcome. In particular, such a classification may depend, for example, on the level of competence. Depending on the class, a teleoperator 11 may then be authorized for specific remote-controlled driving maneuvers and blocked from others, and / or blocked or authorized for specific driving maneuvers in specific teleoperation environments 14, and / or blocked or authorized for specific vehicle types.

Claims

[1] Method for training a teleoperator (11) to carry out a teleoperated driving maneuver of a vehicle (12), comprising the following steps: - identifying a teleoperator (11) on a teleoperation system (1); - determining the degree of competence of the teleoperator (11) with regard to the suitability to carry out a teleoperated driving maneuver with a teleoperation system (1); - depending on the specific level of competence, carrying out training of the teleoperator (11) on the teleoperation system (1). [2] Method according to claim 1, wherein the training is specified on the system side by a training module and the training process is carried out automatically on the teleoperation system (1) by the training module. [3] Method according to claim 1 or 2, wherein upon initial identification of a teleoperator (11), initial training for the teleoperator (11) is carried out on the teleoperation system (1). [4] Method according to claim 3, wherein at least one of the following topics is taught at least during the initial training: - general explanation of the functioning of a teleoperation system (1) and the rules relating thereto; - general explanation of the functioning of the teleoperation system (1) to which the teleoperator is currently identified; - explanation of virtual representations, in particular operating elements and / or functional representations and / or symbols, on at least one screen (4, 5, 6) of the teleoperation system (1), which are displayed as part of a teleoperated driving maneuver and describe the functions and / or states of the vehicle (12) to be maneuvered and / or of the teleoperation system (1); - performing actions until the teleoperation system (1) is in an active state for performing a teleoperated driving maneuver. [5] Method according to one of the preceding claims, wherein at least one of the following aspects is trained at least during the initial training: - Latency times between optical and / or acoustic information between the real vehicle environment and the display on the screen (4, 5, 6) of the teleoperation system (1); - Latency times between an operation of a teleoperation instrument (7, 8, 9, 10) of the teleoperation system (1) and the active implementation of the function of the vehicle (12) linked to this operation. [6] Method according to one of the preceding claims, wherein during the initial training, training videos are displayed on the at least one screen (4, 5, 6) of the teleoperation system (1). [7] Method according to one of the preceding claims, wherein during the initial training an interaction is carried out between the teleoperator (11) to be trained and the system (1, 15), in particular an action-reaction scenario is carried out. [8] Method according to claim 7, wherein the quality of the teleoperator's training behavior during the action-reaction scenario is automatically communicated to the teleoperator visually and / or acoustically by the system (1, 15). [9] Method according to one of the preceding claims, wherein during the initial training at least one training drive is carried out, in which the teleoperator (11) to be trained carries out a teleoperated training driving maneuver with a training vehicle with the teleoperation system (1). [10] Method according to one of the preceding claims, wherein at least during the initial training for optical representations on the at least one screen (4, 5, 6) of the teleoperation system (11) additional explanations (25), in particular associated with the representation, are displayed and / or an avatar is displayed on the screen (4, 5, 6) which shows optical and / or acoustic additional explanations. [11] Method according to one of the preceding claims, wherein during the initial training at least one reference maneuver is displayed and explained on the screen (4, 5, 6) of the teleoperation system (1). [12] Method according to one of the preceding claims, wherein the training is automatically selected and / or automatically configured by the system (1, 15) depending on the type of teleoperation system (1) on which the training is carried out and / or depending on the teleoperation system (1) on which the teleoperator (11) to be trained is to carry out teleoperated driving maneuvers after the training, and / or depending on the degree of competence of the teleoperator (11) to be trained with regard to his current knowledge and / or skills in carrying out teleoperated driving maneuvers, and / or depending on the environment in which teleoperated driving maneuvers are to be carried out in the future by the teleoperator (11) to be trained, and / or depending on the type of vehicles (12) that the teleoperator (11) to be trained is to maneuver in the future by teleoperation. [13] Method according to one of the preceding claims, wherein during the training an examination of the teleoperator (11) to be trained is taken online, and the system (1, 15) automatically groups the teleoperator (11) to be trained into one of at least two different classes of teleoperators, taking into account the result of the examination. [14] Method according to claim 13, wherein, depending on the classification, specific teleoperated driving maneuvers are enabled or disabled for the trained teleoperator (11) by the system (1, 15). [15] Method according to one of the preceding claims, wherein during training, vehicle-specific interior views are displayed on the screen (4, 5, 6) and are trained in a vehicle-associated manner and / or vehicle-specific assistance systems are trained. [16] Method according to one of the preceding claims, wherein, depending on the level of competence achieved by the trained teleoperator (11) during the training, in particular the initial training, a follow-up training (repetition) or an extension training (learning new things) is carried out, wherein the time of the follow-up training or the extension training and / or the extent of the follow-up training or the extension training is determined by the system (1, 15) depending on at least one decision criterion. [17] Method according to claim 16, wherein the following is taken into account as a decision criterion: - number and / or type of teleoperated driving manoeuvres carried out by the teleoperator (11) since the last training; - quality of the teleoperated driving manoeuvres carried out by the teleoperator (11) since the last training; - extent and / or type of training last provided; - current level of competence of the teleoperator (11) with regard to his current knowledge and / or ability to carry out teleoperated driving manoeuvres. [18] Computer-implemented method for training a teleoperator to perform a teleoperated driving maneuver of a vehicle (), comprising the following steps: - Providing identification information of a teleoperator, with which the teleoperator (11) is identified on a teleoperation system (1), to a computing device (3); - determining the degree of competence of the teleoperator (11) with regard to the suitability to carry out a teleoperated driving maneuver with a teleoperation system (1) using the computing device (3); - Depending on the determined level of competence, selecting a training course for the teleoperator (11) with the computing device (3) for training the teleoperator (11) on the teleoperation system (1) on which the teleoperator is identified. [19] A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method at least according to claim 18. [20] Electronic teleoperation system (1) for carrying out a real teleoperated driving maneuver, which has at least one screen (4, 5, 6) and at least one computing device (3), and which has at least one training module with which a teleoperator (11) can be trained in his level of competence for carrying out a teleoperated driving maneuver with a teleoperation system (1). [21] Electronic teleoperation system (1) with at least one screen (4, 5, 6) and a computing device (3), and with a training system (15), wherein the teleoperation system (1) is designed to carry out a method according to one of the preceding claims 1 to 17.

Images