Methods for the autonomous or teleoperated operation of a passenger transport vehicle and autonomously driving or teleoperated passenger transport vehicle

Abstract

The invention relates to a method for the autonomous or teleoperated operation of a passenger transport vehicle, wherein a passenger is transported in the passenger transport vehicle. Acceptance of using an autonomous or teleoperated passenger transport vehicle is increased by displaying a moving image of a driver to the passenger in the passenger transport vehicle.

Description

[0001] The invention relates to a method for the autonomous or teleoperated operation of a passenger transport vehicle having the features of the preamble of patent claim 1. Furthermore, the invention relates to an autonomously driving or teleoperated passenger transport vehicle for carrying out the method having the features of the preamble of patent claim 11.

[0002] Although remote-controlled or autonomous passenger transport vehicles have been described, they have not yet been implemented. Various studies on the acceptance of driverless, i.e., autonomous or teleoperated driving, have shown that people do not trust the technology to drive an autonomous or teleoperated passenger transport vehicle as safely as a human driver present in the vehicle. This is particularly true for older generations. For example, a large majority of Germans surveyed currently cannot imagine being driven autonomously, e.g., in the form of "robo-taxis." The concerns expressed primarily relate to safety. Conversely, proponents of autonomous or teleoperated passenger transport vehicles see safety as one of the greatest advantages.

[0003] The methods and passenger transport vehicles known in the state of the art are not yet optimally designed.

[0004] The disclosed methods have the disadvantage that future passengers must first build trust in order to use an autonomously driving or teleoperated passenger transport vehicle. Such changes may pose acceptance problems.

[0005] The invention is therefore based on the object of providing a method for the autonomous or teleoperated operation of a passenger transport vehicle and a passenger transport vehicle, wherein the method and the passenger transport vehicle are improved and the acceptance of the use of an autonomous or teleoperated passenger transport vehicle is increased.

[0006] This problem underlying the invention is now solved by a method for the autonomous or teleoperated operation of a passenger transport vehicle and a passenger transport vehicle having the features of the independent patent claims.

[0007] The passenger in the passenger transport vehicle is shown a moving image of a driver. Having a human being present as the active driver increases passenger acceptance of the technical system for autonomous or remotely operated passenger transport vehicles. This alleviates uncertainty among broad sections of the population, who, according to studies, will not initially use passenger transport vehicles, and increases acceptance of the "teleoperated driving" system.

[0008] An image of the teleoperator controlling the driverless passenger transport vehicle can be displayed to the passenger. The screen shows the currently moving teleoperator, giving the passenger the feeling of being driven by a competent chauffeur rather than a technical system.

[0009] A control center has a workstation for the teleoperator. The workstation serves the teleoperator to control the passenger transport vehicle. The workstation has at least one camera, which can be used to film the teleoperator, and a camera recording can be or is transmitted to a monitor of the passenger transport vehicle. Thus, an image of the teleoperator, in particular a live image, is displayed on the monitor.

[0010] A tablet in the passenger transport vehicle can serve as a monitor. The monitor displays the filmed teleoperator during the journey.

[0011] Different embodiments are conceivable as to how the perspective is selected. Preferably, a perspective of the driver is shown from behind over the shoulder so that the passenger can see that the driver has his hands on the steering wheel, and / or alternatively a frontal view or a side view of the driver is shown. The perspective of the driver can be shown in the classic way, as in a taxi, from behind over the shoulder so that the passenger can see that the teleoperator has his hands on the steering wheel. Alternatively, the driver's perspective can also be selected from the front and / or the side. It is conceivable that the passenger and / or the teleoperator can choose between different perspectives that are displayed. It is conceivable that several perspectives of the driver are shown on the monitor. Combinations are also conceivable.

[0012] In an alternative embodiment, it is possible for the passenger to be shown an artificially created, computer-generated image of a driver, particularly the teleoperator. Using artificial intelligence (AI) techniques, which are used to create so-called deep fakes, the artificial apparition can be presented as a realistic-looking person.

[0013] Preferably, the artificial appearance is selected depending on a user profile of the passenger. The user profile preferably includes the passenger's gender, age, origin and / or social media data. Based on the data and the user profile of the passenger, in particular gender, age, origin and social media data, the driver's appearance can be changed using filters based on age and / or ethnicity (if any prejudices of the passenger can be identified from social media usage) in order to present the driver with an appearance that appears pleasant and trustworthy to the passenger. The user data can be obtained in part from the customer's user profile in the teleoperation company's app.

[0014] The teleoperator's appearance can thus be given a competent or even customer-tailored, trustworthy appearance through a filter or deepfake process. Furthermore, the customer-tailored artificial driver image / video (e.g., deepfake) would eliminate the need for video data transmission, thus saving bandwidth. The saved bandwidth would allow more vehicles to be remotely controlled within a cell and / or increase the reliability of the teleoperation service.

[0015] For this purpose, an artificial neural network, in particular an autoencoder method, is preferably used as a model. This neural network is trained in advance using at least one facial image of an existing person, in particular multiple facial images of the person and / or a video of the person. The relevant facial expression and lighting information is encoded and extracted from at least one facial image of the teleoperator, in particular multiple facial images of the teleoperator and / or a video of the teleoperator, and a corresponding image of the driver is generated from the encoded information. High-resolution models that can create close-ups of faces in high-resolution videos can already be trained using commercial graphics cards.These models can also be partially deployed in real time, with as little time delay as possible, to generate an image of the teleoperator or a non-existent driver. Only a few minutes of video of the driver are required as training material. The training material should contain as many different facial expressions and perspectives as possible so that the model can learn them to generate the image of the driver.

[0016] The motor vehicle preferably has a microphone and a loudspeaker so that the passenger can communicate with the teleoperator or a chatbot. This allows different head movements, facial expressions, or lip movements to be generated in the driver's image during the conversation. This makes it possible to create visually deceptively realistic videos in which, for example, the driver speaks to the passenger and the facial expressions, head movements, and lip movements are adapted to the driver's statements. The method preferably generates a 3D model of the driver's face in real time.

[0017] Computer-generated image synthesis can create new people who don't exist in reality. Preferably, however, an image of the real teleoperator is generated. Alternatively, it is conceivable that a nonexistent person could be generated using the process.

[0018] It is conceivable that an audio signal is recorded by the teleoperator and transmitted in real time. Furthermore, it is possible for text messages to be transmitted from the teleoperator to the passenger transport vehicle and read out to the passenger using a TTS system on the passenger transport vehicle (TTS: Text-to-Speech). The teleoperator can specify a text, which is then processed by the TTS system and converted into an audio signal. The semantic content of this signal corresponds to that of the specified text, and the speaker-specific characteristics preferably correspond to the represented driver, i.e. in particular to the teleoperator. This reduces the required bandwidth, since only text messages are transmitted from the teleoperator to the passenger transport vehicle.

[0019] Furthermore, the passenger transport vehicle can be equipped with a speech recognition system, which converts the passenger's recorded speech into a text message in real time. The text message can be forwarded to the teleoperator, answered via the chatbot, or answered by the teleoperator. This reduces the bandwidth required for data transmission.

[0020] In one embodiment of the method, the passenger's communication with the driver is generated at least partially by a chatbot. The chatbot uses artificial intelligence to communicate with users, particularly via text-based messages. Machine learning technologies are used to generate responses via the chatbot that sound natural and are relevant to the conversation. The chatbot can operate in a cloud-based manner, or it is conceivable that the chatbot runs on a data processing system of the vehicle.

[0021] The messages transmitted to the vehicle as text-based messages are read aloud to the passenger using the TTS system, while the driver's display is adapted to the pronunciation of the message.

[0022] There are now numerous possibilities for developing and further developing the invention. For this purpose, reference may first be made to the patent claims subordinate to the independent patent claims.

Claims

[1] Method for the autonomous or teleoperated operation of a passenger transport vehicle, wherein a passenger is transported in the passenger transport vehicle, characterized by that the passenger in the passenger transport vehicle is shown a moving image of a driver. [2] Method according to claim 1, characterized by that the passenger is shown an image of the teleoperator controlling the driverless passenger transport vehicle. [3] Method according to claim 1 or 2, characterized by that a workstation of the teleoperator has at least one camera, by means of which at least one camera the teleoperator is filmed, wherein a recording of the at least one camera is transmitted to a monitor of the passenger transport vehicle. [4] Method according to one of the preceding claims, characterized bythat a perspective of the driver from behind over the shoulder is shown so that the passenger can see that the driver has his hands on the steering wheel, and / or a frontal view or a side view of the driver is shown. [5] Method according to one of the preceding claims, characterized by that the passenger is shown an artificially created, computer-generated image of the driver, in particular of the teleoperator, preferably using artificial intelligence methods. [6] Method according to one of the preceding claims, characterized by that the artificial appearance is selected depending on a passenger's user profile. [7] Method according to one of the preceding claims, characterized bythat an artificial neural network, in particular an autoencoder method, is used as a model, wherein the neural network is trained in advance with at least one facial image of an existing human, preferably of the teleoperator, in particular several facial images of the human and / or a video of the human. [8] Method according to one of the preceding claims, characterized by that the vehicle has a microphone and a loudspeaker, whereby the passenger can talk to the teleoperator or a chatbot, whereby during the conversation different head movements, facial expressions or lip movements are generated and displayed in the image of the driver. [9] Method according to one of the preceding claims, characterized bythat text messages are transmitted from the teleoperator to the passenger transport vehicle and these are read out to the passenger using a TTS system of the passenger transport vehicle (TTS: Text-to-Speech System). [10] Method according to one of the preceding claims, characterized by that the passenger transport vehicle is equipped with a speech recognition system, whereby the speech recorded by the passenger is converted into a text message in real time. [11] Autonomously driving or teleoperated passenger transport vehicle for carrying out the method according to one of the preceding claims.