Method for operating an at least partially automated motor vehicle by a teleoperator, computer program product, computer-readable storage medium and assistance system

Abstract

The invention relates to a method for operating an at least partially automated motor vehicle (2) by a teleoperator (3), comprising the steps of: detecting at least one parking space (11) in an environment (12) of the motor vehicle (2); transmitting the detected parking space (11) to the teleoperator (3); detecting a selection of the parking space (11) by the teleoperator (3); and automatically parking (13) the motor vehicle (2) into the selected parking space (11). The invention further relates to a computer program product, a computer-readable storage medium, and an assistance system (1).

Description

The present invention relates to a method for operating an at least partially automatically operated motor vehicle by a teleoperator. The invention further relates to a corresponding computer program product, a computer-readable storage medium and an assistance system.Teleoperated operation of a motor vehicle is already known from the prior art. For this purpose, a teleoperator is located remote from the motor vehicle, in particular not within the visibility range, and can transmit corresponding driving commands, for example for a longitudinal acceleration and a lateral acceleration, to the motor vehicle, for example on the basis of control inputs on an electronic computing device, for example a computer. The motor vehicle is then controlled accordingly on the basis of these corresponding control commands. For this purpose, the motor vehicle has, for example, corresponding longitudinal acceleration and lateral acceleration devices, which can be appropriately controlled on the basis of the control commands. The teleoperated driving mode can be used, for example, in emergency situations of automated driving. If, for example, an automated motor vehicle is no longer able to perform a correct control, for example, on account of a deficient detection of the environment, the driving mode can be transferred to a teleoperator and the teleoperator can assume a safe control of the motor vehicle.Furthermore, teleoperated driving operation is also known, for example, in so-called vallet parking parking garages, in which a user of the motor vehicle releases the motor vehicle accordingly at a release station and the motor vehicle is then guided to a parking facility without the user or driver himself. This guidance between the delivery location to the parking facility can be adopted by the teleoperator, for example. Furthermore, the parking process up to a transfer location, at which the motor vehicle is transferred again to the user, can also be transferred accordingly by the teleoperator.In particular, however, the parking process or also the unparking process in corresponding parking spaces can overload the teleoperator accordingly because of its non-presence within the motor vehicle or is associated with great efforts in order to maneuver the motor vehicle into a narrow parking space, for example.US 2020 / 19 23 51 A1 relates to a method, an apparatus and a computer-readable storage medium for updating the lane using a remote vehicle controller. According to one embodiment, the method, apparatus, and computer readable storage medium of the present disclosure described above generally relate to identifying an obstacle that blocks at least a portion of a road based on data received from one or more vehicle sensors, determining whether a trajectory cannot be found to operate the vehicle with respect to the identified obstacle while the vehicle is operating in the autonomous mode, sending, upon determining that the trajectory cannot be found, a request to a remote controller to navigate the vehicle, operating the vehicle in a remote mode based on instructions received from the remote controller, wherein a motion trajectory of the vehicle is recorded while operating in a remote mode, and updating a navigation map based on at least the recorded movement trajectory.It is an object of the present invention to provide a method, a computer program product, a computer-readable storage medium and an assistance system, by means of which an improved teleoperated driving mode of an at least partially automatically operated motor vehicle or partially autonomously operated motor vehicle can be realized.This object is achieved by a method, a computer program product, a computer-readable storage medium and an assistance system according to the independent patent claims. Advantageous embodiments are specified in the dependent claims.One aspect of the invention relates to a method for operating an at least partially automatically operated motor vehicle by a teleoperator. At least one parking space is detected in the surroundings of the motor vehicle. The detected parking space is transmitted to the teleoperator. A selection of the parking space is detected by the teleoperator and the motor vehicle is automatically parked in the selected parking space.Thus, in particular, improved operation for teleoperated driving can be realized.In particular, for this purpose, for example, the motor vehicle can have a corresponding detection device, in particular an environment detection device, and reliably detect the environment. These data are then in turn transmitted to the teleoperator, which is located in particular externally to the motor vehicle and without being in view with the motor vehicle. For example, the teleoperator can then view the captured environment on a display device and transmit corresponding control commands for the motor vehicle, for example via a keyboard, mouse or a joystick, to the motor vehicle in order to be able to realize a teleoperated driving mode of the motor vehicle. These corresponding commands are then in turn transmitted to the motor vehicle, and a corresponding longitudinal acceleration device or transverse acceleration device of the motor vehicle for at least partially automated operation or partially autonomous operation of the motor vehicle can be controlled accordingly. On the basis of the control commands by the teleoperator, a movement of the motor vehicle can then be initiated in accordance with the control commands.A parking space detected by the motor vehicle and in particular also identified can then be displayed to the teleoperator on the display device, for example. For example, superposition of the captured environment can be realized by a corresponding display of the parking space. For example, a P for the identified parking space can be displayed on the display device at the location of the detected parking space on the display device. By detecting an input, for example this superimposed display and thus a selection of the parking space as a parking space for the motor vehicle, this selection can now be transmitted to the motor vehicle. On the basis of this selection, the motor vehicle can drive into the parking space automatically, for example via its own parking assistant, in particular without further commands by the teleoperator. A teleoperated service can thus be combined with the parking assistant already installed in the motor vehicle.This has the particular advantage that the teleoperator does not have to generate its own control commands for the parking process, which simplifies the work of the teleoperator in particular. In particular in the case of narrow gaps, for example, and since only a part of the environment of the motor vehicle is transmitted to the teleoperator, it is thus possible to prevent incorrect driving commands from being transmitted from the teleoperator to the motor vehicle. For example, in the case of teleoperators that are not yet under way, stress can thus be reduced in the teleoperator and a parking space can nevertheless be reliably detected and correspondingly automatically parked.For this purpose, it can be provided in particular that both the motor vehicle and the teleoperator have a corresponding communication device which communicate with one another. In this case, for example, a respective separately designed electronic computing device can be provided on the part of the teleoperator and the motor vehicle, which electronic computing device can convert corresponding commands, for example for controlling the longitudinal acceleration device and / or the transverse acceleration device in the motor vehicle.In particular, the invention thus proposes a teleoperated driving mode of the motor vehicle with automated parking and / or unparking and / or replay activation (Tracked Parking) and / or RDA activation (Rearward Driving Assistance). The motor vehicle can also perform garage parking or parking taking account of a charging station, for example, after activation.As already mentioned, the motor vehicle can also be operated with the proposed method independently of the parking process accordingly during the unparking process. For this purpose, it can be provided, for example independently of a parking process, that the parked motor vehicle carries out an environment detection and, for example, automatically moves out of the parking space and then once again carries out the teleoperated operation or a transfer for control to the teleoperator after the automated parking process.As a rule, during teleoperated driving operation, a predefined area of the environment is captured by means of a camera system which has at least one camera, wherein a plurality of individual images following one another in time are provided, specifically independently of the captured area of the environment. The camera system can accordingly form an image source or be a part thereof. The individual images are transmitted or transmitted as a video data stream to a remote receiving station and evaluated by the receiving station in order to obtain information about the captured environment and / or its change. In teleoperated driving, the remote site may be formed by a teleoperation center or a control center with the teleoperator. In the case of an at least partially autonomously guided motor vehicle, in particular a keyless motor vehicle, instructions for continuing a driving operation of the motor vehicle can be provided by means of the control center, for example in order to remotely control the motor vehicle or the like.According to an advantageous embodiment, the detected parking space is displayed as a superimposed symbol for the teleoperator. For example, the teleoperator can have a corresponding display device. An environment detection can then be displayed accordingly on the display device. The parking space can then be displayed on the display device as a superimposed symbol, for example by "P" and a corresponding border of the outlines of the parking space. Thus, by simply selecting this displayed parking space, the automated parking process can be initiated by the teleoperator. This makes it possible to provide a simple possibility of carrying out automated parking of the motor vehicle.A further advantageous embodiment provides that a starting position for automated parking is proposed to the teleoperator. For example, after a selection of the teleoperator of the parking space, a start position for the automated parking process can be displayed to the teleoperator. The teleoperator can thus bring the motor vehicle into the corresponding starting position by means of corresponding control commands, after which automated parking can subsequently be carried out, in particular without intervention by the teleoperator. Thus, a reliable automated parking of the motor vehicle into the parking space can be realized.It has furthermore proven advantageous if the method is carried out in a parking garage as the environment. The parking garage can be provided in particular in the form of so-called automated vallet parking. A user can in this case dispense the motor vehicle at a general dispensing station or dispensing location and the motor vehicle is then teleoperated to the vicinity of the parking space. The motor vehicle can then be parked in an automated manner in the vicinity of the parking facility.It can furthermore be provided that the motor vehicle is then also automatically unparked from this parking space, is appropriately taken over by the teleoperator and is received again by the user at a take-over station. An advantageous parking situation for the motor vehicle can thus be created.It has furthermore proven advantageous if a forward-directed parking maneuver and / or a backward-directed parking maneuver is proposed to the teleoperator. In particular, depending on which parking space is detected, different parking maneuvers can be proposed to the teleoperator. The latter can then select which parking maneuvers can be or are to be carried out by the motor vehicle. Thus, a reliable parking of the motor vehicle into the parking space can be realized.In a further advantageous embodiment, it is provided that the environment is detected by means of an ultrasonic sensor device and / or radar sensor device and / or a lidar sensor device and / or a camera. The environment can thus be detected on the basis of different sensor systems of the motor vehicle. This allows reliable environment detection and parking space detection to be realized.In a further advantageous embodiment, it is provided that an automated unparking maneuver from the parking space is proposed in addition to the teleoperator. In particular after parking, a parking-out maneuver can be proposed to the teleoperator. For example, the unparking maneuver can then likewise be carried out in an automated manner. After the parking has ended, the teleoperator can then take over again. As already mentioned, this aspect of the invention is a supplementary aspect, but also an independent aspect. Thus, automated unparking from the parking space and the takeover by the teleoperator after unparking are to be considered as an independent aspect of the invention.Furthermore, it has proven advantageous if a direction of unparking from the parking space is proposed to the teleoperator. For example, an environmental detection can result in a parking-space removal direction to the left appearing possible, but not to the right, because there is, for example, a wall or other vehicles are currently located. Thus, a corresponding unparking maneuver or a direction for unparking can be proposed to the teleoperator. Furthermore, depending on where a corresponding exit of the parking environment is located, a corresponding direction may be proposed. This reduces the effort for the teleoperator.A further advantageous embodiment provides that a takeover position for teleoperated operation is displayed to the teleoperator after the unparking maneuver. In particular, after the teleoperating process has been removed from the park, the teleoperator can therefore also have a takeover position displayed at which he must take over the teleoperated driving mode. Thus, after reaching the motor vehicle in this position, a takeover can take place. The teleoperator is thus informed as to when the automated unparking process is ended. Thus, an improved teleoperated operation of the motor vehicle can be realized.Yet another advantageous embodiment provides that a parking display in the motor vehicle is transmitted to the teleoperator. In particular, as already known from the prior art, a parking display can be displayed for the user in the motor vehicle, for example in a bird's eye view on a display of the motor vehicle. This view, in particular the parking space detected thereby and in the birdview view, can then likewise be transmitted to the teleoperator. This person thus receives, for example, in addition to the detected environment, the birdview view with the displayed parking space. The teleoperator can thus reliably make a selection with respect to the parking space and, for example, monitor a partially autonomous or fully autonomous parking process accordingly.Furthermore, it has proven advantageous if an environment detection of the motor vehicle is transmitted to the teleoperator during the automated parking maneuver. For example, personal monitoring can be carried out. In particular, corresponding warning messages, for example with respect to distances to persons or walls or further motor vehicles, can then also be transmitted to the teleoperator. This can thus reliably monitor the automated parking process.According to a further advantageous embodiment, the plurality of parking spaces is proposed to the teleoperator when detecting a plurality of parking spaces. For example, a plurality of corresponding symbols can be displayed on the display device for the parking spaces. The teleoperator can then initiate a corresponding parking process by selecting one of these parking spaces. Thus, a reliable parking of the motor vehicle can be realized.The presented method is in particular a computer-implemented method. Therefore, a further aspect of the invention relates to a computer program product having program code means which cause an electronic computing device to carry out a method according to the preceding aspect when the program code means are executed by the electronic computing device.Furthermore, the invention also relates to a computer-readable storage medium having at least the computer program product according to the preceding aspect.Yet another aspect of the invention relates to an assistance system for operating an at least partially automatically operated motor vehicle by a teleoperator, wherein the assistance system is designed to carry out a method according to the preceding aspect. The assistance system has, for example, a teleoperator-side electronic computing device and a motor vehicle-side electronic computing device, wherein these in turn communicate with one another. Corresponding control commands can then be transmitted from the teleoperator to the motor vehicle. Furthermore, the assistance system has at least access to a corresponding parking assistant within the motor vehicle. In particular, the method is also carried out by means of the assistance system.Advantageous embodiments of the method are to be regarded as advantageous embodiments of the computer program product, the computer-readable storage medium and the assistance system. For this purpose, the assistance system has, in particular, subject-matter features in order to be able to carry out corresponding method steps.In the present disclosure, a computing unit / electronic computing device may be understood to be, for example, a data processing device having processing circuits. A computing unit can therefore perform computing operations in order to process data. The computational operations may also include indexed accesses to a data structure, such as a look-up table (LUT).A computing unit can comprise in particular one or more computers, one or more microcontrollers and / or one or more integrated circuits, for example one or more application-specific integrated circuits, ASIC ASIC (application-specific integrated circuit), one or more field programmable gate arrays, FPGAs, and / or one or more single-chip systems, SoC (system on a chip). The computing unit may also contain one or more processors, for example one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors, DSPs. The computing unit may also comprise a physical or virtual cluster of computers or other of the aforementioned units.A computing unit may also include one or more hardware and / or software interfaces and / or one or more memory units. In this case, a memory unit can be embodied as a volatile data memory, for example as a dynamic random access memory, DRAM (dynamic random access memory), or as a static random access memory, SRAM (static random access memory), or as a nonvolatile data memory, for example as a read-only memory, ROM (read-only memory), programmable read-only memory, PROM (programmable read-only memory), an erasable programmable read-only memory, EPROM (electrically erasable programmable read-only memory, EEPROM (electrically erasable programmable read-only memory), a flash memory or flash EEPROM, a ferroelectric random access memory, FRAM (ferromagnetic random access memory), a magnetoresistive random access memory, MRAM (Magnetoresistive random access memory), or a phase-change random access memory, PCRAM (Phase-change random access memory).A surroundings sensor system / surroundings detection device, in particular for detecting the surroundings, may be understood, for example, as a sensor system which is capable of generating sensor data or sensor signals which map, represent or reflect a surroundings sensor system. In particular, the ability to detect electromagnetic or other signals from the environment is not sufficient to regard a sensor system as a surroundings sensor system. For example, cameras, radar systems, lidar systems or ultrasonic sensor systems can be considered environment sensor systems.An electronic vehicle guidance system / assistance system can be understood to mean an electronic system which is configured to guide a vehicle fully automatically or fully autonomously, in particular without an intervention in a control by a driver and / or teleoperator being required. The vehicle automatically performs all necessary functions, such as steering, braking and / or acceleration maneuvers, the observation and detection of the road traffic, and corresponding reactions. In particular, the electronic vehicle guidance system may implement a fully automatic or fully autonomous driving mode of the motor vehicle according to level 5 of the classification according to SAE J3016. An electronic vehicle guidance system may also be understood to mean an advanced driver assistance system (ADAS), which assists the driver in partially automated or partially autonomous driving. In particular, the electronic vehicle guidance system may implement a partially automated or semi-autonomous driving mode according to levels 1 through 4 according to the SAE J3016 classification. Here and in the following, "SAE J3016" refers to the corresponding standard in the April 2021 version.The at least partially automatic vehicle guidance may therefore include guiding the vehicle according to a level 5 fully automatic or fully autonomous driving mode according to SAE J3016. The at least partially automatic vehicle guidance may also include guiding the vehicle according to a partially automated or partially autonomous driving mode according to levels 1 to 4 of SAE J3016.The at least one control signal can be provided, for example, to one or more actuators of the motor vehicle, including, for example, one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the motor vehicle. The one or more actuators may influence a longitudinal and / or lateral control of the motor vehicle in order to at least partially automatically guide the motor vehicle.The assistance information can be output via an output device of the motor vehicle, for example a display and / or an audio output system and / or a haptic output system. For use cases or application situations which can result from a method according to the invention and which are not explicitly described herein, provision can be made for an error message and / or a request for inputting a user feedback to be output and / or for a default setting and / or a predetermined initial state to be set according to the method.Further features of the invention are evident from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description and the features and combinations of features mentioned below in the description of the figures and / or shown in the figures can be included by the invention not only in the respectively specified combination but also in other combinations. Embodiments and combinations of features may also be included in the invention, which do not have all features of an originally formulated claim. The invention may also include embodiments and combinations of features that go beyond or depart from the combinations of features set forth in the appended claims.The following are shown: FIG. 1 is a schematic block diagram according to an embodiment of an assistance system; and FIG. 2 shows a schematic view of an embodiment of the method.In the figures, identical or functionally identical elements are provided with the same reference numerals.FIG. 1 shows a schematic block diagram according to one specific embodiment of an assistance system 1 for a teleoperated driving mode of a motor vehicle 2. a teleoperator 3 is shown for this purpose, which in particular does not have a line of sight to motor vehicle 2. The teleoperator 3 can transmit corresponding control commands to the motor vehicle 2, for example, via an electronic computing device 4. For this purpose, in particular a communication device 5 can be provided which communicates by means of a further communication device 6 of the motor vehicle 2. In particular, for example, mobile connectivity can be established at least partially between the electronic computing device 4 and a further electronic computing device 7.For example, the further electronic computing device 7 can be provided on the motor vehicle 2, which is connected to the further communication device 6. The further electronic computing device 7 can communicate, for example, with a corresponding environment detection device 8. Furthermore, the motor vehicle 2 can have a corresponding longitudinal and / or lateral acceleration device 9, wherein the further electronic computing device 7 is also connected to the longitudinal and / or lateral acceleration device 9. Furthermore, the motor vehicle 2 has, in particular, a parking assistance system 10, which the further electronic computing device 7 can likewise access.FIG. 2 shows a schematic plan view for an embodiment of the method. In particular, at least one parking space 11 in an environment 12 of the motor vehicle 2 is detected, for example, by means of the environment detection device 8. the detected parking space 11 is transmitted to the teleoperator 3. the parking space 11 is selected by the teleoperator 3 and the automated parking 13 of the motor vehicle 2 into the selected parking space 11 is carried out.In this case, it can be provided, for example, that the detected parking space 11 is displayed as a superimposed symbol 14 for the teleoperator 3. Furthermore, a starting position 15 for automated parking 13 can be proposed to the teleoperator 3. The method can be carried out, for example, in particular in the parking garage as environment 12.Furthermore, it can be provided that a forward parking maneuver and / or a backward parking maneuver is proposed to teleoperator 3. Furthermore, it can be provided that environment 12 is detected, in particular as environment detection device 8, by means of an ultrasonic sensor device or radar sensor device and / or a lidar sensor device and / or camera.In an additional or alternative embodiment, it can also be provided that a parking maneuver 16 from the parking space 11 is proposed to the teleoperator 3. In this case, for example, a direction of unparking from the parking space 11 can be proposed to the teleoperator 3. Furthermore, a takeover position, for example also in the form of the starting position 15, can also be displayed for the teleoperated operation after the parking maneuver 16.A further advantageous embodiment provides that a parking display in motor vehicle 2 is transmitted to teleoperator 3. During the automated parking maneuver 13, an environmental detection of the motor vehicle 2 can be transmitted to the teleoperator 3. Furthermore, in the case of a plurality of parking spaces 11, the teleoperator 3 can be provided with the plurality of parking spaces 11.FIG. 2 shows that automatic parking is in particular a method in which motor vehicle 2 searches for free parking spaces 11 and uses, for example, the camera or the ultrasonic sensor. An automatic parking maneuver with some options can now be proposed to a driver in motor vehicle 2. This is now proposed to the teleoperator 3 instead of the driver himself.According to the prior art, the teleoperator 3 cannot rely on the automatic parking function in the motor vehicle 2 because it only receives perception data, for example video, ultrasound or speed, and sends driving commands such as, for example, acceleration, speed setting and steering. However, parking maneuvers can be carried out all the time from the distance, in particular in teleoperated driving operation, very stressed by the teleoperator 3, since parking 13 in narrow places requires much attention and concentration of the teleoperator 3.In particular, according to the invention, a connection between the teleoperation service and the automatic parking system of the motor vehicle 2 is now proposed. The automatic parking system has a communication channel via which it transmits the detected parking spaces 11 and, if appropriate, receives a selection command. The teleoperator 3 sees this in particular as an overlay in the life video stream or as a parking assistance view of the motor vehicle 2. the teleoperator 3 can now select one of the available parking spaces 11 in order to start a parking maneuver. During the maneuver, the parking module or the parking assistance system 10 assumes control over the lateral and / or longitudinal control until it reaches the target position. The teleoperator 3 can monitor and / or support the process and / or interrupt it in the event of incorrect behavior. As already mentioned, the same assistance system 1 can also be used for re-parking the motor vehicle 2. The teleoperator 3 can select the direction in which he subsequently wishes to drive, for example left or right, and start the parking maneuver 16 remotely.In particular, the invention has the advantage that already established and proven automatic parking assistance systems 10 are used in the motor vehicle 2. The teleoperator 3 only needs to monitor the parking maneuver or unparking maneuver 16 instead of driving it manually. This reduces the workload and concentration burden. The accuracy of the parking 13 can also be increased since an automatic parking assistant acts very precisely during each parking maneuver or unparking maneuver 16 and is not concentrated after a certain time.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedUS 2020 / 19 23 51 A1

[0005] Cited Non-Patent LiteratureSAE J3016" on the corresponding standard in the April 2021

[0037] version

Claims

Method for operating an at least partially automatically operated motor vehicle (2) by a teleoperator (3), having the steps: - detecting at least one parking space (11) in an environment (12) of the motor vehicle (2); - transmitting the detected parking space (11) to the teleoperator (3); - detecting a selection of the parking space (11) by the teleoperator (3); and - automatically parking (13) the motor vehicle (2) into the selected parking space (11).Method according to Claim 1, characterized in that the detected parking space (11) is displayed as a superimposed symbol (14) for the teleoperator (3).Method according to Claim 1 or 2, characterized in that the teleoperator (3) is provided with a starting position (15) for automated parking (13).Method according to one of the preceding claims, characterized in that the method is carried out in a parking garage as environment (12).Method according to one of the preceding claims, characterized in that a forward parking maneuver (13) and / or a backward parking maneuver (13) is proposed to the teleoperator (3).Method according to one of the preceding claims, characterized in that the environment (12) is detected by means of an ultrasonic sensor device and / or a radar sensor device and / or a lidar sensor device and / or a camera.Method according to one of the preceding claims, characterized in that an automated unparking maneuver (16) from the parking space (11) is additionally proposed to the teleoperator (3).Method according to Claim 7, characterized in that a direction of unparking from the parking space (11) is proposed to the teleoperator (3).Method according to one of Claims 7 or 8, characterized in that a takeover position for the teleoperated operation is displayed to the teleoperator (3) after the unparking maneuver (16).Method according to one of the preceding claims, characterized in that a parking display in the motor vehicle (2) is transmitted to the teleoperator (3).Method according to one of the preceding claims, characterized in that an environmental detection of the motor vehicle (2) is transmitted to the teleoperator (3) during the automated parking maneuver (13).Method according to one of the preceding claims, characterized in that, when a multiplicity of parking spaces (11) is detected, the teleoperator (3) is proposed the multiplicity of parking spaces (11).Computer program product having program code means which cause an electronic computing device (4, 7) to carry out a method according to one of Claims 1 to 12 when the program code means are executed by the electronic computing device (4, 7).A computer readable storage medium comprising at least one computer program product according to claim 12.Assistance system (1) for operating an at least partially automatically operated motor vehicle (2) by a teleoperator (3), wherein the assistance system (1) is designed to carry out a method according to one of Claims 1 to 12.

Images