Method for operating a motor vehicle and a mobile device, motor vehicle and mobile device, computer program and storage medium for carrying out the method

Abstract

The invention relates to a method for operating a motor vehicle (1) having a camera system, and a mobile device (3) which is configured to communicate with the motor vehicle (1) and which is to be operated by a user (2) of the motor vehicle (1) who is arranged outside the motor vehicle (1), comprising the steps of: a) starting the method upon an instruction given by the user (2) via the mobile device (3); d) arranging the motor vehicle (1) and the user (2) relative to one another in such a way that a photo of the user (2) can be taken by means of the camera system; e) indicating to the user (2) by means of the mobile device (2) in which orientation the user (2) has to orient themselves relative to the motor vehicle (1) in order to take the photo; f) taking a photo showing the user (2) with the camera system.The invention also relates to the motor vehicle (1) and the mobile device as well as a computer program and a storage medium for carrying out the method.

Description

[0001] With an autonomously driving motor vehicle, there are applications in which a driver of the motor vehicle sits inside the vehicle while the motor vehicle drives autonomously, and applications in which the driver is outside the motor vehicle and the motor vehicle autonomously follows the driver. In applications in which the motor vehicle follows the driver, it is conceivable that the driver carries out a delivery service in which shipping items are delivered to customers. If the motor vehicle follows the driver, it must be ensured that the motor vehicle follows the driver and not pedestrians other than the driver. The motor vehicle should therefore be able to distinguish the driver from pedestrians other than the driver.

[0002] The object of the invention is therefore to provide a method for operating a motor vehicle with which a user of the motor vehicle can be reliably distinguished from pedestrians who are different from the user. Furthermore, the object of the invention is to provide the motor vehicle, a computer program, and a storage medium for implementing the method.

[0003] The method according to the invention for operating a motor vehicle having a camera system and a mobile device which is configured to communicate with the motor vehicle and which is to be operated by a user of the motor vehicle who is arranged outside the motor vehicle, comprises the steps: a) the method is started upon an instruction given by the user via the mobile device; d) arranging the motor vehicle and the user relative to one another in such a way that a photo of the user can be taken by means of the camera system; e) the user is shown by means of the mobile device in which orientation the user has to orient themselves to the motor vehicle in order to take the photo; f) taking a photo showing the user with the camera system.

[0004] A method has been created by means of which the user can be positioned relative to the motor vehicle in such a way that the user is in an optimal position so that the photo can be taken of the user. Based on the photo, the user can then be identified by the camera system in such a way that the risk of the user being confused with pedestrians other than the user is low. By the user starting the method using the mobile device from outside the motor vehicle and by the user being shown the orientation using the mobile device, the method is also more convenient for the user than if the user had to start the method from inside the motor vehicle, for example, and if the user were to be shown inside the motor vehicle where and in what orientation the user would have to position themselves outside the motor vehicle in order to take the photo.

[0005] It is preferred that the mobile device indicate to the user how the user should position themselves relative to the motor vehicle in order for the relative positioning to take place. It is conceivable that the motor vehicle remains stationary while the user moves. It is preferred that in step d), a map with the position of the motor vehicle and the current position of the user is displayed on the mobile device.

[0006] The motor vehicle has a projection device that, in step d), indicates to the user how the user should position themselves relative to the motor vehicle. This can be done in addition to or alternatively to the mobile device indicating to the user how the relative positioning should be carried out. The projection device can, for example, project corresponding information onto the ground. The projection device can, for example, comprise a laser.

[0007] It is conceivable that in step d) the motor vehicle drives autonomously so that the relative positioning takes place. It is conceivable that the mobile device indicates to the user that he should remain stationary while the motor vehicle drives autonomously. It is also conceivable that a simultaneous and / or sequential movement of the motor vehicle and the user has to take place so that the relative positioning takes place in step d). It is preferred that the motor vehicle decides, in particular based on the surroundings of the motor vehicle recorded by the camera system and / or the position of the user, whether the motor vehicle is driving autonomously and the user has to remain stationary, whether the motor vehicle is driving autonomously and the user has to move, or whether the motor vehicle remains stationary and the user has to move.

[0008] It is preferred that the camera system comprises a plurality of cameras, and the method comprises the steps: b) the motor vehicle detects the position of the user outside the motor vehicle; c) the motor vehicle selects, based on the position, which of the cameras is to be used to take a photo of the user, wherein in step d), the relative arrangement is carried out such that a photo of the user can be taken using the selected camera, and in step f), the photo is taken using the selected camera. This allows the user's walking distances and / or the motor vehicle's travel distances to be shortened in order to carry out the relative arrangement in step d).

[0009] One of the cameras is preferably a front camera which is configured to record a front region of an environment of the motor vehicle, wherein the front region is arranged in the forward direction of travel of the motor vehicle from the motor vehicle, and / or wherein one of the cameras is preferably a rear camera which is configured to record a rear region of an environment of the motor vehicle, wherein the rear region is arranged in the reverse direction of travel of the motor vehicle from the motor vehicle, and / or wherein one of the cameras is preferably a side camera which is configured to record a lateral region of an environment of the motor vehicle, wherein the lateral region is arranged to the side of the motor vehicle.

[0010] It is preferred that the method comprises the step: g) repeating steps d), e) and f), wherein in step g) the relative arrangement is different from the relative arrangement in step d) and in step g) the orientation is different from or the same as the orientation in step e). This makes it possible to take another photo of the user. Based on the additional photo, the user can be identified even more reliably, so that the risk of confusing the user with pedestrians other than the user can be further reduced. It is also conceivable that step g) is repeated one or more times, each time a different relative arrangement and optionally a different orientation taking place.

[0011] It is preferred that the method comprises the step: h) repeating steps e) and f), wherein in step h) the orientation is different from the orientation in step e). For example, a photo of the user can be taken with their face turned toward the motor vehicle, and another photo can be taken with the user facing away from the motor vehicle. It is also conceivable that step h) is repeated one or more times, with a different orientation each time.

[0012] In step f), the user is preferably informed via the mobile device that the photo has been taken.

[0013] The method preferably comprises the step: i) the motor vehicle identifies the user with the camera system based on the photo. It is preferred that the method comprises the step: j) the motor vehicle autonomously follows the user identified in step i).

[0014] The invention is explained in more detail below with reference to the attached schematic drawings. They show: Fig. 1 a view from above of a motor vehicle and a user in a first situation, Fig. 2 the view from Fig. 1 in a second situation, Fig. 3 the view from Fig. 1 in a third situation at a first point in time and Fig. 4 the view from Fig. 3 at a second point in time.

[0015] How it turns out Fig. As can be seen from Figures 1 to 4, a system 10 comprises a motor vehicle 1 and a mobile device 3. The motor vehicle 1 has a camera system configured to record the surroundings of the motor vehicle 1. The motor vehicle 1 and the mobile device 3 are configured to communicate with each other, in particular via Bluetooth. The figures show a user 2 of the motor vehicle 1, who is arranged outside the motor vehicle 1 and has to operate the mobile device 3. The figures illustrate that the mobile device 3 can be portable in one hand. In particular, the mobile device 3 can be a smartphone.The system 10 is configured to perform the following steps: a) the method is started upon instruction given by user 2 via mobile device 3; d) arranging motor vehicle 1 and user 2 relative to one another such that a photo of user 2 can be taken using the camera system; e) the mobile device 2 indicates to user 2 the orientation in which user 2 should orient themselves to motor vehicle 1 in order to take the photo; f) taking a photo showing user 2 using the camera system. In step f), the mobile device 3 can indicate to user 2 that the photo has been taken. In step f), the photo can be taken automatically if motor vehicle 1 detects that the relative arranging has taken place in section d) and, in particular, motor vehicle 1 detects that user 2 has adopted the orientation of step e).Alternatively, the photo may be taken in step f) if the user 2 gives an instruction to take the photo using the mobile device 3.

[0016] The orientation can be, for example, such that a photo of user 2 is taken that shows the front, the back or a side of user 2. In Fig. 4 shows that the user 2 was informed that the back of the user 2 should be turned towards the motor vehicle 1.

[0017] The camera system can have a plurality of cameras, wherein it is conceivable that each of the cameras has a different field of view. In particular, it is conceivable that the camera system is configured to capture the surroundings of the motor vehicle 1 completely around a motor vehicle vertical axis (y-axis). The method can comprise the steps: b) the motor vehicle 1 detects the position of the user 2 outside the motor vehicle 1; c) the motor vehicle 1 selects, based on the position, which of the cameras is to be used to take a photo of the user 2, wherein in step d) the relative arrangement is carried out in such a way that a photo of the user 2 can be taken using the selected camera and in step f) the photo is taken using the selected camera. For example, one of the cameras can be a front camera, one of the cameras can be a rear camera, and one of the cameras can be a side camera.Two side cameras can also be provided, which are configured to record the surroundings on sides of the motor vehicle 1 facing away from one another. The front camera is configured to record a front region 4 of the surroundings of the motor vehicle 1, wherein the front region 4 is arranged in the forward direction of travel of the motor vehicle 1 from the motor vehicle 1. The rear camera is configured to record a rear region 5 of the surroundings of the motor vehicle 1, wherein the rear region 5 is arranged in the reverse direction of travel of the motor vehicle 1 from the motor vehicle 1. The side camera is configured to record a lateral region of the surroundings of the motor vehicle, wherein the lateral region is arranged to the side of the motor vehicle. Fig. 1 and Fig. 2 show a situation in which the user 2 is positioned in the forward direction of the motor vehicle 1. The motor vehicle 1 selects the front camera in step c). Fig. 3 and Fig. 4 shows that user 2 is positioned to the side of motor vehicle 1. Motor vehicle 1 selects the side camera in step c).

[0018] In step b), the motor vehicle 1 can detect the surroundings of the motor vehicle 1 and decide based on the surroundings and the position of the user 1 whether, in a first case, the user 2 has to move and the motor vehicle 1 remains stationary, whether, in a second case, the user 2 and the motor vehicle 1 have to move, or whether, in a third case, the user 2 has to remain stationary and the motor vehicle 1 drives autonomously.

[0019] In the first and second cases, in step d), the user 2 can be shown by means of the mobile device 3 how the user 2 should position themselves relative to the motor vehicle 1 so that the relative positioning takes place. For this purpose, a user position 11 can be displayed to the user 2 (see FIG. Fig. 1, Fig. 3 and Fig. 4) to which the user 2 must move. For example, it is conceivable that a map with the position of the motor vehicle 1, a current position of the user 2, and the user position 11 is displayed on the mobile device 3. Alternatively or additionally, the motor vehicle 1 can have a projection device which, in step d), shows the user 2 how the user 2 should position themselves relative to the motor vehicle 1. The projection device can project the user position 11 onto the ground. The projection device can, for example, have a laser.

[0020] In the second and third cases, in step d), the motor vehicle 1 can drive autonomously so that the relative positioning takes place. In the third case, the user 2 can be prompted to stop by means of the mobile device 3. The third case is in Fig. 2, in which a travel path 7 is also shown, by means of which the motor vehicle 1 moves from a motor vehicle starting position 13 into a motor vehicle position 14, so that the relative arrangement takes place in step d).

[0021] How it turns out Fig. 1 and Fig. 2, the method may comprise the step: g) repeating steps d), e) and f), wherein in step g) the relative positioning is different from the relative positioning in step d) and in step g) the orientation is different from or equal to the orientation in step e). In the first case in Fig. 1, another user position 12 is displayed to user 2, which is different from user position 11, after user position 11 has been displayed and the photo has been taken. After user 2 has moved to the other user position 12, another photo is taken. In the third case in Fig.2, after the photo has been taken, the motor vehicle 1 moves from the motor vehicle position 14 to the further motor vehicle position 15, which is different from the motor vehicle position 14. Another photo is taken in the further motor vehicle position 15. For example, a full-body photo of the user 2 can be taken in the user position 11 or in the motor vehicle position 14, and a section of the user 2 can be taken in the further user position 12 or in the further motor vehicle position 15, wherein the section can show, in particular, the face of the user 2. It is also conceivable that step g) is repeated several times, with a different photo being taken each time.

[0022] It is conceivable that the method comprises the step: h) repeating steps e) and f), wherein in step h) the orientation is different from the orientation in step e). The relative arrangement in step d) and the relative arrangement in step h) can be identical.

[0023] The method may comprise the step: i) the motor vehicle 1 identifies the user 2 with the camera system based on the photo, or based on the photo and the further photo, or based on the photo and several further photos. Furthermore, the method may comprise the step: j) the motor vehicle 1 autonomously follows the user 2 identified in step i).

[0024] The system 10 and the method performed by the system 10 are particularly suitable when the user 2 performs a delivery service.

[0025] A computer program includes instructions that, when executed by system 10, cause system 10 to perform the method. The computer-readable storage medium includes instructions that, when executed by system 10, cause system 10 to perform the method. List of reference symbols 1 motor vehicle 2 users 3 mobile device 4 front area 5 rear area 6 side area 7 Driveway 10 systems 11 User position 12 additional user positions 13 Vehicle starting position 14 Motor vehicle position 15 additional motor vehicle positions

Claims

[1] Method for operating a motor vehicle (1) having a camera system and a mobile device (3) which is configured to communicate with the motor vehicle (1) and which is to be operated by a user (2) of the motor vehicle (1) who is arranged outside the motor vehicle (1), comprising the steps: a) the procedure is started upon an instruction given by the user (2) via the mobile device (3); d) arranging the motor vehicle (1) and the user (2) relative to each other in such a way that a photo of the user (2) can be taken by means of the camera system; e) the user (2) is shown by means of the mobile device (2) in which orientation the user (2) has to orient himself to the motor vehicle (1) in order to take the photo; f) Taking a photo showing the user (2) with the camera system. [2] Method according to claim 1, wherein in step d) the user (2) is shown by means of the mobile device (3) how the user (2) has to position himself relative to the motor vehicle (1) so that the relative arrangement takes place. [3] Method according to claim 2, wherein in step d) a map with the position of the motor vehicle (1) and a current position of the user (2) is displayed on the mobile device (3). [4] Method according to one of claims 1 to 3, wherein the motor vehicle (1) has a projection device which, in step d), indicates to the user (2) how the user (2) has to position himself relative to the motor vehicle (1). [5] A method according to claim 4, wherein the projection device comprises a laser. [6] Method according to one of claims 1 to 5, wherein in step d) the motor vehicle (1) drives autonomously so that the relative positioning takes place. [7] Method according to one of claims 1 to 6, wherein the camera system comprises a plurality of cameras, comprising the steps of: b) the motor vehicle (1) detects the position of the user (2) outside the motor vehicle (1); c) the motor vehicle (1) selects, based on the position, by means of which of the cameras a photo of the user (2) is to be taken, wherein in step d) the relative arrangement is carried out in such a way that a photo of the user (2) can be taken by means of the selected camera and in step f) the photo is taken by means of the selected camera. [8] Method according to claim 7, wherein one of the cameras is a front camera configured to record a front region (4) of an environment of the motor vehicle (1), wherein the front region (4) is arranged in the forward direction of travel of the motor vehicle (1) from the motor vehicle (1), and / or wherein one of the cameras is a rear camera configured to record a rear region (5) of an environment of the motor vehicle (1), wherein the rear region (5) is arranged in the reverse direction of travel of the motor vehicle (1) from the motor vehicle (1), and / or wherein one of the cameras is a side camera configured to record a lateral region (6) of an environment of the motor vehicle (1), wherein the lateral region (6) is arranged to the side of the motor vehicle (1). [9] Method according to one of claims 1 to 8, comprising the step: g) repeating steps d), e) and f), wherein in step g) the relative arrangement is different from the relative arrangement in step d) and in step g) the orientation is different from or equal to the orientation in step e). [10] Method according to one of claims 1 to 9, comprising the step: h) repeating steps e) and f), wherein in step h) the orientation is different from the orientation in step e). [11] Method according to one of claims 1 to 10, wherein in step f) the user (2) is informed by means of the mobile device (3) that the photo has been taken. [12] Method according to one of claims 1 to 11, comprising the step: i) the motor vehicle (1) identifies the user (2) with the camera system based on the photo. [13] Method according to claim 12, comprising the step: j) the motor vehicle (1) follows the user (2) identified in step i) autonomously. [14] Motor vehicle (1) and mobile device (3) which are arranged together to carry out a method according to one of claims 1 to 13. [15] Computer program comprising instructions which, when the program is executed by a motor vehicle (1) and a mobile device (3), cause them to carry out the method according to one of claims 1 to 13. [16] Computer-readable storage medium comprising instructions which, when executed by a motor vehicle (1) and a mobile device (3), cause them to carry out the method according to one of claims 1 to 13.

Images