Method for monitoring a driving maneuver, test device, computer program and medium

Abstract

The invention relates to a method for monitoring a driving maneuver of a motor vehicle (2) from a starting position to a final position, comprising the steps of: a) providing the motor vehicle (2); b) providing a drone (3) having a camera (5) and a laser (6) and / or a projector; c) launching the drone (3); f) projecting a marker (10) onto the motor vehicle and / or onto the surroundings of the motor vehicle (2) during the driving maneuver using the laser (6) and / or the projector; g) recording a video sequence of the motor vehicle (2) and the marker (10) during the driving maneuver using the camera (5) from above, wherein the video sequence optionally shows a surrounding object (25). The invention also relates to a test device (1), a computer program, and a computer-readable medium.

Description

[0001] The invention relates to a method for monitoring a driving maneuver of a motor vehicle, a test device with means for carrying out the method, a computer program with instructions for carrying out the method, and a computer-readable medium on which the computer program is stored. DE 10 2016 215 301 A1 discloses a method for assisting a parking process of a vehicle. US 2019 / 0 051 169 A1 discloses an unmanned aerial vehicle and related methods and systems. DE 10 2021 200 452 A1 discloses a method and a training system for training a camera-based control system.

[0002] A motor vehicle can be configured to perform autonomous driving maneuvers. The driving maneuver could, for example, be a parking maneuver. Another example could be automatic braking when the distance to a vehicle in front becomes too short. The autonomous driving maneuver must be tested in order to identify and correct sources of error. For this purpose, a test scenario must be set up in which the motor vehicle and surrounding objects, such as parked vehicles, are positioned and in which the motor vehicle performs the driving maneuver. A vehicle start position, a vehicle end position, and the positions of the surrounding objects must be laboriously measured, for example using a measuring tape. The test scenario can also be marked with adhesive tape and / or chalk.However, this is disadvantageous because the markings can be damaged by precipitation or if they are driven over. The maneuver can be recorded with a camera. However, this is disadvantageous because certain situations during the maneuver are obscured by the vehicle and / or surrounding objects, so that the images captured by the camera do not provide all the necessary information about the maneuver.

[0003] The object of the invention is therefore to provide an improved method for monitoring a driving maneuver. Furthermore, the object of the invention is to provide a test device, a computer program, and a computer-readable medium for carrying out the method.

[0004] The method according to the invention for monitoring a driving maneuver of a motor vehicle from a motor vehicle start position to a motor vehicle end position comprises the steps of: a) providing the motor vehicle; b) providing a drone having a camera and a laser and / or a projector; c) letting the drone ascend; f) projecting a marker onto the motor vehicle and / or onto an area surrounding the motor vehicle during the driving maneuver using the laser and / or the projector; g) recording a video sequence of the motor vehicle and the marker during the driving maneuver using the camera from above, wherein the video sequence optionally shows an object in the surroundings.

[0005] Because the camera records the maneuver from above, it provides a very good overview with all relevant information about the maneuver, creating an improved method for monitoring the maneuver. By projecting the marking using the laser and / or projector and recording this marking with the camera, the maneuver can be measured with particularly high accuracy, for example, by measuring distances between the vehicle and the marking, which can further improve the process.

[0006] It is preferred that the driving maneuver be one performed at least partially autonomously by the motor vehicle. The driving maneuver may, for example, be a parking maneuver. Another example of a driving maneuver is a braking maneuver. The driving maneuver may also be one performed completely autonomously by the motor vehicle.

[0007] It is preferred that the method comprises the step: d) projecting a preparatory marking using the laser and / or the projector and arranging the motor vehicle and / or the surrounding object relative to the preparatory marking before the driving maneuver is performed. This allows a test scenario in which the motor vehicle is in the vehicle start position and / or the surrounding object or several of the surrounding objects are arranged to be set up more quickly than with a conventional method in which the position and orientation of the motor vehicle and in particular of the surrounding object or objects are measured in a complex manner, in particular using a measuring tape.

[0008] The method preferably comprises the step: e) transmitting motor vehicle data from the motor vehicle to the drone. This advantageously makes it possible to make the motor vehicle data visible by means of the marking.

[0009] The motor vehicle data preferably includes data about a route planned by the motor vehicle to be traveled during the driving maneuver, and the marker displayed in step f) indicates the planned route. This allows deviations from the planned route and the route actually traveled by the motor vehicle during the driving maneuver to be detected particularly quickly and easily, as well as with a high degree of accuracy.

[0010] It is preferred that the motor vehicle data comprise data about a motor vehicle position planned by the motor vehicle, which is to be assumed by the motor vehicle during the driving maneuver, and that the marking displayed in step f) indicates the planned motor vehicle position. This allows deviations from the planned motor vehicle position and an actual motor vehicle position assumed by the motor vehicle during the driving maneuver to be detected particularly quickly and easily and with a high degree of accuracy. The motor vehicle data can also comprise data about several of the planned motor vehicle positions at different times, wherein the motor vehicle positions at the different times are displayed in step f).

[0011] It is preferred that the motor vehicle data comprise data about a motor vehicle orientation planned by the motor vehicle, which is to be assumed by the motor vehicle during the driving maneuver, and that the marking displayed in step f) indicates the planned motor vehicle orientation. This allows deviations from the planned motor vehicle orientation and a motor vehicle orientation actually assumed by the motor vehicle during the driving maneuver to be detected particularly quickly and easily, as well as with a high degree of accuracy. The motor vehicle data can also comprise data about several of the planned motor vehicle orientations at different times, wherein the motor vehicle orientations at the different times are displayed in step f).

[0012] It is preferred that the motor vehicle data contain data about a position of the surrounding object detected by the motor vehicle during the driving maneuver, and that the marker displayed in step f) indicates the position of the surrounding object. This allows for a particularly quick and easy check of how reliably the sensors used in the motor vehicle detect the position of the surrounding object.

[0013] It is preferred that the motor vehicle data include data about a surface detected by the motor vehicle, wherein the marker 10 displayed in step f) indicates the data about the surface 13. The data about the surface 13 may, for example, include the position of a curb.

[0014] The drone is preferably moved to a predetermined drone position in step c) before the driving maneuver. The predetermined drone position is preferably located above the vehicle's starting position, above a planned vehicle's starting position, or above a planned vehicle's ending position. It is conceivable that the predetermined drone position is determined based on a position of the surrounding object detected by the camera and / or based on a marking other than the marking detected by the camera.

[0015] It is preferred that the drone moves along with the motor vehicle during the maneuver, so that the drone is positioned above the motor vehicle during the maneuver, and in particular throughout the entire maneuver. This provides a particularly good overview with all relevant information about the maneuver. The position of the motor vehicle, based on which the drone moves along with the motor vehicle, can be obtained, for example, from one or more images captured by the camera or can be contained in the motor vehicle data transmitted in step e).

[0016] The test device according to the invention comprises means configured to carry out the method according to the invention or a preferred embodiment of the method according to the invention. The means may include, for example, the motor vehicle and the drone.

[0017] The test device preferably has a control unit configured to control the drone. The control unit can be arranged at a distance from the drone and configured to communicate wirelessly with the drone. The control unit can be a computer, for example. Furthermore, the test device can have a screen configured to display the video sequence and / or individual images of the video sequence.

[0018] The computer program according to the invention comprises instructions that cause the test device according to the invention or a preferred embodiment of the test device to execute the method steps of the method according to the invention or a preferred embodiment of the method. The computer program can comprise information about one test scenario or several different test scenarios, so that the test scenario or the several different test scenarios can be monitored with the computer program.

[0019] The computer program is stored on the computer-readable medium according to the invention.

[0020] The invention is explained in more detail below with reference to the attached schematic drawings. Fig. 1 a test device according to the invention, Fig. 2 a test scenario at a first time and a second time and Fig. 3 the test scenario at the second time point up to a sixth time point.

[0021] Fig. 1 to 3 show a test device 1 according to the invention, which has means configured to carry out a method according to the invention for monitoring a driving maneuver of a motor vehicle 2 from a motor vehicle start position to a motor vehicle end position, wherein the method comprises the steps of: a) providing the motor vehicle 2; b) providing a drone 3 having a camera 5 and a laser 6 and / or a projector; c) launching the drone 3; f) projecting a marking 10 onto the motor vehicle and / or onto an environment of the motor vehicle 2 during the driving maneuver by means of the laser 6 and / or the projector; g) recording a video sequence of the motor vehicle 2 and the marking 10 during the driving maneuver by means of the camera 5 from above, wherein the video sequence optionally shows an environmental object 25.

[0022] The video sequence shows the motor vehicle 2 at least in the motor vehicle start position and in the motor vehicle end position, whereby the video sequence can also show the motor vehicle 2 in positions between the motor vehicle start position and the motor vehicle end position.

[0023] The driving maneuver can be a driving maneuver performed at least partially autonomously by the motor vehicle 2 or a driving maneuver performed completely autonomously by the motor vehicle 2. The driving maneuver can be, for example, a parking maneuver and / or a braking maneuver.

[0024] It is conceivable that the laser is configured to emit laser radiation in the visible wavelength range and / or the infrared wavelength range. It is conceivable that the projector does not have a laser.

[0025] Fig. 1 shows that the drone 3 can have at least one propeller 14, by means of which the drone 3 can be launched. Furthermore, the drone 3 can have a mirror 7 that is pivotably arranged, in particular pivotably about two different axes, which can be arranged perpendicular to one another, for example. The laser 6 can be configured to direct a laser beam 9 onto the mirror 7, and the mirror 7 can be configured to redirect the laser beam 9. By pivoting the mirror 7 around the two axes, the laser beam 9 can be directed at different points on the motor vehicle 2 and / or the surroundings of the motor vehicle 2 by changing the orientation of the mirror 7 by pivoting it around one or both of the axes. In step f), the marking 10 is created at the point where the laser beam 9 strikes a surface. It is conceivable that the laser 6 and the mirror 7 are arranged inside the drone 3.The drone 3 may also have a window 8 through which the laser beam 9 leaves the drone 3 after the laser beam 9 has been deflected by the mirror 7.

[0026] It is conceivable that the drone 3 has a plurality of lasers 6 and / or a plurality of projectors, wherein in particular a respective mirror 7 is provided for each of the lasers 6. It is also conceivable that the drone 3 has a plurality of cameras 5.

[0027] Fig. 2 shows that the method may comprise the step: d) projecting a preparation mark 29 by means of the laser 6 and / or the projector and arranging the motor vehicle 25 relative to the preparation mark 29 before the driving maneuver is carried out. Fig. 2 shows the motor vehicle 2 at a first time 2a and at a time 2b, wherein the motor vehicle 2 has been moved starting from the first time 2a in such a way that at the second time 2b the preparation marking 29, which in Fig. 2 is a line, which may be dashed, coincides with the x-axis of the motor vehicle 2 when viewed from above. In addition or alternatively to arranging the motor vehicle 2 relative to the preparation marking 29, it is conceivable that in step d) the environmental object 25 or several of the environmental objects 25 are arranged relative to the preparation marking 29 before the driving maneuver is carried out. By carrying out step d), a test scenario 20 can be created in which the motor vehicle 2 is arranged in the motor vehicle start position, wherein in Fig. 2 the motor vehicle starting position is present at the second time 2b.

[0028] The method may comprise the step of: e) transmitting motor vehicle data from the motor vehicle 2 to the drone 3. As is apparent from Fig. 1, the motor vehicle 2 can have a motor vehicle communication unit 11, which in particular has a transmitter, and the drone 3 can have a drone communication unit 12, which in particular has a receiver. The motor vehicle communication unit 11 and the drone communication unit 12 can be configured to wirelessly transmit the motor vehicle data in step e). It is conceivable that the motor vehicle communication unit 11 transmits the motor vehicle data directly to the drone communication unit 12. Alternatively, it is conceivable that, as in Fig. 1, the test device 1 has a control unit 4, which can have a transmitter and a receiver and is arranged at a distance from the drone 3 and in particular at a distance from the motor vehicle 2, wherein the control unit 4 receives the motor vehicle data from the motor vehicle communication unit 11 and sends it to the drone communication unit 12. The control unit 4 can also be configured to control the drone 3. The drone communication unit 12 can have a transmitter and send the video sequence to the control unit 4 by means of the transmitter, and the control unit 4 can have a screen configured to display the video sequence.

[0029] Fig. 3 shows that the motor vehicle data may include data about a route 21 planned by the motor vehicle 2, which is to be traveled by the motor vehicle 2 during the driving maneuver, and the marker 10 displayed in step f) indicates the planned route 21. In the video sequence, deviations from the planned route 21 and the route actually traveled by the motor vehicle during the driving maneuver can be particularly easily identified and evaluated. In particular, the control unit 4 can be configured to identify and evaluate the deviations. Fig. The planned route 21 shown in Figure 3 can refer to a specific point on the motor vehicle 2, wherein it is conceivable that the specific point is marked on top of the motor vehicle 2, for example, by means of paint and / or an adhesive strip. This makes it particularly easy to detect deviations between the specific point and the planned route 21 indicated by the marker 10.

[0030] Out of Fig. 3 shows that the motor vehicle data can include data about a motor vehicle position 22 planned by the motor vehicle 2, which is to be assumed by the motor vehicle 2 during the driving maneuver, and the marker 10 displayed in step f) indicates the planned motor vehicle position 22. Furthermore, the motor vehicle data can include data about a motor vehicle orientation planned by the motor vehicle 2, which is to be assumed by the motor vehicle 2 during the driving maneuver, and the marker 10 displayed in step f) indicates the planned motor vehicle orientation. In the video sequence, deviations from the planned motor vehicle position and / or the planned motor vehicle orientation and the motor vehicle position and / or motor vehicle orientation actually assumed by the motor vehicle during the driving maneuver can be particularly easily identified and evaluated.In particular, the control unit 4 can be configured to identify and evaluate the deviations. It is conceivable that a plurality of the planned vehicle positions 22 and the planned vehicle orientations are displayed, corresponding to different points in time during the driving maneuver. Fig. 3, the marking 10 has a front marking 23b to 23f extending from one lateral end to another lateral end of the motor vehicle 2 in its planned motor vehicle position 22 and planned motor vehicle orientation, and a rear marking 24b to 24f arranged at a distance from the front marking 23b to 23f and extending from one lateral end to the other lateral end of the motor vehicle 2 in its planned motor vehicle position 22 and planned motor vehicle orientation. The front marking 23b to 23f and the rear marking 24b to 24f indicate the planned motor vehicle position 22 and the planned motor vehicle orientation.

[0031] In Fig. 3, the motor vehicle 2 is shown at the second time 2b, a third time 2c, a fourth time 2d, a fifth time 2e and a sixth time 2f, wherein the time progresses continuously from the second time 2b to the sixth time 2f.

[0032] The second time point 2b corresponds to the vehicle start position, and the sixth time point 2f corresponds to the vehicle end position. At the second time point 2b, the front marker 23b and the rear marker 24b are projected; at the third time point 2c, the front marker 23c and the rear marker 24c are projected; at the fourth time point 2d, the front marker 23d and the rear marker 24d are projected; at the fifth time point 2e, the front marker 23e and the rear marker 24e are projected; and at the sixth time point 2f, the front marker 23f and the rear marker 24f are projected.

[0033] How it turns out Fig. 2 and Fig. 3, the motor vehicle data may include data about a position 26 of the surrounding object 25, which is detected by the motor vehicle 2 during the driving maneuver, and the marking 10 displayed in step f) indicates the position 26 of the surrounding object 25. In the event that the surrounding object 25 is another motor vehicle, the position 26 may be the outer end of an exterior mirror of the other motor vehicle and / or the front and / or rear of the motor vehicle, cf. Fig. 2 and Fig. 3. The motor vehicle data may also include data on the position of several of the surrounding objects 25.

[0034] It is also conceivable that the motor vehicle data include data about a surface 13 detected by the motor vehicle 2, wherein the marker 10 displayed in step f) indicates the data about the surface 13. The data about the surface 13 may, for example, include the position of a curb.

[0035] It is conceivable that in step c) the drone 3 is moved to a predetermined drone position before the driving maneuver. The predetermined drone position can, for example, be located above the motor vehicle start position, above a planned motor vehicle start position, or above a planned motor vehicle end position, with the latter option being Fig. 2 and Fig. 3. The predetermined drone position can be determined based on a position 26 of the surrounding object 25 and / or of several of the surrounding objects detected by the camera 5, and / or based on a marking different from the marking 10, which is detected by the camera 5. It is also conceivable for the drone 3 to be relocated together with the motor vehicle 2 during the driving maneuver, so that the drone 3 is arranged above the motor vehicle 2 during the driving maneuver, in particular during the entire driving maneuver.

[0036] Fig. 2 and Fig. 3 shows that the laser 6 and / or the projector can project a grid 28 during the driving maneuver. Using the grid 28, the driving maneuver can be measured particularly easily in the video sequence. It is also conceivable that the laser 6 and / or the projector projects a coordinate system 27 during the driving maneuver, compare Fig. 2.

[0037] Using this method, software can be adapted particularly quickly for the at least partially autonomous driving maneuver. List of reference symbols 1 test device 2 motor vehicles 2a Motor vehicle at the first time 2b Motor vehicle at the second time 2c Motor vehicle at the third time 2d motor vehicle at the fourth time 2nd motor vehicle at the fifth time 2f Motor vehicle at the sixth time 3 drone 4 Control unit 5 Camera 6 lasers 7 mirrors 8 windows 9 Laser beam 10 Marking 11 Motor vehicle communication unit 12 drone communication unit 13 Underground 14 propellers 20 test scenario 21 planned route 22 planned vehicle positions 23b front mark at the second time point 23c front mark at the third time point 23d front mark at the fourth time point 23e front mark at the fifth time point 23f front mark at the sixth time point 24b rear mark at the second time point 24c rear mark at the third time point 24d rear mark at the fourth time point 24e rear mark at the fifth time point 24f rear mark at the sixth time point 25 Environment object 26 Position of the surrounding object 27 Coordinate system 28 grids 29 Preparation marking

Claims

[1] Method for monitoring a driving maneuver of a motor vehicle (2) from a motor vehicle start position to a motor vehicle end position, comprising the steps: (a) provision of the motor vehicle (2); b) providing a drone (3) having a camera (5) and a laser (6) and / or a projector; c) letting the drone take off (3); f) projecting a marking (10) onto the motor vehicle (2) and / or onto an environment of the motor vehicle (2) during the driving maneuver by means of the laser (6) and / or the projector; g) Recording a video sequence of the motor vehicle (2) and the marking (10) during the driving maneuver by means of the camera (5) from above, wherein the video sequence optionally shows an environmental object (25). [2] Method according to claim 1, wherein the driving maneuver is a driving maneuver carried out at least partially autonomously by the motor vehicle (2). [3] Method according to claim 1 or 2, comprising the step: d) Projecting a preparatory marking (29) by means of the laser (6) and / or the projector and arranging the motor vehicle (2) and / or the surrounding object (25) relative to the preparatory marking (29) before the driving maneuver is carried out. [4] A method according to any one of claims 1 to 3, wherein the method comprises the step: e) transmitting motor vehicle data from the motor vehicle (2) to the drone (3). [5] Method according to claim 4, wherein the motor vehicle data comprise data about a route (21) planned by the motor vehicle (2) to be covered by the motor vehicle (2) during the driving maneuver, and the marker (10) displayed in step f) indicates the planned route (21). [6] Method according to claim 4 or 5, wherein the motor vehicle data comprise data about a motor vehicle position (22) planned by the motor vehicle (2) to be assumed by the motor vehicle (2) during the driving maneuver, and the marker (10) displayed in step f) indicates the planned motor vehicle position (22). [7] Method according to one of claims 4 to 6, wherein the motor vehicle data comprise data about a motor vehicle orientation planned by the motor vehicle (2) to be assumed by the motor vehicle (2) during the driving maneuver, and the marker (10) displayed in step f) indicates the planned motor vehicle orientation. [8] Method according to one of claims 4 to 7, wherein the motor vehicle data comprise data about a position (26) of the surrounding object (25) which is detected by the motor vehicle (2) during the driving maneuver, and the marking (10) displayed in step f) indicates the position (26) of the surrounding object (25). [9] Method according to one of claims 1 to 8, wherein in step c) the drone (3) is moved to a predetermined drone position before the driving maneuver. [10] The method of claim 9, wherein the predetermined drone position is located above the motor vehicle start position, above a planned motor vehicle start position, or above a planned motor vehicle end position. [11] Method according to claim 9 or 10, wherein the predetermined drone position is determined based on the position (26) of the surrounding object (25) detected by the camera (5) and / or based on a marking different from the marking (10) detected by the camera (5). [12] Method according to one of claims 1 to 11, wherein the drone (3) is displaced together with the motor vehicle (2) during the driving maneuver, so that the drone (3) is arranged above the motor vehicle (2) during the driving maneuver. [13] Test device with means arranged to carry out the method according to one of claims 1 to 12. [14] Test device according to claim 13, wherein the test device (1) comprises a control unit (4) which is arranged to control the drone (3). [15] Computer program comprising instructions which cause the test device (1) according to claim 13 or 14 to carry out the method steps according to one of claims 1 to 12. [16] A computer-readable medium on which the computer program according to claim 15 is stored.

Images