Method for determining whether an object can be driven over by a vehicle without causing a collision, and vehicle

Abstract

A method for determining whether an object (24, 28, 30) can be driven over by a vehicle (10) without causing a collision.

Description

[0001] Description

[0002] title

[0003] Method for determining whether an object can be driven over by a vehicle without collision, and vehicle

[0004] State of the art

[0005] The invention relates to a method for determining whether an object can be driven over by a vehicle without collision, and to a vehicle.

[0006] While a vehicle is performing a parking maneuver, objects in the vicinity of the vehicle are typically detected by means of the vehicle's parking assistance system.

[0007] Disclosure of the invention

[0008] The invention aims to provide a method that determines, particularly early and reliably, whether an object can be driven over by a vehicle without collision. Furthermore, the present invention aims to provide a vehicle that is designed to perform the method.

[0009] The problem underlying the invention is solved by a method with the features of claim 1 and by a vehicle with the features of claim 10. Advantageous embodiments of the invention are specified in the dependent claims.

[0010] A method according to the invention is suitable for determining whether an object can be driven over by a vehicle without collision. The method comprises the following steps: a) generating sensor data from the object by detecting the object using a parking aid of the vehicle while the vehicle is performing a parking maneuver; b) creating a data sequence based on a development of the sensor data; and c) determining, by analyzing the data sequence, whether the object can be driven over by the vehicle without collision.

[0011] Advantageously, by analyzing the data sequence it is possible to determine particularly early and reliably whether the object can be driven over by a vehicle without collision or not.

[0012] Another aspect of the invention is that the development of sensor data during the execution of the parking maneuver can be characteristic of a non-collision-free traversable object. Advantageously, by executing the method, the object can be recognized as a non-collision-free traversable object based on the development of the sensor data before the vehicle's parking aid has fully detected the object, in particular its height.

[0013] A collision-free crossing of the object can be understood as the vehicle driving over the object without being damaged.

[0014] The parking maneuver can be performed to place the vehicle in a parking space. The parking maneuver can be described as parking the vehicle in the parking space.

[0015] The object can define the boundaries of a parking space into which the vehicle is placed by the parking maneuver.

[0016] The object could be, for example, a lamppost, a post, a wall, a curb, a parking barrier, a parking limiter, a street sign, a traffic sign, a wall, a fence, a tree, a flower bed or another vehicle.

[0017] The parking aid can include at least one ultrasonic sensor, particularly for generating sensor data from the object. The object detection in step a) can include detection of the object using the ultrasonic sensor. The data sequence can consist of a sequence of sensor data from the ultrasonic sensor. Advantageously, ultrasonic sensors can be particularly suitable for use as parking aids due to their high sensitivity, ease of use, and low price.

[0018] The generation of sensor data from the object can only occur during the execution of the parking maneuver. For example, sensor data from the object cannot be generated if the vehicle is not performing a parking maneuver.

[0019] Generating sensor data from the object can include generating timestamps. Each timestamp can represent the time at which the object was detected by the parking assist system. The data sequence can also contain these timestamps. The sensor data within the data sequence can be arranged relative to each other based on the timestamps. For example, the sensor data can be arranged in the data sequence with ascending timestamps.

[0020] In a further development of the procedure, the development of the sensor data in step b) is a distance-dependent development and / or a temporal development of the sensor data.

[0021] For example, the data sequence can be supplemented with further sensor data from the object if the distance between the vehicle and the object has decreased or increased by a predefined amount. In other words, the sensor data can be acquired periodically at predefined spatial intervals.

[0022] Additionally or alternatively, the data sequence can be supplemented with further sensor data from the object when a predefined value has elapsed since the last sensor data entry was generated. In other words, sensor data can be acquired periodically at predefined time intervals.

[0023] In a further development of the method, the data sequence is created in step b) by storing the sensor data generated in step a). In a further development of the method, the data sequence is analyzed in step c) by forming correlations between adjacent and / or consecutive sensor data within the data sequence. Advantageously, based on these correlations, it can be determined particularly easily whether the object can be driven over by the vehicle without collision.

[0024] In a further development of the method, the data sequence in step c) is analyzed using a recursive neural network. Advantageously, data sequences of varying lengths can be analyzed using the recursive neural network. The recursive neural network can be flexibly scaled. It can be easily adapted or trained for vehicles with different configurations.

[0025] The recursive neural network (RNN) can be abbreviated as "RNN".

[0026] The recursive neural network can reliably determine whether an object can be driven over by a vehicle without collision, even if sensor data was generated by the object in earlier time periods but not in later ones. This can apply, for example, to collision-prone objects that are lower than the mounting position of the parking aid and therefore move out of its detection range at close range. Such objects could be boxes or toy cars, for example.

[0027] In a further development of the method, the procedure includes the following step before step c): e) Training the recursive neural network with at least one training data sequence. The training data sequence may, for example, contain noise. This can advantageously make the recursive neural network less sensitive to noise in the data sequence. The training data sequence can be a data sequence from a previous parking maneuver. Training the recursive neural network can take place outside the vehicle, for example, in a data center. In a further development of the method, the recursive neural network is implemented as a long short-term memory network. The long short-term memory network can be abbreviated as "LSTM".

[0028] In a further development of the procedure, the determination in step c) includes weighting the sensor data.

[0029] More meaningful sensor data can be weighted more heavily. For example, sensor data with a low signal-to-noise ratio can be weighted more heavily than sensor data with a higher signal-to-noise ratio.

[0030] Analyzing the data sequence in step c) can be done depending on the weighting of the sensor data.

[0031] Preferably, the training of the recursive neural network in step e) can be carried out such that the weighting of the sensor data is performed by the recursive neural network. For example, a loss function and / or a weighting metric can be used for training in step e) that weights the sensor data differently.

[0032] In a further development of the method, the sensor data is weighted based on the distance between the object and the vehicle. Additionally or alternatively, the sensor data is weighted based on a temporal sequence of the sensor data.

[0033] The sensor data can be weighted such that data acquired when the object and the vehicle are closer together are weighted more heavily than data acquired when the object and vehicle are farther away. The distance between the object and the vehicle can then be determined based on the sensor data.

[0034] The sensor data can be weighted such that sensor data generated earlier in time are weighted less than sensor data generated later in time. A vehicle according to the invention, in particular a motor vehicle, is designed to carry out a previously described method.

[0035] Possible embodiments of the invention are explained below with reference to the accompanying drawings. These show:

[0036] Fig. 1 is a schematic top view of a vehicle with a parking aid,

[0037] Figures 2 to 4 each show a schematic top view of the vehicle from Figure 1 during the creation of a training data sequence for training a recursive neural network of the vehicle.

[0038] Fig. 5 is a schematic top view of the vehicle from Fig. 1 during a parking maneuver.

[0039] Fig. 6 shows a schematic model of sensor data and the recursive neural network during the execution of the parking maneuver of Fig. 5, and

[0040] Fig. 7 shows an exemplary sequence of a procedure for determining whether an object can be driven over by a vehicle without collision.

[0041] Fig. 1 shows a vehicle 10 with a parking aid 12 in the form of an ultrasound-based driver assistance system. The parking aid 12 has a control unit 14 and a plurality of ultrasound sensors 16. In the embodiment shown in Fig. 1, eight ultrasound sensors 16 are shown by way of example.

[0042] In an alternative embodiment not shown, the parking aid has twelve ultrasonic sensors, with each bumper of the vehicle having six ultrasonic sensors. Of the six ultrasonic sensors on each bumper, two can be laterally oriented.

[0043] The ultrasonic sensors 16 are connected to the control unit 14 via signal transmission. The ultrasonic sensors 16 are located at the rear of the vehicle 10. The control unit 14 has a memory 18 and a recursive neural network 20. The recursive neural network 20 is designed as a long short-term memory network.

[0044] The parking aid 12 is designed to perform a procedure to determine whether an object can be driven over by a vehicle 10 without collision.

[0045] Figures 2 to 4 each show the vehicle 10 during the creation of a training data sequence for training the recursive neural network 20. In Figure

[0046] For elements 1 to 4, the same reference symbols are used for identical and functionally equivalent elements, therefore reference can be made to the corresponding description.

[0047] Fig. 2 shows that the vehicle 10 is moving backwards along a direction of movement 22 towards a post 24. The vehicle 10 cannot drive over the post 24 without colliding with it. As the vehicle 10 moves towards the post 24, each ultrasonic sensor 16 emits ultrasonic pulses 26, which are reflected by the post 24 as echoes. The ultrasonic sensors 16 convert the echoes into sensor data by detecting them.

[0048] The control unit 14 generates a timestamp for each detected echo. The control unit 14 creates a training data sequence from the post 24 by storing the sensor data and the timestamps in the memory 18 of the control unit 14.

[0049] The training data sequence contains noise that occurs during the creation of the sensor data from post 24.

[0050] The recursive neural network 20 is trained outside the vehicle 10, for example in a data center, using the training data sequence from the post 24. A loss function and / or a weighting metric is used to train the recursive neural network 20, weighting the sensor data based on their timestamps. In the illustrated embodiment, sensor data acquired later are weighted more heavily than sensor data acquired earlier. Fig. 3 shows the vehicle 10 reversing along the direction of travel 22 towards a curb 28. The vehicle 10 can cross the curb 28 without collision. As the vehicle 10 approaches the curb 28, each ultrasonic sensor 16 emits ultrasonic pulses 26, which are reflected by the curb 28 as echoes. The ultrasonic sensors 16 convert these echoes into sensor data by detecting them.

[0051] The control unit 14 creates a training data sequence from the curb 28 by storing the sensor data in the memory 18 of the control unit 14. The recursive neural network 20 is trained outside the vehicle 10, for example in a data center, with the training data sequence from the curb 28.

[0052] Figure 4 shows that vehicle 10 is reversing towards another vehicle 30. Vehicle 10 cannot drive over the other vehicle 30 without a collision. The training of the neural network 20 for the other vehicle 30 is carried out in accordance with the training for post 24, therefore reference is made to the description of the training for post 24.

[0053] After completion of the training, the recursive neural network 20 is trained in such a way that the parking aid 12 can determine, based on a data sequence from an object, whether the object can be driven over by a vehicle 10 without collision.

[0054] After training is complete, the trained recursive neural network 20 is transferred to the vehicle. The recursive neural network previously present in the control unit 14 is replaced by the trained recursive neural network 20.

[0055] Fig. 5 shows the vehicle 10 during the execution of a parking maneuver to place the vehicle 10 in a parking space.

[0056] The parking space is located at the edge of a roadway 32 and is bordered by posts 24 and two other vehicles 30. Parking in the parking space is sidewalk parking. In other words, the vehicle 10 is to be positioned laterally on a sidewalk 34, which is separated from the roadway 32 by a curb 28. To park the vehicle 10 in the parking space, the vehicle 10 must drive over the curb 28.

[0057] To park vehicle 10 in the parking space, vehicle 10 performs the parking maneuver. The parking maneuver involves reversing into the parking space.

[0058] During the execution of the parking maneuver, sensor data from the curb 28, the posts 24, and the other vehicles 30 are generated by the parking aid 12. The generation of this sensor data from the curb 28, the posts 24, and the other vehicles 30 is achieved by the ultrasonic sensors 16 emitting ultrasonic pulses 26 and detecting the echoes of these ultrasonic pulses 26. The control unit 14 generates a timestamp for each detected echo.

[0059] The control unit 14 creates a data sequence by storing the sensor data and the timestamps associated with the sensor data in the memory 18 of the control unit 14. In the data sequence, the sensor data are arranged with ascending timestamps. In other words, the data sequence is a temporal evolution of the sensor data.

[0060] The control unit 14 determines, by analyzing the data sequence using the recursive neural network 20, whether the posts 24, the curb 28 and the other vehicles 30 can be driven over by a vehicle 10 without collision.

[0061] The recursive neural network 20 is trained such that, during the analysis of the data sequence, the sensor data are weighted according to the distance between the vehicle and the curb 28, the distance between the vehicle and the posts 24, and the distance between the vehicle and the other vehicles 30. The sensor data can be weighted by the recursive neural network 20 such that the sensor data of objects 24, 28, 30 located close to vehicle 10 are weighted more heavily than sensor data of objects 24, 28, 30 located farther away from vehicle 10. However, this is not mandatory. Any other pattern in the data sequence can be used to determine whether an object can be driven over by the vehicle without collision. The analysis of the data sequence by the recursive neural network 20 shows that the curb 28 can be driven over by vehicle 10.The analysis of the data sequence by the recursive neural network 20 shows that the posts 24 and the other vehicles 30 cannot be driven over by the vehicle 10.

[0062] After a predetermined time interval, for example, after 0.3 seconds, the ultrasonic sensors emit 16 further ultrasonic pulses 26 and detect their echoes. The control unit 14 generates additional timestamps for each detected echo. The sensor data thus obtained from the curb 28, the posts 24, and the other vehicles 30, along with the timestamps assigned to the sensor data, are added to the existing data sequence. In other words, the data sequence is augmented with further sensor data at regular time intervals during the execution of the parking maneuver.

[0063] After each extension of the data sequence with further sensor data, the data sequence is analyzed by the recursive neural network 20. This ensures that, after each extension of the data sequence with further sensor data, it is determined again whether the posts 24, the curb 28 and the other vehicles 30 can be crossed by the vehicle 10 without collision.

[0064] Vehicle 10 continues the parking maneuver depending on whether it can cross the posts 24, the curb 28, and the other vehicles 30 without collision. Vehicle 10 crosses the curb 28 without colliding with the posts 24 or the other vehicles 30.

[0065] Fig. 6 shows a schematic model of the sensor data 36, ​​the data sequence 38, and the recursive neural network 20 while the vehicle 10 performs the parking maneuver. The sensor data 36 form the data sequence 38. After a predetermined time interval, the data sequence 38 is extended by further sensor data 36. After each extension, the recursive neural network 20 can analyze the data sequence 38. This allows the network to determine after each extension whether the posts 24, the curb 28, and the other vehicles 30 can be traversed by the vehicle 10 without collision. Alternatively, the recursive neural network 20 can update an internal state when the data sequence 38 is extended by further sensor data 36. This allows the recursive neural network 20 to start from this internal state when new sensor data 36 is added, rather than having to process the entire data sequence 38.

[0066] In Fig. 6, the result of the determination after each extension of the data sequence 38 with the reference symbols hi , h2, h is shown. L -i , hi. labelled.

[0067] Fig. 7 shows an exemplary sequence of a procedure for determining whether an object can be driven over by the vehicle 10 without collision.

[0068] The procedure comprises the following steps: e) Training the recursive neural network 20 with at least one training data sequence and with a loss function and / or with an evaluation metric such that sensor data of a data sequence are weighted depending on a distance between the object and the vehicle 10; a) Generating sensor data 36 of the object by detecting the object using ultrasonic sensors 16 of the parking aid 12 of the vehicle 10 while the vehicle 10 performs the parking maneuver, wherein the generation of the sensor data 36 of the object includes generating timestamps; b) Creating a data sequence 38 of a distance-dependent or time-dependent evolution of the sensor data 36 by storing the sensor data 36 generated in step a) and the timestamps; and c) Determining, by analyzing the data sequence 38 using the recursive neural network, whether the object can be driven over by the vehicle 10 without collision.

Claims

Claims 1. Method for determining whether an object (24, 28, 30) is connected to a vehicle (10) is traversable without collision, the method comprising the steps of: a) generating sensor data (36) of the object (24, 28, 30) by detecting the object (24, 28, 30) by means of a parking aid (12) of the vehicle (10) while the vehicle (10) is performing a parking maneuver, b) creating a data sequence (38) based on a development of the sensor data (36), and c) determining by analyzing the data sequence (38) whether the object (24, 28, 30) is traversable without collision by the vehicle (10).

2. Method according to claim 1, wherein the development of the sensor data (36) in step b) is a distance-dependent development and / or a temporal development of the sensor data (36).

3. Method according to one of the preceding claims, wherein the creation of the data sequence (38) in step b) is carried out by storing the sensor data (36) generated in step a).

4. Method according to one of the preceding claims, wherein the analysis of the data sequence (38) in step c) is carried out by forming a correlation of adjacent and / or consecutive sensor data (36) in the data sequence (38).

5. Method according to one of the preceding claims, wherein the analysis of the data sequence (38) in step c) is carried out using a recursive neural network (20).

6. Method according to claim 5, wherein the procedure before step c) includes the step: e) Training the recursive neural network (20) with at least one training data sequence.

7. Method according to claim 5 or 6, wherein the recursive neural network (20) is configured as a long short-term memory network.

8. Method according to one of the preceding claims, wherein the determination in step c) comprises weighting the sensor data (36).

9. Method according to claim 8, wherein the weighting of the sensor data (36) is based on a distance between the object (24, 28, 30) and the vehicle (10), and / or wherein the weighting of the sensor data (36) is based on a temporal sequence of the sensor data (36).

10. Vehicle (10), in particular motor vehicle, which is designed to perform a method according to any of the preceding claims.

Images