Method and device for assisting a driver in a vehicle

Abstract

The invention relates to a method for assisting a driver in a vehicle (50), wherein a driven trajectory (20) of the vehicle (50) is recorded on the basis of detected sensor data (10), wherein a repetitive trajectory segment (21) in the recorded driven trajectory (20) is detected, and wherein at least one assistance function (30) is provided after the repetitive trajectory segment (21) has been detected. The invention further relates to a device (1) for assisting a driver of a vehicle (50) and to a vehicle (50) equipped therewith.

Description

[0001] Description

[0002] Method and device for assisting a driver in a vehicle

[0003] The invention relates to a method and a device for assisting a driver in a vehicle.

[0004] Particularly in urban areas, drivers face challenging situations. This includes, in particular, driving through old and therefore usually narrow parking garages. These have a repetitive structure, meaning the individual levels are very similar to one another. For example, changing levels in the parking garage often requires the same maneuver (driving down the spiral ramp, driving up the ramp at the same location but on a different level, etc.).

[0005] The invention is based on the object of creating a method and a device for assisting a driver in a vehicle, with which in particular the situations mentioned at the outset can be mastered in an improved manner.

[0006] The object is achieved according to the invention by a method having the features of patent claim 1 and a device having the features of patent claim 9. Advantageous embodiments of the invention emerge from the subclaims.

[0007] In particular, a method for assisting a driver in a vehicle is provided, wherein a driven trajectory of the vehicle is recorded by means of detected sensor data, wherein a repetitive trajectory section is detected in the recorded driven trajectory, and wherein at least one assistance function is provided after detection of the repetitive trajectory section.

[0008] Furthermore, in particular, a device for assisting a driver of a vehicle is provided, comprising a data processing device, wherein the data processing device is configured to receive detected sensor data, to record a driven trajectory of the vehicle based on the detected sensor data, to detect a repetitive trajectory section in the recorded driven trajectory and, after detecting the repetitive trajectory section, to provide at least one assistance function and / or to initiate the provision of at least one assistance function.

[0009] The method and device enable a driver to receive targeted assistance when a repetitive trajectory section occurs. This can, in particular, support and simplify vehicle guidance in repetitive scenarios (spindle travel in a parking garage, changing levels in a parking garage, searching for a parking space in urban scenarios, e.g., driving around the block, etc.). This is achieved by recording a trajectory driven by the vehicle using acquired sensor data. A repetitive trajectory section is detected in this recorded trajectory. If such a repetitive trajectory section is detected, at least one assistance function is provided and / or its provision is initiated.

[0010] A repetitive trajectory section is, in particular, a section of the vehicle's trajectory that repeats itself, i.e., in particular, a set of temporally and / or spatially ordered sensor data (comprising, for example, x- and y-coordinates and the associated vehicle orientations) that has already occurred in the same way once or several times in the trajectory being driven. If a height coordinate (e.g., z-coordinate) is neglected, the same circle is always traversed during a spindle drive in a parking garage, i.e., the same pairs of x- and y-coordinates are always traversed one after the other (neglecting deviations within a specified tolerance).

[0011] The sensor data is collected using a sensor system configured for this purpose. The sensor system may, for example, include one or more of the following sensors: a vehicle odometry sensor, an environmental sensor such as a camera, a lidar sensor, a radar sensor, an ultrasonic sensor, etc.

[0012] Parts of the device, in particular the data processing unit, can be implemented individually or collectively as a combination of hardware and software, for example, as program code executed on a microcontroller or microprocessor. However, it can also be provided that parts are implemented individually or collectively as an application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA). A vehicle is, in particular, a motor vehicle. In principle, however, the vehicle can also be another land, water, air, or space vehicle, for example, a drone or an air taxi.

[0013] It can be provided that the recorded trajectory only extends back a predetermined period of time and / or a predetermined distance into the past. This allows the time period and / or distance considered for locating and / or recognizing a repetitive trajectory section to be limited to a predetermined duration or a predetermined distance.

[0014] In one embodiment, it is provided that a future trajectory is estimated based on the detected repetitive trajectory section, wherein the future trajectory is displayed on a display device as an assistance function. This can indicate to the driver that a repetitive trajectory section has been detected and that an estimated future trajectory is available. In the simplest case, the estimated future trajectory can correspond to the repetitive trajectory section. In particular, for example, identical coordinates and a respective associated orientation of the vehicle can be taken into account. In particular, x- and y-coordinates (where a z-coordinate is omitted as a height specification) and a yaw angle assigned to them can be taken into account for aligning the vehicle to the respective successive x- and y-coordinate pairs.

[0015] In one embodiment, it is provided that a future trajectory is estimated based on the detected repetitive trajectory section, wherein, as an assistance function, an actuator of the vehicle is controlled such that the estimated future trajectory is executed automatically and / or assisted by the vehicle. This allows the estimated future trajectory to be followed partially or automatically, thus supporting the driver. If the estimated future trajectory is followed automatically, the driver no longer needs to steer the vehicle themselves. This can increase comfort. The estimation of the future trajectory has already been described above for a simple case.

[0016] In a further embodiment, the estimated future trajectory is checked for collision freedom and, if it is not, is corrected. This allows for a changed environment to be taken into account. For this purpose, the sensors record the current surroundings of the vehicle while traveling along the estimated future trajectory. Otherwise, the checking and correction of the estimated future trajectory is performed using conventional methods.

[0017] In one embodiment, it is provided that a repetitive trajectory section is detected if, in the recorded trajectory, a match between a position and an orientation of the vehicle and a previous position and a previous orientation of the vehicle in the recorded trajectory is determined. This is a particularly simple embodiment to implement, which requires only a small amount of sensor data. For example, sensor data from an odometry of the vehicle can be evaluated for this purpose. In particular, x and y coordinates and a yaw angle are taken into account for the orientation of the vehicle. If, after an occurrence of a combination of x and y coordinates and yaw angle, this combination occurs again in the sensor data, it is concluded that the vehicle has traveled a repetitive trajectory section, since a starting pose (i.e.A starting position and starting orientation) and an end pose are the same for this trajectory segment. The successive combinations of x- and y-coordinates and the corresponding yaw angles in between are then combined into a (repetitive) trajectory segment or are recognized as such. In principle, a z-coordinate and a pitch and / or roll angle can also be considered.

[0018] In one embodiment, the detection of the repetitive trajectory section comprises a pattern comparison with sections on the recorded driven trajectory. This allows more complex patterns in sections on the driven trajectory to be used to detect repetitive trajectory sections. It can be provided that patterns for the pattern comparison are stored, for example, in a memory of the data processing device. Such patterns can, for example, correspond to patterns for specific scenarios (spindle travel in parking garages, level changes in parking garages, etc.) that are already stored in the vehicle at the factory or provided in another way. The patterns are detected in particular based on the coordinates observed for multiple points in time or within predetermined time windows and the respective associated orientations.

[0019] In one embodiment, the detection of a repetitive trajectory section includes at least a section-by-section geometric analysis of the recorded trajectory. Examples of geometric patterns that can be detected are circles (e.g., for a spindle drive), rectangles, or rectangular shapes (for a change of level in a parking garage or driving around the block).

[0020] In one embodiment, the recognition of a repetitive trajectory section comprises recognition by means of a trained machine learning method. This makes it possible, in particular, to recognize more complex patterns in the driven trajectory. The trained machine learning method is, for example, a trained neural network. The trained machine learning method, in particular the trained neural network, is or will be trained in particular to recognize certain predetermined repetitive trajectory sections. The training data comprise, for example, time series of coordinates and associated orientations of the repetitive trajectory sections and a corresponding ground truth for the unique identification of the repetitive trajectory section.In the inference phase, the trained machine learning method, in particular the trained neural network, can then estimate the presence of one of the repetitive trajectory sections specified during training based on time series of the recorded sensor data (in particular for coordinates and a respective associated orientation of the vehicle).

[0021] Further features of the device design will become apparent from the description of embodiments of the method. The advantages of the device are the same as those of the embodiments of the method.

[0022] Furthermore, a vehicle is also provided, comprising at least one device according to one of the described embodiments.

[0023] The invention will be explained in more detail below using preferred embodiments with reference to the figures.

[0024] Fig. 1 is a schematic representation of an embodiment of the device for assisting a driver of a vehicle;

[0025] Fig. 2a, 2b are schematic representations of a "level change in a parking garage" scenario to illustrate embodiments of the method and the device. Fig. 1 shows a schematic representation of an embodiment of the device 1 for assisting a driver of a vehicle 50. The device 1 comprises a data processing device 2. The data processing device 2 comprises a computing device 3 and a memory 4. The device 1 is arranged in particular in a vehicle 50, in particular a motor vehicle. The device 1 carries out the method described in this disclosure. For this purpose, it is provided in particular that corresponding program code for carrying out measures of the method is executed by the computing device 2.

[0026] The data processing device 2 is configured to receive acquired sensor data 10 and, based on the acquired sensor data 10, to record a driven trajectory 20 of the vehicle 50. The recorded driven trajectory 20 is stored in the memory 4. The sensor data 10 is acquired in particular by means of a sensor system 51 of the vehicle 50. The sensor system 51 can, for example, comprise one or more of the following sensors: an odometry sensor of the vehicle 50, an environmental sensor, such as a camera, a lidar sensor, a radar sensor, an ultrasonic sensor, etc.

[0027] The data processing device 2 detects a repetitive trajectory section 21 in the recorded driven trajectory 20. After detecting the repetitive trajectory section 21, the data processing device 2 provides at least one assistance function 30 and / or causes the provision of at least one assistance function 30, for example by generating and outputting control signals 31 and / or control commands 32.

[0028] It can be provided that a future trajectory 22 is estimated based on the detected repetitive trajectory section 21, wherein the future trajectory 22 is displayed on a display device 52 as an assistance function 30. In the simplest case, the future trajectory 22 corresponds to the detected repetitive trajectory section 21.

[0029] It can be provided that a future trajectory 22 is estimated based on the detected repetitive trajectory section 21, wherein as an assistance function 30 an actuator 53 of the vehicle 50, in particular for a longitudinal control and a lateral control, is controlled in such a way that the estimated future trajectory 22 is executed automatically and / or assisted by the vehicle 50. For this purpose, the estimated future

[0030] Trajectory 22 is fed to the actuator 53. It can be provided that the actuator 53 or a controller of the actuator 53 includes a trajectory planner that plans a concrete execution of the estimated future trajectory 22. The estimation is carried out in particular by means of the data processing device 2.

[0031] In a further development, it can be provided that the estimated future trajectory 22 is checked for collision freedom and, if it is not, corrected. For example, while traveling along the estimated future trajectory 22, currently acquired sensor data 10 can be examined for obstacles in the vicinity of the vehicle 50, with the trajectory 22 being corrected if an obstacle is detected. The checking and correction is performed in particular by means of the data processing device 2, but can in principle also be performed by a vehicle control system and / or a trajectory planner of the vehicle 50.

[0032] It can be provided that a repetitive trajectory section 21 is detected if, in the recorded driven trajectory 20, a match of a position and an orientation of the vehicle 50 with a previous position and a previous orientation of the vehicle 50 in the recorded driven trajectory 20 is determined.

[0033] This is illustrated schematically in Figures 2a and 2b. Figure 2a shows a level change in a parking garage 60, in which the vehicle (no reference number) changes to a higher level or half-level. The trajectory 20 of the vehicle is essentially rectangular. Figure 2b illustrates (in schematically simplified form) such a (rectangular) trajectory 20 in x and y coordinates. A vehicle position and an orientation (indicated by the arrow) are schematically shown for two times t1, t2. The positions and orientations of the vehicle 50 are the same at both times t1, t2.The intermediate trajectory section 21 is then recognized as a repetitive trajectory section 21 on the basis of the recurring combination(s) of x and y coordinates and associated orientations, since the positions and orientations in the area of ​​the dashed section of the trajectory 20 are already traversed for the second time.

[0034] It can be provided that the recognition of the repetitive trajectory section 21 includes a pattern comparison with sections on the recorded driven trajectory 20. For this purpose, the recorded driven trajectory 20 is compared, for example, section by section with known patterns (e.g., for a spindle travel or a level change in a parking garage). The pattern comparison is carried out by means of the data processing device 2. Known patterns are stored, for example, as pattern trajectories (comprising sequences of coordinates and associated orientations) in the memory 4 and are retrieved from it for comparison or comparison.

[0035] It can be provided that the detection of a repetitive trajectory section 21 comprises at least a section-by-section geometric analysis of the recorded trajectory 20. For example, the recorded trajectory 20 can be compared with circles (for spindle travel) and / or rectangles (for a change of level) or other, even more complex, geometric structures (e.g., semicircles with straight lines or rectangles in between, etc. for more complex parking garages). In the examples shown in Figures 2a and 2b, a comparison with a rectangle of a certain size would result in a match, so that a repetitive trajectory section 21 would be detected.

[0036] It can be provided that the detection of a repetitive trajectory section 21 comprises detection by means of a trained machine learning method 5 (Fig. 1). The trained machine learning method 5, in particular a trained neural network, is provided by the computing device 5, wherein associated structural information and parameters of the trained machine learning method 5 can be stored in the memory 4. In the example shown in Fig. 2b, the trained machine learning method 5 would then detect the repetitive trajectory section 21 in the trajectory 20 and, for example, estimate a high probability value for the shown repetitive trajectory section 21 as an output.

[0037] With the aid of the method and the device 1, comfort and safety can be increased when driving through repetitive scenarios.

[0038] List of reference symbols

[0039] device

[0040] Data processing facility

[0041] computing device

[0042] Memory trained machine learning method acquired sensor data driven trajectory repetitive trajectory section future trajectory

[0043] Assistance function

[0044] control signal

[0045] control command

[0046] vehicle

[0047] Sensor technology

[0048] Display device

[0049] Actuators

[0050] Parking garage

[0051] time

[0052] time

Claims

Claims: A method for assisting a driver in a vehicle (50), wherein a driven trajectory (20) of the vehicle (50) is recorded using acquired sensor data (10), wherein a repetitive trajectory segment (21) in the recorded driven trajectory (20) is detected, and wherein at least one assistance function (30) is provided after detection of the repetitive trajectory segment (21). The method according to claim 1, characterized in that a future trajectory (22) is estimated starting from the detected repetitive trajectory segment (21), wherein the future trajectory (22) is displayed on a display device (52) as the assistance function (30).A method according to claim 1 or 2, characterized in that a future trajectory (22) is estimated starting from the detected repetitive trajectory segment (21), wherein, as an assistance function (30), an actuator (53) of the vehicle (50) is controlled such that the estimated future trajectory (22) is executed automatically and / or with assistance by the vehicle (50). A method according to claim 2 or 3, characterized in that the estimated future trajectory (22) is checked for collision-free operation and corrected if it is not collision-free.A method according to any of the preceding claims, characterized in that a repetitive trajectory segment (21) is detected when a match between a position and orientation of the vehicle (50) and a previous position and orientation of the vehicle (50) in the recorded trajectory (20) is found. A method according to any of the preceding claims, characterized in that the detection of the repetitive trajectory segment (21) comprises a pattern comparison with segments on the recorded trajectory (20). A method according to any of the preceding claims, characterized in that the detection of a repetitive trajectory segment (21) comprises at least a section-by-section geometric analysis of the recorded driven trajectory (20). A method according to any of the preceding claims, characterized in that the detection of a repetitive trajectory segment (21) comprises detection by means of a trained machine learning method (5).Device (1) for assisting a driver of a vehicle (50), comprising: a data processing device (2), wherein the data processing device (2) is configured to receive acquired sensor data (10), to record a driven trajectory (20) of the vehicle (50) based on the acquired sensor data (10), to recognize a repetitive trajectory segment (21) in the recorded driven trajectory (20), and, after recognizing the repetitive trajectory segment (21), to provide at least one assistance function (30) and / or to cause the provision of at least one assistance function (30). Vehicle (50), comprising at least one device (1) according to claim 9.

Images