Method and device for controlling the operation of a vehicle

Abstract

The invention relates to a method for controlling the operation of a vehicle (100) designed for at least partially automated driving, comprising: reading environmental data (103) from at least one environmental sensor (101) of the vehicle (100) and driving data (107) from at least one driving data provisioning device (105) of the vehicle (100), wherein the environmental data (103) represent at least detected free spaces between the vehicle (100) and at least one detected other vehicle, wherein the free spaces are defined separately for a current lane driven by the vehicle (100) and for lanes adjacent to the current lane; feeding a machine learning-trained model (M) with the driving data (107) to determine a trained, driver-specific positioning (115) of the vehicle (100) relative to the free spaces;Performing a comparison of the trained, driver-specific positioning (115) with a current positioning of the vehicle (100) relative to the detected free spaces, determined using the environmental data (103), in order to determine a deviation (117); and generating a control signal (119) to control the operation of the vehicle (100) in order to minimize the determined deviation (117) in the at least partially automated driving operation by means of at least one device (109) for longitudinal and / or lateral guidance of the vehicle (100).

Description

State of the art

[0001] The invention relates to a device or a method according to the preamble of the independent claims. The present invention also relates to a computer program.

[0002] Advanced Driver Assistance Systems (ADAS) are electronic add-ons in motor vehicles designed to support the driver in certain driving situations. Safety aspects are often paramount, but increasing driving comfort is also a key focus. Improved fuel efficiency is another important aspect. Ideally, ADAS should "replace" the driver in certain situations, meaning they should behave like the driver themselves. In reality, however, their behavior can vary significantly from that of a manually controlled vehicle. For example, depending on the driver's style, the system might actually drive closer to the vehicle ahead when following another, accelerate faster if a vehicle ahead swerves to the right, or do the exact opposite.

[0003] DE 10 2021 210 610 A1 concerns an approach to performing an automatic driving maneuver which also takes into account a recommendation to change lanes. Disclosure of the invention

[0004] Against this background, the approach presented here comprises a method, a device that uses this method, and finally a corresponding computer program according to the main claims. Advantageous further developments and improvements of the device specified in the independent claim are possible through the measures listed in the dependent claims.

[0005] According to embodiments, a learned, driver-specific free space utilization can be used for adapted driver assistance systems. In particular, a driver-specific positioning of the ego-vehicle can be learned based on available, drivable, and predicted free spaces with respect to an ego-lane and the adjacent lane(s) and associated static and dynamic model parameters, such as environmental conditions, traffic rules, regulations, ego-speed, etc., as well as their changes, and then applied in assisted, at least partially automated, or autonomous driving mode.

[0006] This approach offers the advantage of enabling driver assistance systems and functions to behave in a manner similar to, or at least closely resemble, that of the driver. This can increase the acceptance of such systems, as they can be more precisely tailored to individual driver behavior within permissible and safe parameters. This, in turn, can generate greater customer interest and, consequently, a greater willingness to use ADAS functions. By personalizing the driver experience of these systems, ADAS, and autonomous driving functions, not only can user acceptance be increased, but driver takeover can also be reduced, as such a system behaves very much like a human driver.

[0007] A method for controlling the operation of a vehicle designed for at least partially automated driving is presented, the method comprising the following steps: Reading environmental data via an interface from at least one environmental sensor of the vehicle and driving data via an interface from at least one driving data provisioning device of the vehicle, wherein the environmental data represent at least detected free spaces between the vehicle and at least one detected foreign vehicle in an environment of the vehicle, wherein the free spaces are defined separately for a current lane traveled by the vehicle and for lanes adjacent to the current lane, wherein the driving data represent a current driving situation of the vehicle and currently applicable boundary conditions;

[0008] Feeding a machine learning-trained model with the driving data in order to determine a trained, driver-specific positioning of the vehicle relative to the free space of the lanes for the current driving situation and the currently applicable boundary conditions;

[0009] Performing a comparison of the trained, driver-specific positioning with a current vehicle positioning relative to the captured free spaces, determined using environmental data, to identify any deviations; and

[0010] Generating a control signal for controlling the operation of the vehicle, wherein the control signal includes commands which, in at least partially automated driving operation, minimize the specified deviation by means of at least one device for longitudinal and / or lateral guidance of the vehicle.

[0011] The vehicle can be a motor vehicle. It can be a vehicle for at least partially automated driving or a vehicle that is already driving at least partially automatically. Automated driving can be understood as at least partially automated driving. The current driving situation of the vehicle and the currently applicable boundary conditions can include the vehicle's speed, traffic rules, regulations, traffic signs, and other related environmental conditions, as well as any changes to them. The driver-individual positioning of the vehicle relative to the free space of the lanes can be understood as the vehicle's positioning relative to available, drivable, and predicted free space. The positioning can include a static component or position and, additionally or alternatively, a dynamic component or movement.The trained model can be a multidimensional characteristic map, Gaussian processes, neural networks, or similar. The trained model can include static and dynamic correlations or model parameters. The deviation can be longitudinal, lateral, and additionally or alternatively diagonal. A lane can also be referred to as a traffic lane.

[0012] According to one embodiment, the trained model fed in the feeding step can be configured to adapt at least one assistance function of the vehicle to a trained, driver-specific following behavior, a trained, driver-specific lane-keeping behavior, and additionally or alternatively to a trained, driver-specific lane-changing behavior. The control signal generated in the creation step can include acceleration commands for at least one longitudinal guidance device and additionally or alternatively steering angle commands for at least one lateral guidance device. Additionally or alternatively, the commands can effect minimization by changing the longitudinal guidance, additionally or alternatively the lateral guidance, which is determined using the trained model. The respective behavior can include at least a distance, a trajectory, etc.Such a design offers the advantage that a following driving function, a lane keeping function and, additionally or alternatively, a lane change function can be specifically individualized and thus improved for a driver in order to increase acceptance and safety.

[0013] The trained model, which is fed during the feeding step, can be configured to adapt at least one assistance function to a driver-specific lane-change trajectory, which is trained separately for left and right lane changes and additionally or alternatively for different road types. Such an embodiment offers the advantage that driver-specific behavior can be taken into account, for example, for both pulling out and merging back into the lane during an overtaking maneuver, and additionally or alternatively depending on the road type.

[0014] Furthermore, during the generation step, a control signal can be generated that includes correction commands. These commands, in at least partially automated driving mode, use at least one longitudinal guidance device and, additionally or alternatively, lateral guidance to achieve a predefined, safe positioning of the vehicle relative to the detected free spaces, provided the commands result in a vehicle positioning that violates a predefined safety criterion. Such an embodiment offers the advantage that safe presets or factory settings for the assistance functions can be used should the trained or learned behavior violate at least one predefined, safety-relevant threshold.

[0015] The process can also include a step of training the model using machine learning during manual vehicle operation, utilizing environmental data and, additionally or alternatively, driving data, to train driver-specific following behavior, lane keeping behavior, and, additionally or alternatively, lane changing behavior. Manual operation can be understood as non-autonomous or non-assisted operation. Such an implementation offers the advantage that the model can be further trained during operation to enable the most precise and reliable individualization to the driver.

[0016] Static model parameters can include vehicle speed, traffic signs, traffic rules, road type, road condition, topography, lighting conditions, weather, open spaces next to the vehicle, and additionally or alternatively, the length, width, curve radii, and curve distance of each specific and predicted open space with respect to a preceding vehicle and additionally or alternatively a following vehicle in the current lane and additionally or alternatively in adjacent lanes. Dynamic model parameters can include changes to the static model parameters. Such an embodiment offers the advantage of enabling particularly precise and safe individualization to the driver, whereby the use of traffic-related open spaces in the vicinity of the vehicle can be considered in detail.

[0017] Furthermore, the training phase can be terminated or training results discarded if it is detected that the environmental data, and additionally or alternatively the driving data, indicate a violation of traffic regulations by the vehicle. Such an implementation offers the advantage that a driver's illegal behavior remains unconsidered during training, while situations arising from the illegal behavior of other road users, for example, can be taken into account during training.

[0018] This method can be implemented, for example, in software or hardware, or in a hybrid form of software and hardware, for example in a control unit or device.

[0019] The approach presented here further creates a device designed to perform, control, and implement the steps of a variant of the method presented here in appropriate facilities. This embodiment of the invention in the form of a device also allows the problem underlying the invention to be solved quickly and efficiently.

[0020] For this purpose, the device may have at least one processing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The processing unit may, for example, be a signal processor, a microcontroller, or the like, and the storage unit may be flash memory or a magnetic storage unit.The communication interface can be configured to read or output data wirelessly and / or via wired connections, whereby a communication interface that can read or output wired data can, for example, read this data electrically or optically from or output it into a corresponding data transmission line.

[0021] In this context, a device can be understood as an electrical device that processes sensor signals and outputs control and / or data signals accordingly. The device may have an interface, which can be implemented in hardware and / or software. In the case of a hardware-based interface, the interfaces can, for example, be part of a so-called system ASIC, which incorporates various functions of the device. However, it is also possible that the interfaces are separate integrated circuits or consist at least partially of discrete components. In the case of a software-based interface, the interfaces can be software modules, which, for example, are located on a microcontroller alongside other software modules.

[0022] Also advantageous is a computer program product or computer program with program code that can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out, implement and / or control the steps of the method according to one of the embodiments described above, in particular if the program product or program is executed on a computer or device.

[0023] Examples of the approach presented here are shown in the drawings and explained in more detail in the following description. It shows: Fig. 1 a schematic representation of a vehicle with an embodiment of a device for controlling an operation of the vehicle; Fig. 2 a flowchart of an exemplary embodiment of a method for controlling the operation of a vehicle; Fig. 3 a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2; Fig. 4 a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2; Fig. 5 a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2; Fig. 6 a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2; and Fig. 7 a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2.

[0024] In the following description of favorable embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and acting similarly, without repeating these elements.

[0025] Fig. Figure 1 shows a schematic representation of a vehicle 100 with an embodiment of a device 110 for controlling the operation of the vehicle 100. The vehicle is designed for at least partially automated driving operation. The vehicle 100 is a motor vehicle for at least partially automated driving operation. The device 110 is designed, in particular, to control the at least partially automated driving operation of the vehicle 100.

[0026] The vehicle 100 comprises at least one environmental sensor 101, at least one driving data provisioning device 105, at least one device 109 for longitudinal and / or lateral guidance of the vehicle 100, and the device 110. The at least one environmental sensor 101, the at least one driving data provisioning device 105, and the at least one device 109 for longitudinal and / or lateral guidance are connected to the device 110 in a data-transmitting manner.

[0027] The at least one environmental sensor 101 is configured to detect the environment of the vehicle 100. The at least one environmental sensor 101 is configured to provide environmental data 103. The environmental data 103 represents at least the detected open spaces between the vehicle 100 and at least one detected other vehicle in the environment of the vehicle 100. The open spaces are defined separately for a lane currently being traveled by the vehicle 100 and for lanes adjacent to the current lane.

[0028] The at least one driving data provision device 105 is designed to acquire and / or provide driving data 107 of the vehicle 100. The driving data 107 represents a current driving situation of the vehicle 100, such as the speed of the vehicle 100, and currently applicable boundary conditions, such as the type of road currently being traveled by the vehicle 100, the traffic rules applicable at the current position of the vehicle 100, etc.

[0029] The at least one device 109 for longitudinal and / or lateral guidance of the vehicle 100 is designed to control and / or effect the longitudinal and / or lateral guidance of the vehicle 100 using at least one actuator.

[0030] The device 110, or control device, comprises a reading device 112, a power supply device 114, a feed-through device 116, and a generation device 118. Furthermore, the device 110 also comprises an input interface 111A and an output interface 111B. According to one embodiment, the device 110 also comprises a training device 113.

[0031] The input device 112 is configured to read the environmental data 103 from the at least one environmental sensor 101 via the input interface 111A and to read the driving data 107 from the at least one driving data provision device 105 via the input interface 111A. Furthermore, the input device 112 is configured to transmit the environmental data 103 and / or the driving data 107 to the power supply device 114, the transmission device 116, and optionally, the training device 113.

[0032] The optional training device 113, provided according to an exemplary embodiment, is configured to train a model M during manual driving of the vehicle 100 using machine learning with the environmental data 103 and / or the driving data 107, in order to train driver-specific following behavior, lane keeping behavior, and / or lane changing behavior. The training device 113 is also configured to transmit the model M to the power supply device 114.

[0033] The data input device 114 is configured to feed at least the driving data 107 to the machine learning-trained model M in order to determine a trained, driver-specific positioning 115 of the vehicle 100 relative to the free space of the lanes for the current driving situation and the currently applicable boundary conditions. The data input device 114 is also configured to transmit the trained, driver-specific positioning 115 in the form of data or a signal to the execution device 116.

[0034] The implementation device 116 is configured to compare the trained, driver-specific positioning 115 with the current positioning of the vehicle 100 relative to the detected open spaces, determined using the environmental data 103, in order to determine any deviation 117 between them. The implementation device 116 is also configured to transmit the deviation 117 in the form of data or a signal to the generation device 118.

[0035] The generating unit 118 is configured to generate a control signal 119 for controlling the operation of the vehicle 100. The control signal 119 comprises commands that, in at least partially automated driving operation, minimize the specified deviation 117 by means of the at least one longitudinal and / or lateral guidance device 109 of the vehicle 100. In this context, the device 110 is also configured to output the control signal 119 to the longitudinal and / or lateral guidance device 109 via the output interface 111B.

[0036] The trained model M, which is trained by the optional training device 113 and / or fed by the power supply device 114, is configured according to an exemplary embodiment to adapt at least one assistance function of the vehicle 100 to a trained, driver-specific following behavior, a trained, driver-specific lane-keeping behavior, and / or a trained, driver-specific lane-changing behavior. The control signal 119 generated by the generation device 118 includes acceleration commands for at least one longitudinal guidance device and / or steering angle commands for at least one lateral guidance device. Additionally or alternatively, the commands effect minimization by means of a change in the longitudinal guidance and / or lateral guidance, which is determined using the trained model M.According to a further embodiment, the trained model M, which is trained by the optional training device 113 and / or fed by the power supply device 114, is also configured to adapt the at least one assistance function to a driver-specific lane-change trajectory, which is trained separately for left and right lane changes and / or specifically for different road types. In particular, static model parameters of model M include the speed of the vehicle 100, traffic signs, traffic rules, road type, road condition, topography, lighting conditions, weather, open spaces next to the vehicle 100 and / or the length, width, curve radii, and curve distance of each specified and predicted open space with respect to a vehicle ahead and / or a vehicle behind in the current lane and / or in adjacent lanes.Furthermore, dynamic model parameters of model M include, in particular, changes to the static model parameters. According to another embodiment, the optional training device 113 is also configured to terminate the training or discard training results when it is detected that the environmental data 103 and / or the driving data 107 indicate a violation of traffic regulations by the vehicle 100.

[0037] According to one embodiment, the generating device 118 is designed to generate a control signal 119 which includes correction commands which, in the at least partially automated driving operation, effect a predefined positioning of the vehicle 100 relative to the detected free areas by means of the at least one longitudinal guidance and / or lateral guidance device 109, if the commands would cause a positioning of the vehicle 100 that violates a predefined safety criterion.

[0038] Fig. Figure 2 shows a flowchart of an embodiment of method 210 for controlling the operation of a vehicle. The vehicle corresponds to or resembles the vehicle from one of the figures described herein. The vehicle is thus designed for at least partially automated driving operation. The method 210 for controlling the vehicle is carried out by means of or using the device from Fig. 1 or a similar device. The control method 210 comprises a reading step 212, a feeding step 214, an execution step 216, and a generation step 218. According to one embodiment, the control method 210 additionally comprises a training step 213.

[0039] In step 212 of the data acquisition process, environmental data is acquired via an interface from at least one environmental sensor of the vehicle, and driving data is acquired via an interface from at least one driving data provisioning unit of the vehicle. The environmental data represents at least the detected open spaces between the vehicle and at least one detected other vehicle in the vicinity of the vehicle. The open spaces are defined separately for a lane currently being traveled by the vehicle and for lanes adjacent to the current lane. The driving data represents the current driving situation of the vehicle and the currently applicable boundary conditions.

[0040] In step 214 of the "Feeding" process, a machine learning-trained model is fed with the driving data to determine a trained, driver-specific positioning of the vehicle relative to the lane's free space, based on the current driving situation and prevailing boundary conditions. In step 216 of the "Execution" process, the trained, driver-specific positioning is compared with the vehicle's current positioning relative to the detected free space, determined using environmental data, to identify any deviation. In step 218 of the "Generation" process, a control signal is generated to control the vehicle's operation. This control signal includes commands that, in at least partially automated driving mode, minimize the determined deviation by means of at least one longitudinal and / or lateral guidance system.

[0041] In the optional step 213 of the training, the model is trained in manual driving mode of the vehicle using machine learning with the help of environmental data and / or driving data in order to train driver-specific following behavior, driver-specific lane keeping behavior and / or driver-specific lane changing behavior.

[0042] Fig. Figure 3 shows a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2. Here, vehicle 100 are made of Fig. 1 or a similar vehicle and a foreign vehicle 300 are depicted on a road 330 during a following journey. The depiction in Fig. 3 is divided into a first partial representation for a current time t0 and a second partial representation for a future time t0+tp with predicted conditions.

[0043] In each of the partial representations, the other vehicle 300 is driving in front of vehicle 100 on a lane 332 of road 330 that is currently being used by vehicle 100. An adjacent lane 334 of road 330 is clear. Vehicle 100 is following behind the other vehicle 300.

[0044] Furthermore, the first part of the diagram shows a trajectory 100A travelled by vehicle 100 and a trajectory 300A travelled by the other vehicle 300, and the second part shows a predicted trajectory 100B travelled by vehicle 100 and a predicted trajectory 300B travelled by the other vehicle 300.

[0045] The first section of the diagram shows current free spaces F1, F2, F3, F4, F5, while the second section shows predicted free spaces F1, F2, F3, F4, F5, which are designated for vehicle 100 (or the ego vehicle) with respect to the other vehicle 300. A first free space F1 represents a specific free space for vehicle 100 with respect to the preceding other vehicle 300 in the adjacent lane 334 (or a side lane). A second free space F2 represents a specific free space for vehicle 100 with respect to a following other vehicle in the adjacent lane 334. A third free space F3 represents a specific free space for vehicle 100 with respect to the preceding other vehicle 300 in the current lane 332 (or the ego lane). A fourth free space F4 represents a specific free space for vehicle 100 with respect to a following other vehicle in the current lane 332.A fifth free space, F5, represents a free space next to vehicle 100, which may or may not be free. Between the two partial representations or the associated time points, a predicted change in at least some of the free spaces F1, F2, F3, F4, and F5 can be observed.

[0046] Fig. Figure 4 shows a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2. The representation in Fig. 4 corresponds to the representation from Fig. 3 with the exception that further foreign vehicles 300 are shown, a following foreign vehicle 300 in the current lane 332, a preceding foreign vehicle 300 in the adjacent lane 334 and a following foreign vehicle 300 in the adjacent lane 334. Thus, it shows Fig. 4 current and predicted free spaces F1, F2, F3, F4, F5 for following vehicle 100 and passing traffic on the adjacent left lane 334.

[0047] Fig. Figure 5 shows a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2. The representation in Fig. 5 corresponds to the representation from Fig. 4 except that, in particular, the foreign vehicles 300 on the adjacent lane 334 have different positions relative to vehicle 100. Furthermore, in Fig. Figure 5 shows a total of five sub-representations, where a first sub-representation 551 represents a current point in time, a second sub-representation 552 represents a first future point in time with predicted conditions, a third sub-representation 553 represents a second future point in time with predicted conditions, a fourth sub-representation 554 represents a third future point in time with predicted conditions, and a fifth sub-representation 555 represents a fourth future point in time with predicted conditions. Thus, it illustrates Fig. 5 a temporal change of the free spaces F1, F2, F3, F4, F5 for following the vehicle 100 or ego vehicle and passing traffic on the adjacent lane 334 to the left.

[0048] Fig. Figure 6 shows a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2. The representation in Fig. 6 corresponds to the representation from Fig. 5 except that one of the foreign vehicles 300 on the adjacent lane 334 is a longer vehicle. This results in, compared to Fig. 5 at least partially different open spaces F1, F2, F3, F4, F5 in the partial representations 551, 552, 553, 554 and 555. Thus illustrated Fig. 6 Temporal change of the open spaces F1, F2, F3, F4, F5 for following vehicle 100 or ego vehicle and passing traffic with longer foreign vehicle 300 on the left adjacent lane 334.

[0049] Fig. Figure 7 shows a schematic representation of a traffic situation in connection with the device made of Fig. 1 and / or the procedure from Fig. 2. Here, a first partial representation 751, a second partial representation 752, and a third partial representation 753 are shown. The first partial representation 751 corresponds to or resembles the first partial representation from one of the Fig. 4, Fig. 5 to Fig. 6, where only the positions of the other vehicles 300 on the adjacent lane 334 differ relative to vehicle 100. This also results in partially different open spaces F1, F2, F3, F4, F5. The second partial representation 752 represents a future point in time with a first variant of predicted conditions for the case where vehicle 100 accelerates relative to the other vehicles 300. The third partial representation 753 represents a future point in time with a second variant of predicted conditions for the case where vehicle 100 decelerates relative to the other vehicles 300. Thus, it shows Fig.7 a prediction of the free spaces F1, F2, F3, F4, F5 for following driving once with accelerating and once with decelerating vehicle 100 or ego vehicle and traffic on the left adjacent lane 334.

[0050] With reference to the figures described herein, exemplary embodiments, advantages of exemplary embodiments and further variants as well as fundamentals are summarized again and explained in other words.

[0051] When the learned or trained static and dynamic model M is executed by the device 110 and / or the method 210 within the scope of the vehicle 100's assistance system, the model M is continuously fed with the aforementioned currently determined parameters, i.e., the environmental data 103 and / or the driving data 107, and, for example, at least one learned, determined length of the free area F3 and / or F4 to the preceding and / or following vehicle 300 in the current lane 332 or ego lane is determined from the static model and compared with, for example, at least the currently determined length of the free area F3 to the preceding vehicle 300 and / or the free area F4 to the following vehicle 300 in the current lane 332 and the adjacent lane(s) 334.

[0052] By changing the speed, i.e., accelerating or decelerating, of vehicle 100, at least the currently determined length of the free space F3 and / or F4 is adjusted to the minimum learned or trained length of the free space F3 and / or F4 relative to the preceding vehicle 300 and / or following vehicle 300 in the current lane 332 and the adjacent lane(s) 334. The initial amount of acceleration or deceleration with which the adjustment of vehicle 100 occurs is determined by the dynamic model based on the change in the learned or trained speed.

[0053] The fundamentals, background, and / or prerequisites for the actions of the device 110 and / or the steps of the procedure 210 include, for example, the following: If there are no changes to traffic regulations and / or no other road users, in particular other vehicles 300, within the range of the ADAS sensors of the vehicle 100, the vehicle 100 drives at a fuel-saving, maximum permissible, or other speed along the route, regardless of the operating strategy of autonomous or assisted driving. Only when other road users, in particular other vehicles 300, are detected within the range of the ADAS sensors of the vehicle 100 do questions arise regarding a planned ego trajectory 100B, taking into account the predicted trajectories 300B of the other road users. This is essential for trajectory planning and the execution of the control of motor vehicles, both manually and with assistance.Autonomous operation involves the detection and prediction of (drivable) open spaces F1-F5, within which, in most situations, there is freedom regarding vehicle control (speed, acceleration / deceleration, steering angle) along possible trajectories 100B. Open space determination is possible, for example, using computer vision, such as semantic segmentation and assignment to the lanes around the vehicle 100, to detect drivable open spaces F1-F5. This is supported, for example, by lane detection, lane marking detection, road surface detection, etc., using known kinematics of the camera, vehicle, distance of image pixels to the road surface in front of the vehicle, known lane widths, dimensions of lane markings, or structure from motion, etc.The distance to each pixel in the open area can be determined, or, depending on the camera type, is already available for each pixel, thus allowing the exact determination of the extent of the open areas F1-F5. Road users or other vehicles 300 can be identified using classification and localization methods. Based on the prediction of the trajectories 300B of the surrounding road users or other vehicles 300, traffic infrastructure, and relevant traffic regulations / rules, the open area F1-F5 that is safely navigable for the vehicle 100 is continuously determined and predicted. The information can also be made available in the electronic horizon (eHorizon) and / or in semantic and / or GNSS-based digital maps.

[0054] According to exemplary implementations, for example, in manual or non-assisted operation, the driver learns and trains where and how to safely position the vehicle 100 within a specific and predicted free space F1-F5, typically taking into account traffic rules / regulations, road type, road condition, weather, etc. This includes how the driver typically handles the available free space F1-F5 and its changes. These driver-specific characteristics include variations in trajectories, speeds, etc., within the framework of the applicable traffic rules / regulations regarding distances to other road users.

[0055] In the case of extreme / necessary driving maneuvers of the vehicle 100, such as emergency braking, evasive maneuvers, limiting cornering speeds / lateral acceleration, etc., when it is a matter of ensuring safety by pushing the limits of physics using the associated assistance systems, driver-specific configurations are generally neither possible nor practical. Therefore, these maneuvers are ignored and not practiced or learned. Likewise, any manual driving maneuvers of the vehicle 100 that violate traffic rules / regulations within the designated and predicted free area F1-F5 are not considered in the learning or training, such as following too closely and thus maintaining an insufficient safety distance, overtaking on the right, merging into safety distances, disregarding traffic rules indicated by traffic signs, etc. These violations can be detected by computer vision in the vehicle 100 and then ignored.

[0056] Since driving maneuvers always also relate to lanes 332 and 334 – there are different types of roads, adjacent lanes 334 with or without oncoming traffic, lane-specific traffic rules, etc. – a subdivision of the determined and predicted free space F1-F5 according to lanes 332 and 334 is useful and simplifies procedure 210.

[0057] If no other road users or vehicles 300 are detectable in the vicinity, and thus the maximum possible open area F1 - F5 is limited solely by the range or field of view of the environmental sensors 101, then the driver's driving style, i.e., how the driver reacts to and follows the traffic rules, traffic signs, and the surroundings / topography (inclines, curves, vegetation, buildings, etc.), including changes / shifts, road type, road condition, weather, etc., is learnable. This also applies to how strongly the driver preferentially accelerates (with / without shifting) or decelerates (coasting / braking) when traffic signs change, for example, the permitted maximum speed, i.e., also taking into account the technical capabilities and specifics of the vehicle 100, as well as the driver's environmental awareness, driving style, and individual characteristics, etc.

[0058] If road users or foreign vehicles are recognizable in the vicinity, the following is possible, among other things, according to the following examples: Model "Following" Learning:

[0059] Driver- and vehicle-specific learning of static correlations, such as how to follow a vehicle ahead and / or drive ahead of a following vehicle, in non-autonomous and non-assisted operation, is based on the following parameters: vehicle speed 100; length, width, curve radii, and curve distance of the defined and predicted free area F3 with respect to the preceding foreign vehicle 300 on the current lane 332 or ego lane; length, width, curve radii, and curve distance of the defined and predicted free area F4 with respect to the following foreign vehicle 300 on the ego lane 332; length, width, curve radii, and curve distance of the defined and predicted free area F1 with respect to the preceding foreign vehicle 300 on the adjacent overtaking or driving lane 334; Length, width, curve radii, and curve distance of the determined and predicted free space F2 with respect tofollowing foreign vehicle 300 on the adjacent overtaking or driving lane 334 to the left; length, width, curve radii, and curve distance of the specified and predicted clear space with respect to a preceding foreign vehicle 300 on an adjacent overtaking or driving lane to the right; length, width, curve radii, and curve distance of the specified and predicted clear space with respect to a following foreign vehicle 300 on the adjacent overtaking or driving lane to the right; clear spaces F5 next to vehicle 100 on the right / left, which are clear or occupied; traffic signs and / or traffic rules (e.g., maximum speed limit, ...); road type (opposing directions of travel on 2 adjacent lanes or carriageway with x lanes), road condition, etc.; topography (e.g., road gradient, vegetation (forest entrance / exit), buildings (tunnel), etc.); Lighting conditions, weather (e.g., risk of ice, rain, wind, etc.) etc.

[0060] Driver- and vehicle-specific learning of the dynamic correlations, such as how to drive ahead of a foreign vehicle 300 and / or a following foreign vehicle 300, in non-autonomous and non-assisted operation, is carried out based on changes in the aforementioned parameters.

[0061] The static and dynamic parameters can provide insights into the driver's individual driving style and preferences, including: the extent to which the driver adheres to speed limits, how they accelerate, how they approach vehicles ahead, etc.; under what conditions they drive at which (maximum) speed, for example, when speed limits are lifted, etc.; the extent to which they are influenced in their driving behavior by following vehicles, for example, aspects of tailgating, increasing the safety distance behind, etc.; what conditions must prevail for them to perform an overtaking maneuver or lane change that they perceive as safe, etc.; and how they are influenced by environmental conditions (topography, light, weather, etc.), which driving style they typically choose under which environmental conditions, etc.

[0062] Should the driver attempt to change lanes 332 / 334, for which indicators such as turn signals and / or intention detection based on steering angle and / or gaze direction detection via a driver monitoring camera are conceivable, the learning / training process is terminated. Without considering navigation-related lane changes, the system implicitly learns when the driver typically follows another vehicle (300) and when not. This learned / trained information can be used for operating strategies of driver-specific autonomous driving, in which, for example, autonomous lane changes or overtaking maneuvers could be planned. The data can also be used in traffic flow simulations and / or navigation planning.

[0063] Should the vehicle violate traffic rules, the learning / training process will be terminated and / or the values ​​will not be considered in the learning. However, if other road users violate traffic rules, this will be implicitly taken into account in the learning process via the detected free areas F1-F5. This is because the trained / learned values ​​should, in assisted or autonomous driving, not only adhere to traffic rules but also be able to account for the misbehavior of other road users. Model “Follow” version:

[0064] In automatic or assisted driving mode, the trained static and dynamic model M is continuously fed with the above, currently determined parameters, i.e., the driving data 107 and / or the environmental data 103, and at least one learned, specific length of the free area F3 to the preceding foreign vehicle 300 in the current lane 332 is determined from the static model or static part of model M and compared with at least the currently determined length of the free area F3 to the preceding foreign vehicle 300. By changing the speed (accelerating or decelerating) of the vehicle 100, at least the currently determined length of the free area F3 is adjusted to the at least learned, specific length of the free area F3 to the preceding foreign vehicle 300. The initial amount of acceleration or deceleration with which the adjustment of the vehicle 100 occurs results from the dynamic model or...The dynamic part of model M is adjusted based on the change in the learned speed. Further necessary adjustments to the vehicle's acceleration or deceleration, for example due to model inaccuracies, latencies, tolerances, etc., are based on the remaining difference between at least the currently determined length of the free space F3 and at least the learned determined length of the free space F3 to the preceding foreign vehicle 300 (distance control, time-to-collision control, speed control, ...).

[0065] The "Follow" model is suitable, for example, for driver-specific customization of adaptive cruise control, brake assist, speed assist, traffic jam assist (stop & go), etc., by using the learned or trained parameters of model M within the scope of the assistance systems' customization options. For use with a lane keeping assist system, the additionally learned width of the free space F3 and / or F4 of the current lane 332 can enable further driver-specific customization. Should the assistance systems equipped with driver customization detect an increased risk (for example, a rear-end collision), the factory or standard settings and / or application data are used. This means that the factory or standard settings take priority over driver customization when there is imminent danger.Ultimately, the data from the training also shows the extent to which the driver follows the traffic, obeys the traffic rules and / or is influenced by other road users, environmental influences, etc. Learning model "changing lanes":

[0066] Driver- and vehicle-specific training and learning of the static and dynamic correlations regarding whether, when, and how lanes 332 and 334 are changed in non-autonomous or non-assisted operation: As listed above, compared to following another vehicle 300, the parameters regarding length, width, and curve radii and distance of the defined and predicted free spaces F1 and F2 with respect to preceding and following other vehicles 300 in the adjacent overtaking or driving lane 334, as well as the road type (e.g., adjacent lanes with oncoming traffic) and the traffic rules (e.g., no lane changes), etc., are crucial compared to the "following" model. Likewise, driver- and vehicle-specific learning takes place regarding the speeds, accelerations, free spaces F1-F5 (available distances and lane widths), etc., and their changes when changing lanes 332 and 334.Since the learned speeds and accelerations are learned in three coordinates, the lateral acceleration is available, with which the driver-specific lane change takes place.

[0067] Should the driver abort the lane change, violate traffic rules, or obstruct or endanger traffic (for example, by failing to maintain safe following distances), the learning / training process is terminated and / or the learned values ​​are discarded. However, if other road users do not adhere to traffic rules, this is also taken into account during the learning / training process. This is because the trained values, when used in assisted or autonomous driving, are intended to not only adhere to traffic rules but also to account for the errors of other road users. Model “Lane Change” version:

[0068] In autonomous or assisted driving mode, the learned static and dynamic model M is continuously fed with the currently determined parameters above. At least one learned, specific length of the free space F3 to the preceding foreign vehicle 300 in the current lane 332 is determined from the static model or static part of model M and compared with the currently determined length of the free space F3 to the preceding foreign vehicle 300. This applies if no following foreign vehicles 300 are detected in the vehicle's own lane 332 and no preceding or following foreign vehicles 300 are detected in the adjacent lane 334. However, as soon as further foreign vehicles 300 are detected, the other free spaces, i.e., F1 and / or F2 and / or F4 and / or F5, must be determined and fed into the model to determine whether and how to change lanes.By changing the speed (accelerating or decelerating) of vehicle 100, the minimum currently determined length of the free area F3 is adjusted to the minimum learned length of the free area F3 relative to the preceding vehicle 300. The initial amount of acceleration or deceleration with which the adjustment occurs is derived from the dynamic model relationship within the dynamic part of model M based on the change in the learned or trained speed of vehicle 100. Further necessary adjustments to the acceleration or deceleration of vehicle 100, for example due to model inaccuracies, latencies, tolerances, etc., are based on the remaining difference between the minimum currently determined length of the free area F3 and the minimum learned length of the free area F3 relative to the preceding vehicle 300 (distance control, time-to-collision control, speed control, etc.).Lateral acceleration is ultimately triggered automatically by a specific steering angle.

[0069] As soon as vehicle 100 has completely left the current lane 332, the described control process takes place using the values ​​of the (formerly) adjacent lane 334 as the current lane 332 (at least the learned minimum length of the clearance to the vehicle ahead in the adjacent lane, etc.). Ultimately, the control process (longitudinal and lateral acceleration) results in the driver-specific learned or trained overtaking trajectory, taking into account the other learned parameters. This information can be used for driver-specific lane change and overtaking assistants. Should the driver-specific assistance systems detect an increased risk (for example, a risk of an accident), the factory settings or application data are used. This means that the factory settings take priority over the driver-specific settings if there is imminent danger.

[0070] The "lane change" model is learned / trained and executed separately for left and right lane changes. This allows, for example, the learning and execution of an overtaking maneuver involving a lane change to overtake and another lane change to merge back into the original lane 332, or lane changes on multi-lane carriageways (332, 334). Since the learning parameters also include the road type (adjacent left lanes in the opposite direction / oncoming traffic or adjacent lane(s) with the same direction of travel), driver-specific overtaking behavior with and without oncoming traffic can be learned. During execution, the road type and the ego lane or current lane 332 can be determined based on the GNSS position and maps.

[0071] According to the exemplary embodiments, training of the respective model M with the specifically measured values ​​of the learning parameters of the driver and their vehicle can take place online in the vehicle electronics, as well as offline in a data cloud or in the laboratory after data transfer. Suitable model types include, for example, multidimensional characteristic maps, Gaussian processes, neural networks, etc. Because known driver assistance systems can, depending on the situation, specify a safe range of variation or limits for driver individualization according to the exemplary embodiments, a safety-critical state can be prevented by method 210, since a risky variation is always limited or would be limited. Ultimately, the driver individualization of assistance systems / ADAS / autonomous driving functions, etc.,Increased user acceptance means fewer driver takeovers, as the assistance systems act similarly to the driver, doubts about the correctness of the maneuvers can decrease, and this can increase trust in the systems.

[0072] If an embodiment includes an “and / or” connection between a first feature and a second feature, this is to be read as meaning that the embodiment according to one embodiment has both the first feature and the second feature, and according to another embodiment either only the first feature or only the second feature. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2021 210 610 A1

[0003]

Claims

[1] Method (210) for controlling the operation of a vehicle (100) designed for at least partially automated driving operation, wherein the method (210) comprises the following steps: Reading (212) environmental data (103) via an interface (111A) from at least one environmental sensor (101) of the vehicle (100) and driving data (107) via an interface (111A) from at least one driving data provisioning device (105) of the vehicle (100), wherein the environmental data (103) represent at least detected free spaces (F1, F2, F3, F4, F5) between the vehicle (100) and at least one detected other vehicle (300) in an environment of the vehicle (100), wherein the free spaces (F1, F2, F3, F4, F5) are defined separately for a current lane (332) driven by the vehicle (100) and for lanes (334) adjacent to the current lane (332), wherein the driving data (107) represent a current driving situation of the vehicle (100) and currently applicable boundary conditions; Feeding (214) a machine learning trained model (M) with the driving data (107) to determine a trained, driver-specific positioning (115) of the vehicle (100) relative to free spaces (F1, F2, F3, F4, F5) of the lanes (332, 334) for the current driving situation and the currently applicable boundary conditions; Performing (216) a comparison of the trained, driver-specific positioning (115) with a current positioning of the vehicle (100) relative to the recorded free spaces (F1, F2, F3, F4, F5) determined using the environmental data (103) in order to determine a deviation (117); and Generating (218) a control signal (119) for controlling the operation of the vehicle (100), wherein the control signal (119) includes commands which, in the at least partially automated driving operation, effect a minimization of the specified deviation (117) by means of at least one device (109) for longitudinal and / or lateral guidance of the vehicle (100). [2] Method (210) according to claim 1, wherein the trained model (M) fed in step (214) of feeding is configured to adapt at least one assistance function of the vehicle (100) to a trained, driver-specific following behavior, a trained, driver-specific lane keeping behavior and / or a trained, driver-specific lane change behavior, wherein the control signal (119) generated in step (218) of generating comprises acceleration commands for at least one longitudinal guidance device (109) and / or steering angle commands for at least one lateral guidance device (109), and / or wherein the commands effect minimization by means of a change in the longitudinal guidance and / or lateral guidance, which is determined using the trained model (M). [3] Method (210) according to claim 2, wherein the trained model (M) fed in step (214) of feeding is configured to adapt the at least one assistance function to a driver-specific lane change trajectory, which is trained separately for left and right lane changes and / or specifically for different road types. [4] Method (210) according to one of the preceding claims, wherein in the step (218) of generating a control signal (119) is generated which includes correction commands which, in the at least partially automated driving operation, by means of the at least one device (109) for longitudinal guidance and / or lateral guidance, effect a positioning of the vehicle (100) relative to the detected free areas (F1, F2, F3, F4, F5) that is predefined as safe, if the commands cause a positioning of the vehicle (100) that violates a predefined safety criterion. [5] Method (210) according to one of the preceding claims, comprising a step (213) of training the model (M) by machine learning in a manual driving operation of the vehicle (100) using the environment data (103) and / or the driving data (107) to train a driver-specific following behavior, a driver-specific lane keeping behavior and / or a driver-specific lane change behavior. [6] Method (210) according to claim 5, wherein static model parameters of the model (M) comprise a speed of the vehicle (100), traffic signs, traffic rules, road type, road condition, topography, lighting conditions, weather, free spaces (F5) next to the vehicle (100) and / or length, width, curve radii and curve distance of each specific and predicted free space (F1, F2, F3, F4) with respect to a preceding vehicle (300) and / or a following vehicle (300) on the current lane (332) and / or on adjacent lanes (334), wherein dynamic model parameters of the model (M) comprise changes to the static model parameters. [7] Method (210) according to one of claims 5 to 6, wherein the training step (213) is terminated or training results are discarded when it is recognized that the environmental data (103) and / or the driving data (107) indicate a violation of traffic rules by the vehicle (100). [8] Device (110) configured to perform and / or control the steps of the method (210) according to any of the preceding claims in corresponding units (112, 113, 114, 116, 118). [9] Computer program configured to execute and / or control the steps of the method (210) according to any one of claims 1 to 7. [10] Machine-readable storage medium on which the computer program according to claim 9 is stored.

Images