Method and device for the predictive identification of a lane change

Abstract

The present invention relates to a method for the predictive identification of a lane change by a road user travelling ahead from an adjacent lane into the same lane as a motor vehicle, wherein aggregated features are formed from data from a sensor of the motor vehicle, wherein a lane-change manoeuvre is determined by means of an artificial neural network, wherein the aggregated features are used as input variables into the artificial neural network. The invention also relates to a device which is designed to carry out the corresponding method, and to a corresponding computer program.

Description

[0001] Description

[0002] title

[0003] Procedures and one

[0004] The present invention relates to a method for predictively detecting a lane change of a preceding road user from an adjacent lane into a lane of a motor vehicle. Aggregated features are formed from data from a sensor of the motor vehicle. A lane change maneuver is determined using an artificial neural network, and the aggregated features are used as input variables to the artificial neural network. Furthermore, the invention relates to a device configured to carry out the corresponding method, as well as to a corresponding computer program.

[0005] State of the art

[0006] The present invention describes an AI-based approach for the predictive detection of lane changes (cut-in maneuvers). Especially on highways with large speed differentials, lane-changing maneuvers are often critical when vehicles in the target lane fall below the safety distance from the vehicle in the lane when changing lanes. If an ego vehicle can detect another vehicle's intention to change lane early, it is possible to defuse the situation early, for example, by cooperatively braking or by the ego vehicle proactively leaving the lane and making way.

[0007] Patent application DE 10 2020 124 174 A1, for example, is known from the prior art. This document relates to a method for providing a machine-learned control function for vehicle control based on provided vehicle sensor data. The method comprises the method step: providing the control function, in this example the lane change warning system, based on an artificial neural network for predicting a lane change maneuver of surrounding vehicles (LCMP), comprising an outputtable control command. The control commands can be output based on the LCMP, which can predict a "right lane change," a "left lane change," and "no lane change" of a surrounding vehicle. The use of sensor data generally generates very large networks in machine learning methods.

[0008] Disclosure of the invention

[0009] Advantageously, the method and device according to the invention enables working closely with sensor data while still enabling small networks (e.g., approximately 1,500 parameters). Data fusion is still not necessary. In addition, automatic labeling is enabled. Since the proposed AI architectures are small, there is also a comparatively low data requirement for training the approach. This is made possible according to the invention by the features specified in the independent patent claims. Further embodiments of the invention are the subject of subclaims.

[0010] According to the invention, a method is provided for the predictive detection of a lane change of a preceding road user from an adjacent lane into a lane of a motor vehicle, the method comprising the steps of:

[0011] • Providing sensor data of the motor vehicle from monitoring the surroundings of the motor vehicle;

[0012] • Determination of aggregated characteristics regarding a road user ahead based on the sensor data of the motor vehicle;

[0013] • Feeding the aggregated features of the preceding road user to at least one machine learning model;

[0014] • Evaluation of the aggregated characteristics of the preceding road user using at least one machine learning model;

[0015] • Determination of a lane change maneuver by the road user in front, taking into account the evaluation of the machine learning model.

[0016] A lane change detector is therefore provided which uses artificial intelligence to determine an impending lane change by another road user. In this case, the sensor data is not used directly as input for the machine learning model, but rather aggregated features. The aggregated features describe in particular a relative size between the possible target object (a road user driving ahead) and the ego vehicle. Alternatively or additionally, the aggregated features can also describe a size between the possible target object and other objects in its driving environment or the driving situation in general. This allows so-called context knowledge to be integrated. An image sensor, for example, can be used as the sensor; advantageously, a video camera of a motor vehicle can be used. These aggregated features can be processed using classic perception algorithms and / or, for example,an optical flow from the sensor data.

[0017] A machine learning model can be understood as an LSTM network, a convolutional neural network (CNN), or a fully connected network. By using aggregated data (e.g., object lists and their features), a small network (1,500 parameters) can be implemented that can run on the CPU subarchitecture of a camera in a motor vehicle. Furthermore, only the aggregated features of a potential target object are used as input variables for the machine learning model. This allows the number of input variables to be kept constant and low (despite a potentially changing number of objects in the scene).In this sense, a method for the predictive detection of a lane change of a preceding road user from an adjacent lane into a lane of a motor vehicle is described, wherein aggregated features are formed from data of a sensor of the motor vehicle, wherein a lane change maneuver is determined by means of an artificial neural network, wherein aggregated features regarding the preceding road user are used as input variables in the artificial neural network.

[0018] Furthermore, it is intended to enable not only the detection of a lane change that has already occurred, but also the predictive detection of an impending lane change. This makes it possible, for example, to determine an initiated lane change even though the merging vehicle has not yet left its lane. For this purpose, a ground truth structure is proposed in which predictive states (e.g. phases preceding the actual lane change) are introduced before the actual lane change. In the state of the art, however, the time of the actual lane change is usually shifted back in time in the ground truth data in order to achieve a prediction. This results in low robustness and short prediction times. The proposed approach, in contrast, enables prediction times of 500 ms with respect to the time of contact between the object wheel and the lane marking.

[0019] The lane change is determined by calculating a value for the probability of a lane change, whereby the calculated value lies above a defined threshold. The probability values ​​can be advantageously used in situation analysis, e.g., to determine the probability of a lane change by a vehicle ahead. Alternatively or additionally, such information can be used in target object selection for a driver assistance system. Such information enables an early reaction to the lane change of the other road user. For example, the ego vehicle can perform cooperative braking to ensure a sufficient safety distance. Alternatively, an independent lane change to another lane is also conceivable to make way for the merging road user.Information or a warning can also be sent to the driver of the motor vehicle to make him aware of the detected impending lane change of the road user in front.

[0020] In an advantageous embodiment, the method is characterized in that the lane change maneuver is determined using a non-binary labeling scheme.

[0021] State-of-the-art approaches generally use a binary label schema for lane changes. Differentiating between “left lane change,” “right lane change,” and “no lane change” also corresponds to a binary labeling schema. For example, the ground truth label is set from 0 to 1 as soon as the front wheel touches the line. To create a predictive character, the state-of-the-art requires shifting the label into the past, forcing the network to predict the lane change from the features. This method creates a limited look ahead because a two-class problem is imposed on the network. This problem is to be avoided by using a multidimensional label schema for the lane change process. For this purpose, a lane change process is divided into several phases, or classes. These phases can have different lengths and be related to different input features and their temporal progression.Such a phase division also allows different decision thresholds to be selected in an early uncertain phase of the lane change than when the lane-changing vehicle has already touched the line.

[0022] In one possible embodiment, the method is characterized in that the lane change maneuver is divided into a plurality of phases, in particular into more than two phases, preferably into four phases.

[0023] In a preferred embodiment, the method is characterized in that different decision thresholds are defined in different phases of a determined lane change maneuver.

[0024] This means, for example, that a lane change from the right has the following phases: First, the vehicle is on the right (P1, “stay right”), then it approaches its own lane (P2, “approach ego”), then the vehicle arrives in its own lane (P3, “arrive ego”) and then stays in the ego vehicle’s own lane (P4, “stay ego”). A lane change to the right, for example, can have the following four phases: First, the vehicle is in the ego vehicle’s own lane (P4, “stay ego”), then it approaches the right adjacent lane (P5, “approach right”), then the vehicle arrives in the right adjacent lane (P6, “arrive right”), and then stays in the right adjacent lane (P1, “stay right”).

[0025] In an alternative development, the method is characterized in that an output vector of the machine learning model comprises a plurality of classes for the lane change maneuver, in particular comprises more than two classes, preferably comprises six classes.

[0026] This means, for example, that an output vector of an artificial neural network comprises a plurality of classes for the lane-changing maneuver, in particular more than two classes, preferably six classes. In an advantageous embodiment, the method is characterized in that the aggregated features are determined from a two-dimensional image space. This means that the features are advantageously taken from the 2D image space and are therefore available with low latency. This enables fast (since no 3D conversion is implemented) and low-error prediction.

[0027] In one possible embodiment, the method is characterized in that the aggregated features comprise at least one feature which describes a relationship between the preceding road user and its surroundings, in particular which describes a predicament of the preceding road user.

[0028] This means that one of the features to be considered describes the context of the driving situation of the preceding road user to be evaluated. A possible influence of another road user on the preceding road user to be evaluated (e.g., a slow-moving vehicle ahead of it) can be taken into account as a context. A possible influence of a road situation on the preceding road user to be evaluated can also be taken into account as a context (e.g., the dissolution / integration of their lane into another lane). One possible feature should be explicitly named here: the situational TTC (Time-To-Collision). This describes the TTC between a vehicle in front and the vehicle in front. This is a useful feature for detecting vehicle predicaments. Since the situational TTC refers to an object to be predicted, the number of input features can be kept fixed.

[0029] In an alternative embodiment, the method is characterized in that the aggregated features describe one of the following classes:

[0030] - Identification size of a vehicle-object relationship;

[0031] - Characteristic value of a vehicle-environment relationship.

[0032] In a preferred development, the method is characterized in that the totality of the aggregated features is kept constant as input variables in the machine learning model. This means that the input vector has a defined constant size. The size of the input vector is dimensioned such that it includes all relevant aggregated features with regard to an environmental object. The lane change probability must therefore be determined separately for each object to be analyzed, i.e. each relevant road user. This allows the number of input variables to be kept constant and low (despite a changing number of objects in the scene). For example, it is provided that the totality of the aggregated features is kept constant as input variable in an artificial neural network.

[0033] In an advantageous further development, the method is characterized in that the aggregated characteristics are determined:

[0034] - from sensor data, and / or

[0035] - from an optical flow, and / or

[0036] - as a derived quantity.

[0037] As already mentioned, features that represent a relative size between the target object (a road user driving ahead) and the ego vehicle or other objects in the driving environment are preferably used as aggregated features. These features can, for example, be determined from the video sensor data. Examples include: relative position, relative acceleration, relative speed, absolute speed. Alternatively or additionally, the features can be derived from the optical flow. Examples include: transverse TTC (time-to-collision). In addition, the features can be determined indirectly as so-called generated features. Examples include: situational TTC, vehicle next to target, distance to lane markings, free space in front of the ego vehicle, number of objects in the scene. Features derived in this way can originate from object detection in the camera images, if necessary.in combination with data preprocessing and, for example, a high-resolution road map.

[0038] Furthermore, a method for training at least one machine learning

[0039] A model is provided for use in the method described above, wherein the training method comprises at least the following steps: providing training data representing aggregated features relating to a road user traveling ahead of a motor vehicle who is changing lanes; feeding the training data to the machine learning model to be trained, so that this machine learning model uses the method to determine a probability of a lane change by the road user traveling ahead; training the machine learning model using the training data.

[0040] In an advantageous embodiment, the method is characterized in that a cost function is implemented for the training of the machine learning model, which enables a non-binary weighting of the reward or punishment, in particular that a wiggle score is implemented as a cost function

[0041] This means that a function can be used as a cost function (also called a loss function) and / or for the CPI evaluation, which creates free space at the class transitions. Blurredness or smooth transitions are deliberately permitted. For example, a wiggle score can be used for this purpose. The artificial neural network can be trained accordingly using the wiggle score for punishment and reward. In one possible embodiment, the method is characterized in that the artificial neural network is trained using a loss function designed in such a way that a derivable, i.e. smooth, transition between reward and punishment is formed.

[0042] In a possible further development, the method is characterized in that a cost function is implemented for the training of the machine learning model, which allows a different weighting of the reward or punishment for different defined phases of a lane change maneuver.

[0043] This means that a rating is defined for each class (e.g., 1 for reward, -1 for punishment), and an appropriate weighting can be applied depending on the current lane change phase (specified by the ground truth). The artificial neural network is therefore trained using a loss function with a phase-specific weighting of the punishment or reward. For example, in the case of a misclassification between "arrive ego" and "stay ego," a punishment of -0.1 (instead of -1) can be applied. In other words, an artificial neural network can be trained with the loss function configured such that a different phase-specific weighting of the reward or punishment is implemented for each of the four defined phases of a lane change.

[0044] In a preferred embodiment, the method is characterized in that, in order to create training data, an automated labeling of sensor data is carried out to determine a lane change maneuver for training the machine learning model, wherein, in a preceding step, lane change sequences are determined in the sensor data and, in a subsequent step, the determined lane change sequences are divided into subsequences which correspond to defined lane change phases, and the subsequences are assigned to defined ground truth classes.

[0045] This means that sequences with lane changes are automatically filtered from a data set, divided into defined sub-sequences (lane change phases), and labeled accordingly. For this purpose, lane changes are detected ex-post in long sequences by comparing the position of the object and the lane marker. The determined lane change sequences are broken down into phases based on the defined phases of a lane change, and the ground truth class is generated. The labeled data can then be used as training data for the artificial neural network.

[0046] The described method can be performed by one or more computers having one or more data processing units. The term "data processing unit" can be understood as any type of entity that enables the processing of data or signals. The data or signals can, for example, be processed according to at least one (i.e., one or more than one) specific function performed by the data processing unit. A data processing unit can comprise or be formed from an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an integrated circuit, a programmable gate array (FPGA), or any combination thereof.Any other way of implementing the respective functions described in more detail herein may also be understood as a data processing unit or logic circuit arrangement. One or more of the method steps described in detail herein may be performed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0047] According to various embodiments, the method is therefore particularly computer-implemented.

[0048] The prediction task can be used to generate a control signal for a device, or robotic device. The term "robotic device" can be understood as referring to any technical system (with a mechanical part whose movement is controlled), such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.

[0049] A corresponding device can include an assistance system for executing emergency braking and / or adaptive cruise control (ACC) for a motor vehicle, an assistance system for automatically controlling longitudinal and / or lateral guidance, an environment detection device, in particular a camera, a lidar and / or a radar, a central or decentralized control unit configured to control one of the aforementioned devices or to execute the described method. A device can also be understood as a device for outputting information to the driver. Alternatively or additionally, the device can comprise an actuator system for longitudinal and / or lateral control of the motor vehicle. In a broad sense, the device can comprise the entire vehicle.

[0050] Various embodiments can use sensor signals or sensor data from various sensors such as video, radar, LiDAR, ultrasound, motion, thermal imaging, etc. as input data. In particular, this includes image data (i.e., the sensor data from a camera). Image data can be understood as any type of data in image form (i.e., values ​​assigned to pixels), and not necessarily only as color image data. Accordingly, an optical flow may be an optical flow between sensor data assigned to pixels other than color image data. The sensor task can include classifying the sensor data or performing a detection on the sensor data, for example, to detect the presence of objects (in the environment in which the sensor data was obtained). In particular, embodiments are applicable to controlling and monitoring the execution of manipulation tasks, e.g., in assembly lines.

[0051] Embodiments can also be used to train (including validating and testing) a machine learning system by selecting suitable training data. For example, data with lower plausibility regarding the performance of the prediction task is selected, transmitted to a computer (thereby reducing data traffic because only selected data is transmitted), and subsequently used for training, validation, or testing. Embodiments can thus be used for training (including learning a control policy), generating training data, and testing, verifying, or validating to check whether a trained ML system can be operated safely. An ML system trained in this way can then be used for the prediction task (where it can then continue to be monitored using the monitoring task).

[0052] Although specific embodiments have been shown and described herein, it will be recognized by those skilled in the art that the specific embodiments shown and described may be substituted for a variety of alternative and / or equivalent implementations without departing from the scope of the present invention. This application is intended to cover adaptations or variations of the specific embodiments discussed herein. Therefore, it is intended that this invention be limited only by the claims and the equivalents thereof.

[0053] Embodiments It should be noted that the features listed individually in the description can be combined with one another in any technically reasonable manner and demonstrate further embodiments of the invention. Further features and the usefulness of the invention will become apparent from the description of exemplary embodiments with reference to the accompanying figures.

[0054] From the figures shows:

[0055] Fig. 1 is a representation of an embodiment of the invention; and

[0056] Fig. 2 is an alternative representation of an embodiment of the invention; and

[0057] Fig. 3 shows a CNN architecture of an embodiment of the invention; and

[0058] Fig. 4 different defined phases of a lane change as ground truth; and

[0059] Fig. 5 shows the inference of a CNN according to an embodiment of the invention; and

[0060] Fig. 6 a weighting function implemented as a wiggle score; and

[0061] Fig. 7 shows a schematic procedure for automatic labeling.

[0062] Fig. 1 shows a representation of a first embodiment of the invention. Shown here is a motor vehicle 1 (also called ego motor vehicle or ego vehicle) in environment 10. The motor vehicle 1 is located in a lane 12a (called its own lane) of a roadway 12 (also called a driving corridor). The roadway also has an adjacent lane 12b. The own lane is, for example, the middle lane of a three-lane motorway, and the adjacent lane 12b is, for example, the right-hand lane. Fig. 1 also shows a road user 11 traveling ahead, who is changing lanes from the adjacent lane 12b into the own lane 12a of the motor vehicle 1. Fig. 2 shows a representation of such a situation from a different perspective.

[0063] The motor vehicle 1 has a sensor system 2 configured as a front camera for detecting the surroundings. The sensor system 2 records sensor data from the surroundings 10 of the motor vehicle 1. This sensor data is evaluated by an evaluation device 3 (also called a computing unit) using evaluation software. Conventional algorithms can be used to determine the aggregated features. The aggregated features are then evaluated using an artificial neural network to predictively determine a lane change by a road user ahead. The artificial neural network can be integrated into the evaluation device 3 or an alternative device.

[0064] Furthermore, in the illustrated embodiment, a device 4 is configured for evaluating the driving situation. The calculated probabilities for a lane change by a vehicle ahead can be further processed. For example, a target object can be selected based on the probabilities for a lane change calculated using the artificial neural network. This device 4 can alternatively be integrated into the evaluation device 3 or into the driver assistance system 5.

[0065] Signals are generated from the evaluation device 3 and / or the device 4 for evaluating the driving situation and forwarded to the driver assistance system 5 (also called a driving assistant). The driver assistance system 5 can, for example, be configured as an autopilot that fully performs a driving task. Alternatively, the driver assistance system 5 can comprise adaptive cruise control and / or automated lane keeping, a (partially) automated lane change, an automated braking function to a standstill, or an emergency stop assistant, and / or other functions. The driver assistance system 5 is therefore advantageously configured to control a corresponding actuator system 6 for longitudinal and / or lateral control of the motor vehicle. Alternatively, the driver assistance system 5 can also comprise such an actuator system.

[0066] In the illustrated embodiment, the driver assistance system 5 can communicate specific information to the driver via a device 7. The device can be configured, for example, as a display and communicate information to the driver visually. Alternatively or additionally, the device 7 can be configured, for example, as a loudspeaker and communicate information to the driver acoustically. Similarly, the device 7 can be configured, for example, as a haptic actuator and communicate information to the driver, for example, via steering wheel vibrations and / or seat vibrations.

[0067] Fig. 2 shows an alternative representation of an embodiment of the invention. Here, an ego motor vehicle 1 is shown in its own lane 12a of a three-lane roadway 12. Furthermore, a road user 11 traveling ahead is shown in the right adjacent lane 12b. The road user 11 traveling ahead has initiated a lane change from the adjacent lane 12b to the own lane 12a, but is still completely in the adjacent lane 12b. Furthermore, Fig. 2 shows the feature sTTC (situational time-to-collision). This feature describes the TTC of a specific vehicle relative to the road user traveling ahead (vehicle 13 in Figure 2). This feature is suitable for detecting vehicle constraints. This feature can be used as an input vector for the artificial neural network to determine a lane change. Since the situational TTC relates to an object to be predicted, the number of input features can be kept fixed.

[0068] Fig. 3 shows a CNN architecture according to one embodiment of the invention. Alternatively, an LSTM network or a fully connected neural network is also conceivable. The CNN architecture provides, for example, an input vector EV comprising, for example, 20 features. A time series t is taken into account here. Also shown are levels 0 (L0, Layer 0) to level 4 (L4, Layer 4). Level 0 can, for example, be designed as a convolutional filter. Furthermore, a convolutional filter can be used between level 1 and level 2. Level 4 can, for example, be designed as a softmax function. The output vector (AV) comprises the defined six classes of a lane change (e.g.: (1) “stay right”, (2) “approach right”, (3) “arrive ego”, (4) “stay ego”, (5) “approach right”, (6) “arrive right”): The CNN shows advantageous performance at small network size (approximately 1,500 parameters).A feedforward network design is shown (without recurrent components / memory of an LSTM or RNN). The network works with aggregated data (feature time series), which results in a small network. To incorporate changes in the input variables over time, the features from the last T-1 cycles are used in the input. Without recurrent components, the network trains faster and more reliably because there are fewer local minima. Using a CNN makes the network particularly small, as the receptive fields specifically focus on temporal changes within a feature (layer 0) and are then convolved between the features to map correlations (layer 1). To keep the size of the input vector fixed, the lane change probability is determined individually for each object. The network is small and can easily be called for each object. A design in which the size of the feature vector is kept constant is advantageous.This allows a uniformly balanced network to be created.

[0069] Fig. 4 shows various defined phases of a lane change as ground truth. Fig. 4 shows four phases of a lane change from the right into the ego lane as the ground label (GrL). Initially, the vehicle is on the right (P1, "stay right"), then it approaches its own lane (P2, "approach ego"), then the vehicle arrives (P3, "arrive ego", whereby at point 50 the vehicle is approximately in the middle of the lane line) and then stays in the lane of the ego vehicle (P4, "stay ego"). The length of the individual phases can be varied and determines the length of the prediction. In the example case shown, it is 0.5 seconds. Variation is possible, for example, by using different input features, a different perception to determine the aggregated features and / or using different sensors.

[0070] For a vehicle leaving the ego lane, two additional checkpoints are introduced: P5 ("approach right"; vehicle approaches the right lane) and P6 ("arrive right"; vehicle arrives in the right lane). This corresponds to the six outputs of the CNN in Fig. 3. Phases P5 and P6, together with phases P4 ("stay ego") and P1 ("stay right"), again describe four phases for a lane change maneuver.

[0071] Fig. 5 shows the inference of a CNN, an embodiment of the invention, showing the result of the inference for an object changing lanes from the right into its own lane with the probabilities (W, Probabilities) at the network outputs and the class decision (pred) determined by MaxLikelihood. For example, up to frame (t, frame) 13, a probability of approximately 0.9 for class K1 ("stay right") is determined by the artificial neural network. A threshold is applied to the determined probability, and based on this, the class decision (pred) is made. In this case, this is P1 ("stay right"). This corresponds to the ground label for the corresponding time steps (t). From frame 13 to frame 25, the probability for class K1 decreases continuously, while the probability for class K2 ("approach ego") increases.From lane 17 onwards, the probability for class K2 predominates and the corresponding class decision (pred) is made for P2 (“stay right”). This corresponds to the ground label (GrL) for this lane. The same applies to the other classes K3 (“arrive ego”) and K4 (“stay ego”). The classes K5 (“approach right”) and K6 (“arrive right”) are currently not relevant, as they are reserved for an alternative lane change, as already explained above.

[0072] Fig. 6 shows a weighting function implemented as a wiggle score. A wiggle score can be used as a loss function and for the CPI evaluation, creating gaps in class transitions. For this purpose, a score is defined for each class (e.g., 1 for reward, -1 for punishment), and an appropriate weighting is applied depending on the current lane change phase (specified by the ground truth). For example, confusing the classes "stay ego-lane" and "arrive ego-lane" is penalized very little, since both classes justify a lane change, and the distinction was only made to give the network the freedom to focus on different features in an early phase than in a later phase. In the illustrated embodiment, a punishment is not given with a value of -1, but only with a value of -0.1, if the ground-truth phase “arrive ego” was classified as “stay ego” (see G4, in the journals t= 45 - 60).The same applies if the ground-truth phase “stay ego” was classified with the class “arrive ego” (see G3, in the journals t= 65 - 95).

[0073] Fig. 7 shows a schematic procedure for automatic labeling. Known prior art methods generally use manually generated ground truth data. The problem here is that without a clear criterion, labeling noise arises. In addition, the manual effort is high. Automatic labeling is therefore advantageously provided. For this purpose, ex-post lane changes are detected in long sequences by comparing the position of the object (a vehicle in front) and the lane marking. Then, as shown in Fig. 4, the sequence is broken down into phases, and the ground truth class is generated. Fig. 7 shows the process as follows: In step S1, the raw data is recorded on a test vehicle (e.g., a motor vehicle). The raw data is saved as Z1. In step S2, the raw data is locally evaluated to identify the relevant sequences of lane change maneuvers.The determined sequences are stored as Z2 in a sequence database. Filtering and feature generation then occur in S3. In step S4, the artificial neural network is trained using the automatically labeled lane change sequences divided into defined phases.

Claims

Claims 1 . Method for the predictive detection of a lane change of a preceding road user from an adjacent lane into a lane of a motor vehicle, • Providing sensor data of the motor vehicle from monitoring the surroundings of the motor vehicle; • Determination of aggregated characteristics regarding a road user ahead based on the sensor data of the motor vehicle; • Feeding the aggregated features of the preceding road user to at least one machine learning model; • Evaluation of the aggregated characteristics of the preceding road user using at least one machine learning model; • Determination of a lane change maneuver by the road user in front, taking into account the evaluation of the machine learning model.

2. Method according to claim 1, characterized in that the lane change maneuver is determined using a non-binary identification scheme.

3. Method according to one of the preceding claims, characterized in that the lane change maneuver is divided into a plurality of phases, in particular into more than two phases, preferably into four phases.

4. Method according to claim 3, characterized in that different decision thresholds are defined in different phases of a determined lane change maneuver.

5. Method according to one of the preceding claims, characterized in that an output vector of the machine learning model comprises a plurality of classes for the lane change maneuver, in particular comprises more than two classes, preferably six classes.

6. Method according to one of the preceding claims, characterized in that the aggregated features comprise at least one feature which describes a relationship between the preceding road user and its surroundings, in particular which describes a predicament of the preceding road user.

7. Method according to one of the preceding claims, characterized in that a totality of the aggregated features is kept constant as input variables in the machine learning model.

8. A method for training at least one machine learning model for use in the method according to one of claims 1 to 7, comprising the following steps: • Providing training data that represents aggregated characteristics of a road user ahead of a motor vehicle who is changing lanes; • feeding the training data to the machine learning model to be trained, so that this machine learning model determines a probability of a lane change of the preceding road user using the method according to one of claims 1 to 7; • Training the machine learning model using the training data.

9. The method according to claim 8, characterized in that a cost function is implemented for training the machine learning model, which enables a non-binary weighting of the reward or punishment, in particular that a wiggle score is implemented as a cost function.

10. Method according to claim 8 to claim 9, characterized in that a cost function for training the machine learning model is implemented, which has a different weighting which enables reward or punishment for different defined phases of a lane change maneuver.

11. Method according to claim 8 to claim 10, characterized in that, in order to create training data, an automated labeling of sensor data is carried out to determine a lane change maneuver for training the machine learning model, wherein, in a preceding step, lane change sequences are determined in the sensor data and, in a subsequent step, the determined lane change sequences are divided into subsequences which correspond to defined lane change phases, and the subsequences are assigned to defined ground truth classes.

12. Device (1, 2, 3, 4, 5, 6, 7) which is arranged to carry out the method according to one of claims 1 to 11.

13. A computer program which, when executed by a device according to claim 12, is configured to carry out the method according to one of claims 1 to 11.

14. A machine-readable storage medium on which the computer program according to claim 13 is stored.

Images