Method and device for predictive detection of a lane change

The procedure employs small artificial neural networks with aggregated features to predict lane changes, addressing the limitations of existing technologies by enabling early and robust detection with reduced data and network complexity.

DE102023208736A1Pending Publication Date: 2025-03-13ROBERT BOSCH GMBH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
DE102023208736
Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-09-08
Publication Date
2025-03-13

AI Technical Summary

Technical Problem

Existing technologies for predictive detection of lane changes in vehicles often require large machine learning networks, high data requirements, and binary labeling schemes, which limit their effectiveness and robustness.

Method used

A procedure using small artificial neural networks with aggregated features from sensor data, enabling efficient predictive recognition of lane changes with low data requirements and non-binary labeling schemes.

Benefits of technology

This approach allows for early detection of impending lane changes with improved robustness and prediction times of up to 500ms, while maintaining a small network size and low data requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The present invention relates to a method for the predictive detection of a lane change by a vehicle ahead from a secondary lane into its own lane, wherein aggregated features are generated from data of a sensor of the vehicle, and wherein a lane change maneuver is determined by means of an artificial neural network, the aggregated features being used as input variables into the artificial neural network. The invention further relates to a device configured to execute the corresponding method and to a corresponding computer program.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present invention relates to a method for the predictive detection of a lane change by a vehicle ahead from a secondary lane into its own lane, wherein aggregated features are generated from data of a sensor of the vehicle, and wherein a lane change maneuver is determined by means of an artificial neural network, the aggregated features being used as input variables into the artificial neural network. The invention further relates to a device configured to execute the corresponding method and to a corresponding computer program. State of the art

[0002] The present invention describes an AI-based approach for the predictive detection of lane changes (cut-in maneuvers). Particularly on highways with large speed differentials, lane change maneuvers are often critical when vehicles, during the lane change, fall below the safe following distance of the vehicle in the target lane. If an autonomous vehicle can detect another vehicle's intention to change lanes early, it becomes possible to defuse the situation early, for example, by cooperating with the other vehicle's braking or by proactively leaving the lane and making room.

[0003] Patent application DE 10 2020 124 174 A1 is known from the prior art. This application relates to a method for providing a machine-learned control function for vehicle control based on provided vehicle sensor data. The method comprises the following steps: providing the control function, in this example the lane change warning system, based on an artificial neural network for predicting lane change maneuvers by surrounding vehicles (LCMP), and issuing a comprehensive control command. The control commands can be issued based on the LCMP, which can predict a "lane change right," a "lane change left," and "no lane change" by a surrounding vehicle. The use of sensor data typically generates very large networks in machine learning methods. Disclosure of the invention

[0004] Advantageously, the method and device according to the invention allow for working closely with sensor data while enabling small networks (e.g., approximately 1,500 parameters). Data fusion is not required. Furthermore, automatic labeling is enabled. Since the proposed AI architectures are small, the data requirement for training the approach is comparatively low. This is made possible according to the invention by the features specified in the independent claims. Further embodiments of the invention are the subject of dependent claims.

[0005] According to the invention, a method for the predictive detection of a lane change by a preceding road user from a secondary lane into a dedicated lane of a motor vehicle is provided, wherein the method comprises the steps: • Providing sensor data from the vehicle based on monitoring of the vehicle's surroundings; • Determination of aggregated characteristics regarding a preceding road user based on the sensor data of the motor vehicle; • Feeding the aggregated characteristics regarding the road user ahead into at least one machine learning model; • Evaluation of the aggregated characteristics regarding the road user ahead using at least one machine learning model; • Determining a lane change maneuver by the vehicle ahead, taking into account the evaluation of the machine learning model.

[0006] A lane change detector is planned, which uses artificial intelligence to detect an impending lane change by another road user. Here, the sensor data is not used directly as input for the machine learning model, but rather aggregated features. These aggregated features describe, in particular, a relative value between the potential target object (the road user ahead) and the ego vehicle. Alternatively or additionally, the aggregated features can also describe a value between the potential target object and other objects in its driving environment or the driving situation in general. This allows for the integration of so-called contextual knowledge. An image sensor, for example, can be used as the sensor; advantageously, a video camera from a vehicle can be employed. These aggregated features can be processed, for example, using classic perception algorithms and / or, for example, [further examples would be needed here].an optical flow is generated from the sensor data.

[0007] A machine learning model can be, for example, an LSTM network, a convolutional neural network (CNN), or a fully connected neural network. By using aggregated data (e.g., object lists and their attributes), a small network (1,500 parameters) can be implemented that runs on the CPU sub-architecture of a camera in a vehicle. Furthermore, only the aggregated attributes relating to a potential target object are used as inputs for the machine learning model. This allows the number of inputs to be kept constant and low (despite potentially changing numbers of objects in the scene).In this sense, a method for the predictive detection of a lane change by a preceding road user from a secondary lane into a dedicated lane of a motor vehicle is described, wherein aggregated features are formed from data of a sensor of the motor vehicle, wherein a lane change maneuver is determined by means of an artificial neural network, wherein aggregated features relating to the preceding road user are used as input variables in the artificial neural network.

[0008] Furthermore, the system is designed not only to detect lane changes that have already occurred, but also to predictively detect impending lane changes. This allows, for example, the detection of an initiated lane change even before the merging vehicle has left its lane. For this purpose, a ground truth structure is proposed in which predictive states (e.g., phases preceding the actual lane change) are introduced before the actual lane change takes place. In contrast, current technology typically shifts the time of the actual lane change into the past in ground truth data to achieve prediction. This results in low robustness and short prediction times. The proposed approach, however, enables prediction times of 500 ms with respect to the moment the object wheel contacts the lane marking.

[0009] Lane changes are determined by calculating a probability value, where the calculated value exceeds a defined threshold. These probability values ​​can be advantageously used in situational analysis, for example, to determine the probability of a lane change by a vehicle ahead. Alternatively or additionally, this information can be used for target selection in a driver assistance system. Such information enables an early reaction to the lane change of another road user. For example, the autonomous vehicle can initiate cooperative braking to ensure a sufficient safety distance. Alternatively, it is also conceivable to change lanes to make way for the merging vehicle.It can also be used to provide information or a warning to the driver of the motor vehicle to alert him to the detected imminent lane change of the road user ahead.

[0010] In an advantageous embodiment, the method is characterized in that the determination of the lane-changing maneuver is carried out using a non-binary identification scheme.

[0011] Prior art approaches typically use a binary label scheme for lane changes. Differentiating between "lane change left," "lane change right," and "no lane change" also corresponds to a binary labeling scheme. For example, the ground truth label is set from 0 to 1 as soon as the front wheel touches the line. To generate predictive functionality, prior art approaches require shifting the label into the past and forcing the network to predict the lane change from the features. This method creates limited predictive capability because it imposes a two-class problem on the network.

[0012] This problem is to be avoided by using a multidimensional labeling scheme for the lane-changing process. For this purpose, a lane-changing process is divided into several phases, or classes. These phases can vary in length and are related to different input features and their temporal progression. Such a phase division also allows for the selection of different decision thresholds in an early, uncertain phase of the lane change than when the lane-changing vehicle has already touched the lane marking.

[0013] In one possible embodiment, the method is characterized in that the lane-changing maneuver is divided into a plurality of phases, in particular into more than two phases, preferably into four phases.

[0014] In a preferred embodiment, the method is characterized by the fact that different decision thresholds are defined for a determined lane-change maneuver in different phases.

[0015] This means, for example, that a lane change from the right has the following phases: First, the vehicle is on the right (P1, "stay right"), then it approaches its own lane (P2, "approach ego"), then the vehicle arrives in its own lane (P3, "arrive ego") and then remains in the ego vehicle's own lane (P4, "stay ego"). A lane change to the right can, for example, have the following four phases: First, the vehicle is in the ego vehicle's own lane (P4, "stay ego"), then it approaches the right-hand adjacent lane (P5, "approach right"), then the vehicle arrives in the right-hand adjacent lane (P6, "arrive right"), and then remains in the right-hand adjacent lane (P1, "stay right").

[0016] In an alternative further training, the procedure is characterized by the fact that an output vector of the machine learning model comprises a plurality of classes for the lane-changing maneuver, in particular more than two classes, preferably six classes.

[0017] This means, for example, that an output vector of an Artificial Neural Network includes a plurality of classes for the lane-changing maneuver, in particular more than two classes, preferably six classes.

[0018] In an advantageous embodiment, the method is characterized by the fact that the aggregated features are determined from a two-dimensional image space. This means that the features are advantageously extracted from the 2D image space and are therefore available with low latency. This enables fast (since no 3D conversion is performed) and low-error prediction.

[0019] In one possible embodiment, the method is characterized by the fact that the aggregated features include at least one feature that describes a relationship between the road user ahead and its environment, in particular one that describes a predicament of the road user ahead.

[0020] This means that one of the features to be considered describes the context of the driving situation of the vehicle ahead being evaluated. Context can include, for example, the potential influence of another vehicle on the vehicle ahead being evaluated (e.g., a slow-moving vehicle ahead). Context can also include the potential influence of a road situation on the vehicle ahead being evaluated (e.g., the merging of its lane into another). One possible feature to be explicitly mentioned here is the situational TTC (Time-To-Collision). This describes the TTC of a vehicle relative to the vehicle in front. This is a useful feature for recognizing situations where a vehicle is in a difficult position. Since the situational TTC refers to a predicted object, the number of input features can be fixed.

[0021] In an alternative embodiment, the method is characterized in that the aggregated features describe one of the following classes: - Identification size of a vehicle-object relationship; - Characteristic size of a vehicle-environment relationship.

[0022] In a preferred advanced training method, the procedure is characterized by the fact that a totality of the aggregated features is kept constant as input variables in the machine learning model.

[0023] This means that the input vector has a defined, constant size. The size of the input vector is such that it encompasses all relevant aggregated features relating to an environment object. Therefore, the lane-change probability for each object to be analyzed, i.e., each relevant road user, must be determined separately. This allows the number of input variables to be kept constant and low (despite a changing number of objects in the scene). For example, it is intended that the entirety of the aggregated features be kept constant as the input variable for an artificial neural network.

[0024] In an advantageous further training, the procedure is characterized by the fact that the aggregated characteristics are determined: - from sensor data, and / or - from an optical flow, and / or - as a derived quantity.

[0025] As mentioned previously, aggregated features are preferably those that represent a relative size between the target object (ahead road user) and the ego vehicle, or to other objects in the driving environment. These features can be derived, for example, from the video sensor data. Examples include: relative position, relative acceleration, relative speed, and absolute speed. Alternatively or additionally, the features can be derived from the optical flow. Examples include: lateral TTC (time-to-collision). Furthermore, the features can be determined indirectly as so-called generated features. Examples include: situational TTC, vehicle next to the target, distance to the lane markings, clearance in front of the ego vehicle, and the number of objects in the scene. Features derived in this way can originate from object detection in the camera images, if applicable.in combination with data preprocessing and, for example, a high-resolution street map.

[0026] Furthermore, a method for training at least one machine learning model for use in the previously described procedure is provided, wherein the training method comprises at least the following steps: providing training data representing aggregated features relating to a road user ahead of a motor vehicle who is performing a lane change; feeding the training data to the machine learning model to be trained, so that this machine learning model uses the method to determine a probability of a lane change by the road user ahead; performing the training of the machine learning model using the training data.

[0027] In an advantageous embodiment, the method is characterized by the implementation of a cost function for training the machine learning model, which allows for a non-binary weighting of the reward or punishment, in particular by the implementation of a Wiggle score as a cost function.

[0028] This means that a function can be used as a cost function (also called a loss function) and / or for KPI evaluation that creates leeway in the class transitions. Intentional ambiguities or soft transitions are permitted. For example, a wiggle score can be used for this purpose. The training of the artificial neural network can accordingly be carried out using the wiggle score for punishment and reward. In one possible embodiment, the method is characterized by the fact that the artificial neural network is trained using a loss function designed in such a way that a derivable, i.e., soft, transition between reward and punishment is formed.

[0029] In a possible further development, the procedure is characterized by the implementation of a cost function for training the machine learning model, which allows for different weighting of the reward or punishment for different defined phases of a lane-changing maneuver.

[0030] This means that a value is defined for each class (e.g., 1 for reward, -1 for punishment), and an appropriate weighting can be applied depending on the current lane-change phase (defined by the ground truth). The artificial neural network is thus trained using a loss function with a phase-specific weighting of the punishment or reward. For example, a misclassification between "arrive ego" and "stay ego" can result in a punishment of -0.1 (instead of -1). In other words, an artificial neural network can be trained where the loss function is designed such that a different phase-specific weighting of the reward or punishment is implemented for each of the four defined phases of a lane change.

[0031] In a preferred embodiment, the method is characterized in that, for the creation of training data, an automated labeling of sensor data is carried out to determine a lane change maneuver for training the machine learning model, wherein in a preceding step lane change sequences in the sensor data are determined and in a subsequent step the determined lane change sequences are divided into sub-sequences which correspond to defined lane change phases, and the sub-sequences are assigned to correspondingly defined ground truth classes.

[0032] This means that sequences with lane changes are automatically filtered from a dataset, divided into defined sub-sequences (phases of the lane change), and labeled accordingly. To achieve this, lane changes in long sequences are detected ex-post by comparing the position of the object and the lane marker. The identified lane change sequences are then decomposed into phases based on the defined phases of a lane change, and the Groundtruth class is generated. The labeled data can then be used as training data for the artificial neural network.

[0033] The described procedure can be performed by one or more computers with one or more data processing units. The term "data processing unit" can be understood as any type of entity that enables the processing of data or signals. The data or signals can, for example, be processed according to at least one (i.e., one or more than one) specific function performed by the data processing unit. A data processing unit can comprise or be composed of an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an integrated circuit, a programmable gate array (FPGA), or any combination thereof.Any other method for implementing the respective functions described in more detail herein may also be understood as a data processing unit or logic circuit arrangement. One or more of the process steps described in detail herein may be executed (e.g., implemented) by a data processing unit through one or more special functions performed by the data processing unit.

[0034] According to various explanations, the process is therefore implemented primarily using computers.

[0035] The prediction task can be used to generate a control signal for a device or robotic device. The term "robotic device" can be understood as referring to any technical system (with a mechanical part whose movement is controlled), such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.

[0036] A device in question can include an assistance system for performing emergency braking and / or adaptive cruise control (ACC) for a motor vehicle, an assistance system for automated longitudinal and / or lateral control, an environmental sensing device, in particular a camera, lidar, and / or radar, a central or decentralized control unit configured to control one of the aforementioned devices or to execute the described procedure. A device for outputting information to the driver can also be considered a device. Alternatively or additionally, the device can include an actuator system for longitudinal and / or lateral control of the motor vehicle. In a broader sense, the device can encompass the entire vehicle.

[0037] Various embodiments can use sensor signals or sensor data from different sensors, such as video, radar, LiDAR, ultrasound, motion, thermal imaging, etc., as input data. This includes, in particular, image data (i.e., the sensor data from a camera). Image data can be understood as any type of data in image form (i.e., values ​​assigned to pixels), and therefore not necessarily only as color image data. Accordingly, an optical flow may be an optical flow between other sensor data assigned to pixels, such as color image data. The sensor task can include classifying the sensor data or performing a detection on the sensor data, for example, to detect the presence of objects (in the environment where the sensor data was acquired). In particular, embodiments are applicable to controlling and monitoring the execution of manipulation tasks, e.g., in assembly lines.

[0038] These embodiments can also be used to train (including validation and testing) a machine learning system by selecting suitable training data. For example, data with lower plausibility regarding the execution of the prediction task is selected, transmitted to a computer (thus reducing data traffic because only selected data is transferred), and subsequently used for training, validation, or testing. Thus, these embodiments can be used for training (including learning a control policy), generating training data, and testing, verifying, or validating to check whether a trained ML system can be operated safely. A ML system trained in this way can then be used for the prediction task (where it can then continue to be monitored using the monitoring task).

[0039] Although specific embodiments have been presented and described herein, it will be recognized by those skilled in the art that the specific embodiments shown and described can be replaced by a variety of alternative and / or equivalent implementations without departing from the scope of protection of the present invention. This application is intended to cover adaptations or variations of the specific embodiments discussed herein. Therefore, it is intended that this invention is limited only by the claims and their equivalents. Embodiments

[0040] It should be noted that the features listed individually in the description can be combined in any technically sensible way and demonstrate further embodiments of the invention. Further features and the suitability of the invention will become apparent from the description of exemplary embodiments with reference to the accompanying figures.

[0041] The figures show: Fig. 1. A representation of one embodiment of the invention; and Fig. 2 an alternative representation of an embodiment of the invention; and Fig. 3 a CNN architecture according to an embodiment of the invention; and Fig. Four different defined phases of a lane change as ground truth; and Fig. 5 the inference of a CNN according to an embodiment of the invention; and Fig. 6 a weighting function implemented as a Wiggle score; and Fig. 7. A schematic procedure for automatic labeling.

[0042] Fig. Figure 1 shows a representation of a first embodiment of the invention. A motor vehicle 1 (also called an ego vehicle or ego car) is shown in environment 10. The motor vehicle 1 is located on a lane 12a (called its own lane) of a roadway 12 (also called a driving corridor). The roadway also has a secondary lane 12b. The own lane is, for example, the middle lane of a three-lane highway, and the secondary lane 12b is, for example, the right-hand lane. Fig. Figure 1 shows a road user 11 ahead, who is changing lanes from the adjacent lane 12b into the lane 12a of the vehicle 1. Fig. Figure 2 shows a representation of such a situation from a different perspective.

[0043] The vehicle 1 has a sensor system 2 designed as a front camera for environmental sensing. Sensor system 2 records sensor data from the vehicle 1's surroundings 10. This sensor data is evaluated by an evaluation unit 3 (also called a processing unit) using evaluation software. Classical algorithms can be used to determine the aggregated features. The aggregated features are then evaluated using an artificial neural network to predict a lane change by a vehicle ahead. The artificial neural network can be integrated into the evaluation unit 3 or an alternative device.

[0044] Furthermore, in the illustrated embodiment, a device 4 for evaluating the driving situation is provided. The calculated probabilities for a lane change by a vehicle ahead can be further processed by this device. For example, a target object selection can be made based on the probabilities for a lane change calculated using the artificial neural network. This device 4 can alternatively be integrated into the evaluation device 3 or into the driver assistance system 5.

[0045] Signals are generated from the evaluation device 3 and / or the device 4 for evaluating the driving situation and forwarded to the driver assistance system 5 (also called driving assistant). The driver assistance system 5 can, for example, be designed as an autopilot that fully executes a driving task. Alternatively, the driver assistance system 5 can include adaptive cruise control and / or automated lane keeping, a (partially) automated lane change function, an automated braking function to standstill, an emergency stop assistant, and / or other functions. The driver assistance system 5 is therefore advantageously designed to control a corresponding actuator system 6 for longitudinal and / or lateral control of the vehicle. Alternatively, the driver assistance system 5 can also include such an actuator system.

[0046] In the illustrated embodiment, the driver assistance system 5 can communicate specific information to the driver via a device 7. The device 7 can, for example, be designed as a display and communicate information to the driver visually. Alternatively or additionally, the device 7 can, for example, be designed as a loudspeaker and communicate information to the driver acoustically. Similarly, the device 7 can, for example, be designed as a haptic actuator and communicate information to the driver via steering wheel and / or seat vibrations.

[0047] Fig. Figure 2 shows an alternative representation of an embodiment of the invention. Here, an ego-motor vehicle 1 is shown on a dedicated lane 12a of a three-lane roadway 12. Furthermore, a road user 11 is shown ahead in the right-hand adjacent lane 12b. The road user 11 ahead has initiated a lane change from the adjacent lane 12b to the dedicated lane 12a, but is still entirely in the adjacent lane 12b. Furthermore, in Fig. 2. The feature sTTC (situational time-to-collision) is shown. This feature describes the time to collision (TTC) that a particular vehicle has with the road user ahead (in the Fig. 2: Vehicle 13). This feature is suitable for recognizing a vehicle's predicament. This feature can be used as an input vector for the artificial neural network to determine a lane change. Since the situational TTC refers to a single object to be predicted, the number of input features can be kept fixed.

[0048] Fig. Figure 3 shows a CNN architecture according to one embodiment of the invention. Alternatively, an LSTM network or a fully connected neural network is also conceivable. The CNN architecture includes, for example, an input vector EV, which comprises, for example, 20 features. A time series t is taken into account. Also shown are levels 0 (L0, Layer 0) to 4 (L4, Layer 4). Level 0 can, for example, be implemented as a convolutional filter. Furthermore, a convolutional filter can be used between Level 1 and Level 2. Level 4 can, for example, be implemented as a softmax function. The output vector (AV) comprises the defined six classes of a lane change (e.g., (1) "stay right", (2) "approach right", (3) "arrive ego", (4) "stay ego", (5) "approach right", (6) "arrive right"): The CNN demonstrates advantageous performance with a small network size (approximately 1,500 parameters). A feedforward network configuration is shown (without the recurrent components / memory of an LSTM or RNN).The network uses aggregated data (feature time series), resulting in a small network size. To incorporate changes in input variables over time, the features from the last T-1 cycles are used as input. Without recurrent components, the network trains faster and more reliably because there are fewer local minima. Using a CNN makes the network particularly small, as the receptive fields focus on the temporal changes within a feature (Layer 0) and are then convolved between features to represent correlations (Layer 1). To keep the size of the input vector constant, the track-change probability is determined individually for each object. The network is small and can easily be called per object. A design that keeps the size of the feature vector constant is advantageous, as it allows for the creation of a uniformly balanced network.

[0049] Fig. Figure 4 shows different defined phases of a lane change as ground truth. Fig. Figure 4 shows four phases of a lane change from the right into the ego lane as a ground label (GrL). First, the vehicle is on the right (P1, "stay right"), then it approaches its own lane (P2, "approach ego"), then the vehicle arrives (P3, "arrive ego," where at time step 50 the vehicle is approximately in the middle of the lane line), and then remains in the ego vehicle's lane (P4, "stay ego"). The length of each phase can be varied and determines the prediction length. In the example shown, it is 0.5 seconds. Variations are possible, for example, by using different input features, a different perception method for determining the aggregated features, and / or different sensors.

[0050] For a vehicle leaving the Ego lane, two additional check-in counters are introduced: P5 ("approach right," vehicle approaching the right lane) and P6 ("arrive right," vehicle arriving in the right lane). This corresponds to the six exits of the CNN in [location missing]. Fig. 3. Phases P5 and P6, together with phases P4 (“stay ego”) and P1 (“stay right”), again describe four phases for a lane change maneuver.

[0051] Fig. Figure 5 shows the inference of a CNN according to an embodiment of the invention. Figure 5 shows the result of the inference for a lane change of an object approaching from the right into its own lane, with the probabilities (W, Probabilities) at the network outputs and the class decision (pred) determined by MaxLikelihood. For example, up to time step (t, frame) 13, the artificial neural network determines a probability of approximately 0.9 for class K1 ("stay right"). A threshold is applied to the determined probability, and the class decision (pred) is made based on this. In this case, it is P1 ("stay right"). This corresponds to the ground label for the respective time steps (t). From time step 13 to time step 25, the probability for class K1 decreases continuously, while the probability for class K2 ("approach ego") increases.From time step 17 onwards, the probability for class K2 predominates, and the corresponding class decision (pred) is made in favor of P2 ("stay right"). This corresponds to the ground label (GrL) for this time step. The same applies analogously to the other classes K3 ("arrive ego") and K4 ("stay ego"). Classes K5 ("approach right") and K6 ("arrive right") are currently not relevant, as they are reserved for an alternative lane change, as explained above.

[0052] Fig. Figure 6 shows a weighting function implemented as a Wiggle Score. A Wiggle Score can be used as a loss function and for KPI evaluation, creating flexibility in the class transitions. For this purpose, a score is defined for each class (e.g., 1 for reward, -1 for penalty), and an appropriate weight is applied depending on the current lane change phase (determined by the ground truth). For example, confusing the classes "stay ego-lane" and "arrive ego-lane" is only penalized very slightly, since both classes justify a lane change, and the distinction was only made to give the network flexibility to use different features in an early phase than in a later phase. In the illustrated embodiment, a penalty is not applied with a value of -1, but only with a value of -0.1 if the Ground-Truth Phase “arrive ego” was classified as “stay ego” (see G4, in time steps t= 45 - 60).The same applies if the Ground-Truth phase “stay ego” was classified with the class “arrive ego” (see G3, in time steps t= 65 - 95).

[0053] Fig. Figure 7 shows a schematic procedure for automatic labeling. Known state-of-the-art methods typically use manually generated ground truth data. The problem here is that labeling noise occurs without a clear criterion. Additionally, the manual effort is high. Therefore, automatic labeling is advantageously provided. For this purpose, lane changes in long sequences are detected ex-post by comparing the position of the object (ahead vehicle) and the lane marking. Then, as in Fig. 4 shown - the sequence is broken down into phases and the Groundtruth class is generated. Fig.Figure 7 illustrates the process as follows: In step S1, raw data is recorded on a test platform (e.g., a motor vehicle). The raw data is stored as Z1. In step S2, the raw data is locally analyzed to identify the relevant lane-change sequences. The identified sequences are stored as Z2 in a sequence database. Subsequently, filtering and feature generation are performed in step S3. In step S4, the artificial neural network is trained using the automatically labeled lane-change sequences, which are divided into defined phases. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2020 124 174 A1

[0003]

Claims

[1] Method for the predictive detection of a lane change of a preceding road user from an adjacent lane into a lane of a motor vehicle, • Providing sensor data of the motor vehicle from monitoring the surroundings of the motor vehicle; • Determination of aggregated characteristics regarding a road user ahead based on the sensor data of the motor vehicle; • Feeding the aggregated features of the preceding road user to at least one machine learning model; • Evaluation of the aggregated characteristics of the preceding road user using at least one machine learning model; • Determination of a lane change maneuver by the road user in front, taking into account the evaluation of the machine learning model. [2] Method according to claim 1, characterized bythat the lane change maneuver is determined using a non-binary labeling scheme. [3] Method according to one of the preceding claims, characterized by that the lane change manoeuvre is divided into a plurality of phases, in particular into more than two phases, preferably into four phases. [4] Method according to claim 3, characterized by that different decision thresholds are defined in different phases of a lane change maneuver. [5] Method according to one of the preceding claims, characterized by that an output vector of the machine learning model comprises a plurality of classes for the lane change maneuver, in particular comprises more than two classes, preferably six classes. [6] Method according to one of the preceding claims, characterized bythat the aggregated features comprise at least one feature which describes a relationship between the road user driving ahead and its surroundings, in particular which describes a predicament of the road user driving ahead. [7] Method according to one of the preceding claims, characterized by that a set of aggregated features is kept constant as input variables to the machine learning model. [8] A method for training at least one machine learning model for use in the method according to any one of claims 1 to 7, comprising the following steps: • Providing training data that represents aggregated characteristics of a road user ahead of a motor vehicle who is changing lanes; • feeding the training data to the machine learning model to be trained, so that this machine learning model determines a probability of a lane change of the preceding road user using the method according to one of claims 1 to 7; • Training the machine learning model using the training data. [9] Method according to claim 8, characterized by that a cost function is implemented for training the machine learning model, which enables a non-binary weighting of the reward or punishment, in particular that a wiggle score is implemented as a cost function. [10] Method according to claim 8 to claim 9, characterized by that a cost function is implemented for training the machine learning model, which allows different weighting of the reward or punishment for different defined phases of a lane change maneuver. [11] Method according to claim 8 to claim 10, characterized by that in order to create training data, sensor data is automatically labeled to determine a lane change maneuver for training the machine learning model, whereby in a preceding step, lane change sequences are determined in the sensor data and in a subsequent step, the determined lane change sequences are divided into subsequences which correspond to defined lane change phases, and the subsequences are assigned to defined ground truth classes. [12] Device (1, 2, 3, 4, 5, 6,7) arranged to carry out the method according to one of claims 1 to 11. [13] A computer program which, when executed by a device according to claim 12, is adapted to carry out the method according to any one of claims 1 to 11. [14] A machine-readable storage medium on which the computer program according to claim 13 is stored.

Citation Information

Patent Citations

  • Method for providing a machine-learned control function for vehicle control based on provided vehicle sensor data

    DE102020124174A1