VEHICLE TRAJECTORY PREDICTION USING ROAD TOPOLOGY AND ROAD USER OBJECT CONDITIONS

Abstract

System (104) for controlling a vehicle (102), wherein the system (104) comprises the following: an electronic processor (200) designed to: To take, via a camera (112), a first image; Determine, within the first image, of road traffic factors which road lines (402A-402E) and at least one road user near the vehicle (102) are included; Generating, based on sensor information from one or more sensors (114) of the vehicle (102), a second image (400) depicting an environment surrounding the vehicle, wherein the second image (400) includes the road traffic factors; where the road lines (402A-402E) in the second image (400) have a visual pattern or color which indicates the designated direction of travel; Determine, based on the detected road traffic factors and the second image (400), a predicted trajectory (408A-408D) of the road user near the vehicle; wherein the trajectory (408A-408D) in the second image (400) is represented by a gradient of a visual pattern or color, where the lightest portion is the earliest position and the darkest portion is the latest position of the road user near the vehicle (102); Generating a steering command for the vehicle (102) based on the

Description

BACKGROUND OF THE INVENTION

[0001] The present invention relates to a system and a method for controlling a vehicle. Modern vehicles include semi-autonomous driving functions, such as adaptive cruise control, collision avoidance systems, self-parking, and the like. One aspect of autonomous driving systems is behavior planning / prediction.

[0002] DE 10 2015 224 338 A1 discloses a method for automated driving of a motor vehicle, comprising the following steps: providing environmental data, detecting objects in the environmental data, selecting a current scenario by a scenario interpretation device based on the detected objects, wherein a scenario has at least one elementary situation to which at least one monitoring area and one release criterion are assigned as attributes, furthermore, successively for each of the elementary situations of the current scenario: estimating a target point, planning a trajectory to the target point by a maneuver planning device, querying information about the monitoring area from the environmental perception device by the scenario interpretation device, checking whether the at least one release criterion is present by the scenario interpretation device, and if the at least one release criterion is fulfilled,Automated driving along the trajectory to the destination point by a control system. SUMMARY

[0003] As mentioned above, one aspect of autonomous driving systems is behavior planning / prediction. In neural networks, sensor information is often processed by a perception module. After the sensor information has been processed in a perception module, the next step is to use the processed information to determine a trajectory for a vehicle to follow. Once the vehicle's trajectory is determined, the future trajectories of other road users (other vehicles on the road or in the surrounding environment) are also taken into account. Many current behavior prediction algorithms consider the current state of road users to determine their future trajectories. However, this may not be enough information to guarantee an accurate, predicted trajectory.

[0004] Factors that can be considered when determining a road user's future trajectory include road topology, traffic signs / rules, and the like. The future behavior of these road users depends, to some extent, on these characteristics.

[0005] The invention provides a system and a method for controlling a vehicle according to independent claims 1 and 6, respectively.

[0006] Therefore, embodiments described herein include a system and a method for driving a vehicle based on predictions of the trajectories of road users located near the vehicle, while taking into account certain traffic factors that influence the probability of certain future trajectories. Embodiments provide, among other things, information on traffic signs, lane markings, and the like, in addition to the past dynamics of one or more road users within the vehicle's surrounding environment, in a simple image format for a neural network. Using a simple image format simplifies and improves the accuracy of classifying objects in the vehicle's surrounding environment. The accuracy of predicted road user trajectories is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings, in which identical reference numerals refer to identical and functionally similar elements across the separate views, are included in the patent specification together with the detailed description below and form a part thereof, and serve to further illustrate embodiments of concepts that include the claimed invention and to explain various principles and advantages of these embodiments. Fig. Figure 1 is a block diagram of a system for driving a vehicle according to one embodiment. Fig. 2 is a block diagram of an electronic control system of Fig. 1 according to one embodiment. Fig. 3 is a flowchart of a procedure for using the control of Fig. 2 for driving the vehicle from Fig. 1, according to one embodiment. Fig. 4 is an image generated by controlling Fig. 2 according to a block diagram of a vehicle control system installed in a vehicle of the system of Fig. 1 is included, according to one embodiment. Fig. 5 is a flowchart illustrating a behavior prediction process carried out by the system of Fig. 1 during the execution of the procedure of Fig. 3 is implemented according to one embodiment.

[0008] Those skilled in the art will recognize that elements in the figures are illustrated for the sake of simplification and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to improve the understanding of embodiments of the present invention.

[0009] The equipment and process components have optionally been represented by conventional symbols in the drawings, which show only those specific details relevant to understanding the embodiments of the present invention, so that the disclosure is not obscured by details which are readily apparent to persons skilled in the art with knowledge of the description herein. DETAILED DESCRIPTION

[0010] One embodiment provides a system for driving a vehicle. The system includes an electronic processor designed to acquire, via a camera, a first image; determine, within the first image, a traffic factor; generate, based on sensor information from one or more sensors of the vehicle, a second image depicting the environment surrounding the vehicle, the traffic factor being included in the second image; determine, based on the detected traffic factor and the second image, a predicted trajectory of a road user near the vehicle; and generate a steering command for the vehicle based on the predicted trajectory.

[0011] Another embodiment provides a device for controlling a vehicle. The device includes one or more sensors, including a camera, which are communicatively coupled to an electronic processor, wherein the electronic processor is designed to receive, via the camera, a first image; determine, within the first image, a traffic factor; generate, based on sensor information from the one or more sensors of the vehicle, a second image depicting the environment surrounding the vehicle, wherein the second image includes the traffic factor; determine, based on the detected traffic factor and the second image, a predicted trajectory of a road user near the vehicle; and generate a steering command for the vehicle based on the predicted trajectory.

[0012] Another embodiment provides a method for controlling a vehicle. The method includes capturing, via a camera, a first image; determining, within the first image, a traffic factor; generating, based on sensor information from one or more sensors of the vehicle, a second image depicting the environment surrounding the vehicle, including the traffic factor; determining, based on the detected traffic factor and the second image, a predicted trajectory of a road user near the vehicle; and generating a steering command for the vehicle based on the predicted trajectory.

[0013] Before any embodiments are explained in detail, it should be understood that this disclosure is not intended to be limited in its application to the details of the construction and arrangement of components set forth in the following description or illustrated in the following drawings. Embodiments are capable of other configurations and can be implemented or carried out in various ways.

[0014] A variety of hardware- and software-based devices, as well as a variety of different structural components, can also be used to implement various embodiments. Additionally, embodiments may include hardware, software, and electronic components or modules, which may, for discussion purposes, be illustrated and described as if the majority of the components were implemented solely in hardware. However, a person skilled in the art, based on reading this detailed description, would recognize that in at least one embodiment, the electronics-based aspects of the invention may be implemented in software (for example, stored on a non-transitory, computer-readable medium) executable by one or more processors.For example, “control units” and “controllers” described in the specification may include one or more electronic processors, one or more memory modules, including a non-transitory computer-readable medium, one or more communication interfaces, one or more application-specific integrated circuits (ASICs), and various connections (for example, a system bus) that connect the various components.

[0015] Fig. Figure 1 illustrates a System 100 for autonomous driving. System 100 includes a Vehicle 102. The Vehicle 102 can encompass various types and designs of vehicles. For example, the Vehicle 102 can be a car, a motorcycle, a truck, a bus, a semi-trailer truck, and others. The Vehicle 102 includes at least some autonomous functionality but may also require a driver or operator to perform driving functions.

[0016] In the illustrated example, the system 100 includes several hardware components including an electronic control 104, an input / output (I / O) interface 106, a braking system 108, a steering system 109, an acceleration system 110, other vehicle systems 111, a camera 112 and additional sensors 114.

[0017] The electronic control unit 104, the braking system 108, the steering system 109, the acceleration system 110, image sensors 112, additional sensors 114, and the other vehicle systems 111, as well as other various modules and components of the system 100, are interconnected by one or more control or data buses (for example, a CAN bus), enabling communication between them. The use of control and data buses for connecting the various modules and components and for exchanging information between them would be apparent to a person skilled in the art with regard to the description provided herein.

[0018] The camera 112 is designed to capture one or more images of the environment surrounding the vehicle 102, according to their respective fields of view. Although described here with regard to camera images, it is understood that the camera 112 may, in some embodiments, be or include a thermal imaging device and / or a radar device and / or a sonar device, and the like. In some embodiments, the camera 112 includes several types of imaging devices / sensors, each of which may be located in different positions on the interior or exterior of the vehicle 102. Although described with regard to a single camera 112, it is understood that the camera 112 may, in some embodiments, be multiple image sensors.

[0019] The electronic control unit 104 communicates with the steering system 109, the braking system 108, other vehicle systems 111, the camera 112, and additional sensors 114 via various wired or wireless connections. For example, in some embodiments, the electronic control unit 104 is directly coupled to each of the vehicle 102 components listed above via dedicated wiring. In other embodiments, the electronic control unit 104 communicates with one or more of the components via a common communication link, such as a vehicle communication bus (e.g., a Controller Area Network bus or CAN bus) or a wireless connection. It is understood that each of the vehicle 102 components can communicate with the electronic control unit 104 using different communication protocols. Fig. The illustrated embodiment provides only one example of the components and connections of the vehicle 102. Therefore, the components and connections of the vehicle 102 may be constructed in ways other than those illustrated and described here.

[0020] Fig. Figure 2 is a block diagram of an embodiment of the electronic control 104 of the system 100 of Fig. 1. The electronic control unit 104 comprises several electrical and electronic components that provide power, operational control, and protection for the components and modules within the electronic control unit 104. The electronic control unit 104 includes, among other things, an electronic processor 200 (such as a programmable electronic microprocessor, microcontroller, or similar device) and a memory 205 (for example, a non-transitory machine-readable memory). The electronic processor 200 is communicatively connected to the memory 205. The electronic processor 200, in coordination with the memory 205 and the communication interface 210, is designed, among other things, to implement the procedures described herein.

[0021] In some embodiments, the electronic control unit 104 includes several electrical and electronic components that provide power, operational control, and protection for the components and modules within the electronic control unit 104. The electronic control unit 104 may include or be one or more electronic control units, including, for example, a motor control module, a powertrain control module, a power transmission control module, a general electronic module, and the like. The electronic control unit 104 may include submodules containing additional electronic processors, additional memory, or additional application-specific integrated circuits (ASICs) for handling communication functions, processing signals, and performing the procedures listed below.In other embodiments, the electronic control unit 104 includes additional, fewer, or different components. The electronic processor 200 and the memory 205, as well as the other various modules, are connected by one or more control or data buses. In some embodiments, the electronic control unit 104 is partially or entirely implemented in hardware (for example, using a field-programmable gate array (“FPGA”), an application-specific integrated circuit (“ASIC”), or other devices).

[0022] The memory 205 can include one or more non-transient, computer-readable media and comprises a program storage area and a data storage area. As used in the present application, “non-transient, computer-readable media” includes all computer-readable media but does not consist of a transitory, propagating signal. The program storage area and the data storage area can include combinations of different types of memory, for example, read-only memory (“ROM”), random-access memory (“RAM”), electrically erasable programmable read-only memory (“EEPROM”), flash memory, or other suitable digital storage devices. The electronic processor 200 is connected to the memory 205 and executes software, including firmware, one or more applications, program data, filters, rules, one or more program modules, and other executable instructions.The electronic processor 200 retrieves instructions related to the control processes and procedures described here from memory 205 and executes them. In other embodiments, the electronic control unit 104 may include additional, fewer, or different components.

[0023] The memory 205 of the electronic control 104 contains software which, when executed by the electronic processor 200, instructs the electronic processor 200 to perform the operation described in Fig. 3 illustrated procedure 300 initiated. For example, the one in Fig. Figure 2 illustrates memory 205, a neural network 215, and object detection software 220. The neural network 215 can be a deep neural network (for example, a convolutional neural network (CNN) or a recursive neural network (RNN)). The neural network 215 includes one or more input channels that allow the neural network 215 to simultaneously analyze image data from the camera 112 (and, in some embodiments, sensor data from the additional sensors 114) in order to classify an object in the vehicle's surrounding environment and an action performed by the object. The object could, for example, be a road user (another vehicle / another driver) on the road where the vehicle 102 is located, and the road user's action should be determined.In some embodiments, the neural network 215 is trained to classify objects and the actions they perform using a training set of images and / or sensor data corresponding to the one or more participants on the road.

[0024] In some embodiments, the electronic processor 200, when executing the object detection software 220, uses machine learning techniques to detect objects that could impede the movement of the vehicle 102 in an image received from the camera 112. For example, the object detection software 220 may include a convolutional neural network trained to recognize vehicles, people, animals, a combination thereof, and the like. Other types of sensor data may also be used by the object detection software 220.

[0025] As mentioned above, when determining a vehicle's future trajectory, the system 100 can determine the appropriate trajectory based on the movement trajectories of other road users within the vehicle's surrounding environment. As explained in more detail below, the electronic control 104 is designed to generate an image depicting the vehicle's surrounding environment, including road traffic factors such as traffic signs and lane markings, in addition to other road users, which must be considered when predicting the trajectories of other road users.

[0026] Back to Fig. 1, in which the braking system 108, the steering system 109, and the acceleration system 110 each include components involved in autonomous or manual control of the movement of the vehicle 102. The electronic control unit 104 may be configured to control some or all of the functionality of one or more of the systems 108, 109, and 110 to steer and drive the vehicle. In some embodiments, the control unit 104 may exercise limited control over the systems 108, 109, and 110, and some or all of the driving may be controlled by the driver of the vehicle 102.

[0027] The other vehicle systems 111 include controllers, sensors, actuators, and the like for controlling aspects of the operation of the vehicle 102 (for example, acceleration, braking, gear shifting, and the like). The other vehicle systems 111 are designed to send and receive data relating to the operation of the vehicle 102 to and from the electronic control unit 104.

[0028] Fig. Figure 3 illustrates an example procedure 300 for driving a vehicle based on a predicted trajectory of a vehicle 102. In step 305, the electronic processor 200 captures a first image using a camera (for example, camera 112) and determines a road traffic factor (block 308) within the first image. Based on sensor information from one or more sensors (camera 112 and / or additional sensors 114) of the vehicle, the electronic processor 200 generates a second image depicting the environment surrounding the vehicle 102 (block 310), which, with regard to Fig. 3 is described in more detail below.

[0029] A road traffic factor consists of an object whose presence (e.g., action or significance) within the environment surrounding vehicle 102 can affect / influence a future trajectory of one or more road users near vehicle 102. The term "near" should be understood as being within a predetermined distance of vehicle 102. In one example, the distance is determined within 24 feet of vehicle 102, for example, by video or image analysis, ultrasonic distance measurement, or radio signal transmission distance. In some embodiments, the predetermined distance is set based on the speed of vehicle 102.For example, if vehicle 102 is traveling at a highway speed (e.g., approximately 60 mph), the predetermined distance may be greater than if the vehicle is traveling at a residential speed (e.g., approximately 30 mph). For instance, a road traffic factor could be a traffic sign, a traffic signal, a road marking, and / or the trajectory of an object within a predetermined distance from the road user. With regard to the traffic sign, traffic signal, and road marking, the certain significance of each is taken into account when determining the predicted trajectory of road users near vehicle 102 within the environment surrounding vehicle 102.For example, if the electronic processor 200, using the image sensor(s) 112 and the object detection software 220, detects a stop sign in the vicinity of the front of the vehicle 102, the electronic processor 200 can predict that (or add weight to the probability that) one or more road users near the vehicle 102, traveling in the same general direction, will slow down and stop before the stop sign. The object within a predetermined distance (for example, approximately 5 feet) from the road user can be an object whose position and / or certain trajectory influences the future trajectory of the road user. The object can be, for example, a pedestrian or an obstacle, such as a branch, a stopped vehicle, or another object.The object may be in a position, or move into one, that intersects the current trajectory of a road user, making it likely that the road user will change their trajectory. The road traffic factor, as well as other objects within the environment surrounding the vehicle 102, can be determined using object detection techniques via object detection software 220, for example, CNNs.

[0030] In block 315, the electronic processor 200 is designed to determine, based on the detected road factor (and an image, as described in more detail below), a predicted trajectory of a road user near the vehicle 102. In some embodiments (for example, as shown below in Fig. (as shown in Figure 5) the electronic processor 200 determines the predicted trajectory based on the detected road factor and the image using a deep learning network. In block 320, the electronic processor 200 generates a steering command for the vehicle 102 based on the image 400 and the predicted trajectory (or trajectories). The steering command can be an automated / semi-automated driving function, which is implemented by the electronic processor 200 by braking, accelerating, and / or turning the vehicle 102 (for example, via the braking system 108, the steering system 109, and / or the acceleration system 110).

[0031] Fig. 4 is an example image 400, which is in block 310 of procedure 300 of Fig. Figure 400 illustrates the traffic environment surrounding vehicle 102. In the provided example, Figure 400 depicts a road topology (lanes, road curvature, traffic direction, trajectory of road users, and the like) of the environment surrounding vehicle 102 in the form of a static image. In the illustrated embodiment, the road lines are shown as solid black lines 402A–402E. Different patterns and / or color markings can be used to indicate different types of road lanes (for example, whether the lane marking indicates that traffic in the same or opposite direction can overtake, whether there is a bicycle lane, etc.).The road lanes 404A–404E incorporate a visual pattern / color indicating the designated direction of traffic (for example, both lanes 404A and 404B incorporate a horizontally striped pattern indicating a primary direction of traffic, whereas lanes 404C–404E incorporate a diagonally striped pattern indicating a secondary direction of traffic). Figure 400 further depicts the emerging curve 406 of lane 404E over the bend in road line 402E.

[0032] The image 400 includes one or more road users 408A - 408D and their respective previous positions (forming a respective previous trajectory 410A - 410D) determined by the electronic processor 200 based on data from the camera 112 (and / or additional sensors 114). Similar to lane and line markings, each road user can be indicated by a certain color / pattern or shape. In some embodiments, a road user's image representation can be based on its previous trajectory, vehicle type, or other characteristics. The trajectories 410A - 410D can be, as in Fig. Figure 4 illustrates this via a gradient, where the lightest portion (indicated by the more widely spaced halftone lines) represents the earliest position and the darkest portion (indicated by the more closely spaced halftone lines) represents the latest position. The trajectories 410A–410D can be determined by the processor 200 based on prior sensor information from the camera 112 and / or additional sensors 114. In some embodiments, the processor 200 uses Gaussian blurring in determining the trajectories 410A–410D.

[0033] Fig. Figure 5 is a flowchart 500 illustrating a prediction process that utilizes the neural network 215 of the electronic processor 200, according to some embodiments. The process can be used by the processor 200 in block 315 of the method 300 described above. As in Fig. As illustrated in Figure 5, the neural network 215 can, in some embodiments, be a behavioral prediction deep neural network. For the sake of clarity, the flowchart 500 is described with regard to a single road user (here, road user 408D). It is understood that a similar process can be applied to additional road users (for example, those of Fig. 4) can be applied.

[0034] Image 400 is fed to a neural network 215 as input. In some embodiments, a modified version of image 400 is fed to the neural network 215. The modified version of image 400 can contain information about the road user closest to vehicle 102 and / or indicate that its previous trajectory intersects or will intersect the current trajectory of vehicle 102. For example, the road users in the modified image may be located within the same lane as the current lane, in a lane adjacent to the current lane in which vehicle 102 is located, within a certain radius of vehicle 102, or about to enter such a lane.

[0035] For example, in the illustrated embodiment, the image fed to neural network 215 is a modified image 502 of image 400. The modified image 502 is an edited version of image 400 that includes a single road user (here, road user 408C). It is understood that if a future trajectory is determined by other road users, image 400 will be modified accordingly to omit those other road users. It is understood that, although described here with respect to the modified image 502, the process illustrated in flowchart 500 can use image 400 in the same way. The edited image 502 omits road users 408A and 408B from image 400 ( Fig.4) because they are located in lanes 404A and 404B, which are opposite to the direction of traffic in lane 404D, where vehicle 102 is located. Road user 408D is omitted, even though it is traveling in lane 404E, which is adjacent to lane 404D and has the same direction of traffic, because the curvature 406 indicates that lane 404 separates from lane 404D ahead. Road user 408C (and its corresponding past trajectory 410C) is included in the modified image 502 because it is close to vehicle 102 and, considering its past trajectory 410C, is moving from the adjacent lane 404C to lane 404D, intersecting the current trajectory of vehicle 102. In some embodiments, the modified image 502 includes visual information relating to more than one road user.Road topology information can likewise be omitted / modified based on its proximity to vehicle 102.

[0036] In addition to image 502, the object state information 504 relates to the road users in image 502, which may not be obtainable from image 502 alone. Such information may include, for example, the speed (the velocity and direction of vehicle 102) and / or acceleration (a change in the speed of vehicle 102 over time). Accordingly, in some embodiments, the predicted trajectory is determined based on one or both of the speed and acceleration of the road user.

[0037] In some embodiments, the predicted trajectory of the road user nearest vehicle 102 is determined based on a predicted trajectory of a second road user. For example, if there were another road user in lane 404D ahead of road user 408C, for whom processor 200 predicts that they will decelerate, this prediction can be taken into account when determining the future trajectory of road user 408C (this reduces, for example, the probability that road user 408C would enter lane 404D).

[0038] Based on image 502 and object state information 504, the behavior prediction neural network 215 determines one or more predicted (future) trajectories of the road user within image 502 (in the illustrated embodiment, user 408C). During this determination, the neural network 215 considers the one or more detected traffic factors that may influence the future trajectory of the road user illustrated in image 502. For example, if there is a stop sign directly in front of lanes 404C and 404D, the processor 200 can determine, via the neural network 215, that road user 408C will slow down in the immediate future (depending on the distance from the stop sign).

[0039] In the illustrated embodiment, the identifier 506 (one of several possible future trajectories) is determined with the best cost function 508 (the one with the highest probability of being accurate). In other words, based on the detected road traffic factor and the image, the processor 200 determines a predicted trajectory of a road user near vehicle 102. Accordingly, as described above, the electronic processor 200 generates a suitable steering command for vehicle 102 based on the predicted trajectory (and image 400).

[0040] Specific embodiments were described in the preceding patent specification. However, the person skilled in the art in the field recognizes that various modifications and changes can be made without deviating from the scope of protection of the invention as set forth in the following claims. Accordingly, the patent specification and the figures should be regarded as illustrative rather than limiting, and all such modifications are to be included within the scope of protection of the present teachings.

[0041] In this document, relational terms, such as first and second, upper and lower, and the like, may be used solely to distinguish one entity or action from another, without actually requiring or implying any such relationship or order between such entities or actions. The terms "includes," "comprising," "indicates," "indicating," "includes," "containing," "including," "containing," or any other variations thereof are to be understood as referring to non-exclusive inclusion, such that a process, procedure, object, or facility which includes, exhibits, includes, or contains a list of elements may contain not only those elements but also other elements not expressly listed or inherent to such process, procedure, object, or facility.An element preceded by "comprises...a", "has...a", "includes...a", or "contains...a" does not, without further limitations, preclude the presence of additional identical elements in the process, method, article, or device that includes, has, includes, or contains the element. The term "a" is defined as one or more unless expressly stated otherwise herein. The terms "essentially", "generally", "approximately", "about", or any other version thereof are defined according to the understanding of a person skilled in the art and, in one non-restrictive embodiment, are defined as being within 10%, in another embodiment as being within 5%, in yet another embodiment as being within 1%, and in still another embodiment as being within 0.5%.The term "coupled," as used here, is defined as connected, though not necessarily directly and not necessarily mechanically. A device or structure that is "designed" in a certain way is designed at least in that way, but may also be designed in ways not listed.

[0042] Various features, advantages and embodiments are set forth in the following claims.

Claims

[1] System (104) for controlling a vehicle (102), wherein the system (104) comprises: an electronic processor (200) designed to: To take, via a camera (112), a first image; Determine, within the first image, of road traffic factors which road lines (402A-402E) and at least one road user near the vehicle (102) are included; Generating, based on sensor information from one or more sensors (114) of the vehicle (102), a second image (400) depicting an environment surrounding the vehicle, wherein the second image (400) includes the road traffic factors; where the road lines (402A-402E) in the second image (400) have a visual pattern or color which indicates the designated direction of travel; Determine, based on the detected road traffic factors and the second image (400), a predicted trajectory (408A-408D) of the road user near the vehicle; wherein the trajectory (408A-408D) in the second image (400) is represented by a gradient of a visual pattern or color, where the lightest portion is the earliest position and the darkest portion is the latest position of the road user near the vehicle (102); Generating a steering command for the vehicle (102) based on the [2] System (104) according to claim 1, wherein the electronic processor (200) determines the predicted trajectory (408A-408D) based on the detected road factors and the second image (400) using a deep learning network (215). [3] System (104) according to claim 1, wherein the electronic processor (200) is further configured to determine the predicted trajectory (408A-408D) based on one or both of a speed and an acceleration of the road user. [4] System (104) according to claim 1, wherein the road traffic factors further comprise a traffic sign or a traffic signal system. [5] System (104) according to claim 1, wherein the electronic processor (200) is further configured to determine the predicted trajectory of the road user near the vehicle (102) based on a predicted trajectory (408A-408D) of another road user. [6] Method (300) for steering a vehicle (102), the method comprising the following: Taking (305), via a camera (112), a first image; Determine (308), within the first image, of road traffic factors which include road lines (402A-402E) and at least one road user near the vehicle (102); Generating (310), based on sensor information from one or more sensors (114) of the vehicle (102), a second image (400) depicting an environment surrounding the vehicle (102), wherein the second image (400) includes the road traffic factors; Determine (315), based on the detected road traffic factors and the second image (400), a predicted trajectory (408A-408D) of the road user near the vehicle; wherein the trajectory (408A-408D) in the second image (400) is represented by a gradient of a visual pattern or color, the lightest portion being the earliest position and the darkest portion being the latest position of the road user near the vehicle (102). Generating (320) a steering command for the vehicle (102) based on the predicted trajectory (408A-408D). [7] Method according to claim 6, wherein the predicted trajectory (408A-408D) is determined on the basis of the detected road factors and the second image (400) using a deep learning network. [8] Method according to claim 6, wherein the predicted trajectory (408A-408D) is determined on the basis of one or both of the speed and acceleration of the road user. [9] System according to claim 6, wherein the road traffic factors further comprise a traffic sign or a traffic signal system. [10] Method according to claim 6, wherein the predicted trajectory (408A-408D) of the road user near the vehicle (102) is determined on the basis of a predicted trajectory of another road user.

Images