Autonomous vehicle user interface with predicted trajectories

Abstract

The present disclosure relates to autonomous vehicle user interfaces with predicted trajectories. Systems and methods are provided for generating trajectories for a vehicle user interface showing a driver's perspective view. The method includes generating a host vehicle predicted trajectory for a host vehicle and generating at least one roadblocker predicted trajectory for a roadblocker located outside the host vehicle. After generating the predicted trajectories, the method continues with determining that at least one predicted trajectory overlaps with an object or another predicted trajectory when displayed on a user interface showing a driver's perspective view. The method includes modifying the at least one roadblocker predicted trajectory to eliminate the overlap. The method then continues with updating the display of the user interface to include any modified roadblocker predicted trajectories. The system includes a trajectory prediction module for performing the method.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to user interfaces for autonomous vehicles, and more particularly, to systems and methods for modifying and displaying predicted trajectories on a user interface providing a driver's viewgraph. BACKGROUND

[0002] The background description provided herein is for the purpose of generally presenting the context of the disclosure. To the extent that work of the presently named inventor(s) described in this background section is not otherwise fully conveyed by the claims, neither is it intended to be fully set forth as disclaimed prior art.

[0003] In various applications, a vehicle system can predict a trajectory of a vehicle, sometimes referred to herein as an "ego-vehicle." For example, a parallel autonomous vehicle, such as a vehicle including an advanced driver assistance system (ADAS), is a vehicle whose control can be shared between a human driver and an autonomous driving system. The human driver can retain control over certain aspects of driving such a vehicle (e.g., steering), while the ADAS monitors the driver's actions and intervenes as necessary to prevent accidents. Thus, predicting a trajectory of the ego-vehicle is an important aspect of such an ADAS. The vehicle system can display the ego-vehicle trajectory on a user interface.

[0004] The vehicle system can also predict trajectories of one or more road agents located outside the vehicle, and display the road agent trajectories on the user interface display. Examples of road agents include various types of other vehicles (e.g., cars, motorcycles, or bicycles) and pedestrians. One goal of an autonomous vehicle or a parallel autonomous vehicle is to travel a route without colliding with road agents encountered along the way. Since the driver of an autonomous vehicle or a parallel autonomous vehicle typically does not positively know the intentions of a road agent or its driver, predicting the trajectory of the road agent can facilitate this goal.

[0005] However, when many road agent trajectories are provided on a user interface, it can quickly become overly complex, especially when the display is provided as a viewgraph of the driver (e.g., as compared to an overhead plan view). Thus, it would be desirable to provide an improved trajectory prediction system to adequately account for the presence of numerous and / or overlapping trajectories that can be provided on a user interface, which can result in a less complex display. SUMMARY

[0006] This section provides a summary of the disclosure, and is not a comprehensive disclosure of the full scope or all the features of the present disclosure.

[0007] In various aspects, the present teachings provide a system for generating trajectories for a vehicle user interface showing a view of a driver's perspective. The system includes one or more processors and a memory communicably coupled to the one or more processors. The memory stores a trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform a series of steps. For example, the trajectory prediction module can include instructions for generating a host vehicle predicted trajectory for a host vehicle and generating at least one pedestrian predicted trajectory for a pedestrian located outside of the host vehicle. The instructions can include a step for determining that the at least one pedestrian predicted trajectory, when displayed on a user interface showing a view of a driver's perspective, extends out of a display area and has an unknown direction of travel. The instructions can include a step of modifying the at least one pedestrian predicted trajectory to provide an indication of the direction. A control module can also be provided that includes instructions that, when executed by the one or more processors, cause the one or more processors to update the user interface to include any modified pedestrian predicted trajectory.

[0008] In other aspects, the present teachings provide a system for generating trajectories for a vehicle user interface showing a view of a driver's perspective. The system includes one or more processors and a memory communicably coupled to the one or more processors. The memory stores a trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform a series of steps. For example, the trajectory prediction module can include instructions for generating a host vehicle predicted trajectory for a host vehicle and generating at least one pedestrian predicted trajectory for a pedestrian located outside of the host vehicle. The instructions can include a step for determining that the at least one predicted trajectory, when displayed on a user interface showing a view of a driver's perspective, overlaps with an object or another predicted trajectory. The instructions can include a step for modifying the at least one pedestrian predicted trajectory to eliminate the overlap. A control module can also be provided that includes instructions that, when executed by the one or more processors, cause the one or more processors to update the user interface to include any modified pedestrian predicted trajectory.

[0009] In other aspects, the present teachings provide a method for generating trajectories for a vehicle user interface showing a view of a driver's perspective. The method includes generating a host vehicle predicted trajectory for a host vehicle and generating at least one pedestrian predicted trajectory for a pedestrian located outside of the host vehicle. After generating the predicted trajectories, the method continues with determining that the at least one predicted trajectory, when displayed on a user interface showing a view of a driver's perspective, overlaps with an object or another predicted trajectory. The method includes modifying the at least one pedestrian predicted trajectory to eliminate the overlap. The method then continues with updating a display of the user interface to include any modified pedestrian predicted trajectory.

[0010] Other applicable fields and various methods for enhancing the above technology will become clear from the description provided herein. The description and specific examples in this invention are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0011] This teaching will be more fully understood based on the specific description and accompanying drawings, wherein:

[0012] Figure 1 This is a schematic diagram illustrating exemplary aspects of a vehicle in which the systems and methods disclosed herein according to the present technology can be implemented;

[0013] Figure 2 It's a diagram. Figure 1 A schematic diagram illustrating an exemplary aspect of the trajectory prediction system provided in the diagram;

[0014] Figure 3 The illustration shows a partial perspective view of an interior compartment of an exemplary vehicle interior compartment provided with multiple display systems according to various aspects of the present technology. The multiple display systems can be used individually or in combination to provide one or more user interface displays.

[0015] Figure 4A Figure 4E shows five example displays of a user interface with images representing the vehicle and at least one obstructing vehicle or pedestrian with corresponding trajectories.

[0016] Figures 5A to 5E The five illustrated user interface examples show the overlap of the trajectory and the vehicle, with the user interface displaying the relationship between the trajectory and the vehicle. Figure 4A Images of the vehicle and one or more roadblock vehicles or pedestrians and their corresponding trajectories in the same scene as the scene provided in Figure 4E, in a frontal perspective view or driver's view.

[0017] Figure 6 This is a flowchart of a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object, according to the illustrative aspects of this technology.

[0018] Figure 7 This is another flowchart of a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object, according to the illustrative aspects of this technology;

[0019] Figures 8A to 8C The illustration depicts a modification of at least one predicted trajectory based on the overlap with an object, according to an illustrative aspect of the present technology.

[0020] Figures 9A to 9C The illustration depicts a modification of at least one predicted trajectory based on the overlap of two objects, according to an illustrative aspect of the present technology.

[0021] Figure 10 is a flowchart of a method of modifying at least one predicted trajectory based on overlap with an object by hiding, diluting, and / or mixing the predicted trajectory, in accordance with various aspects of the present technology;

[0022] FIGS. 11A-11E illustrate various modifications of at least one predicted trajectory, in accordance with a method of Figure 10

[0023] Figure 12 is a flowchart of a method of modifying at least one predicted trajectory based on different overlap selecting different techniques, in accordance with various aspects of the present technology;

[0024] FIGS. 13A-13B illustrate modifications of at least one predicted trajectory, in accordance with a method of Figure 12

[0025] Figure 14 is a flowchart of a method of modifying at least one predicted trajectory based on overlap of at least two predicted trajectories, with optional use of a confidence score, in accordance with illustrative aspects of the present technology;

[0026] Figure 15 is a flowchart of a method of modifying at least one predicted trajectory based on a shortest distance between two adjacent predicted trajectories, in accordance with illustrative aspects of the present technology;

[0027] FIGS. 16A-16E illustrate modifications of at least one predicted trajectory, in accordance with a method of Figure 16C Figure 14

[0028] Figure 17 FIGS. 18A-18E illustrate user interface displays from a driver’s perspective view, in the case of an obstruction overlapping a predicted trajectory;

[0029] Figures 18A to 18E FIGS. 19A-19E illustrate modifications of at least one predicted trajectory, in accordance with a method of Figures 14 to 15 Figure 17

[0030] Figure 19 is a flowchart of a method of modifying at least one predicted trajectory based on overlap of at least two predicted trajectories, in the case of determining a priority intersection, in accordance with illustrative aspects of the present technology;

[0031] FIG. 20 illustrates a set of four vehicles with predicted trajectories, where four intersections have various priorities;

[0032] FIGS. 21A-21E illustrate modifications of at least one predicted trajectory, in accordance with a method of Figure 21B Figure 19

[0033] ​​​​​​​​ Figure 22 is a flowchart of a method of modifying at least one predicted trajectory based on overlap of the predicted trajectory with an object in the case where the object is determined to be a partial or complete roadblock, in accordance with illustrative aspects of the present technology;

[0034] Figures 23A to 23D illustrates modification of at least one predicted trajectory according to the method of Figure 22 ;

[0035] Figures 24A to 24D illustrates additional modification of at least one predicted trajectory according to the method of Figure 22 ;

[0036] Figure 25 is a flowchart of a method of modifying at least one predicted trajectory based on presence of a sloped terrain, in accordance with illustrative aspects of the present technology;

[0037] Figures 26A to 26D and Figures 27A to 27D illustrates modification of at least one predicted trajectory according to the method of Figure 25 ;

[0038] Figure 28 is a flowchart of a method of modifying at least one predicted trajectory based on at least one trajectory extending from a display area having an unknown travel direction, in accordance with illustrative aspects of the present technology;

[0039] Figures 29A to 29E illustrates modification of at least one predicted trajectory according to the method of Figure 28 ;

[0040] Figure 30 is a flowchart of a method of modifying at least one predicted trajectory based on overlap of the predicted trajectory with an object or another predicted trajectory in the case where the predicted trajectory is determined to overlap a plurality of static objects, in accordance with illustrative aspects of the present technology;

[0041] Figures 31A to 31C illustrates modification of at least one predicted trajectory according to the method of Figure 30 ;

[0042] Figure 32 is a flowchart of a method of modifying at least one predicted trajectory based on overlap of the predicted trajectory with an object or another predicted trajectory in the case where the predicted trajectory is determined to be a past tense roadblock trajectory, in accordance with illustrative aspects of the present technology;

[0043] Figures 33A to 33C illustrates modification of at least one predicted trajectory according to the method of Figure 32 ;

[0044] Figure 34is a flowchart of a method of selecting a type of display to be provided in a user interface based on complexity in accordance with illustrative aspects of the technology;

[0045] Figures 35A to 35C illustrates variations of predicted trajectories with different lines, 2D patterns, and 3D shapes;

[0046] Figures 36A to 36C illustrates 2D and 3D predicted trajectories in the case where two predicted trajectories overlap;

[0047] Figures 37A to 37C illustrates 2D and 3D predicted trajectories in the case where a predicted trajectory overlaps with at least one static object; and

[0048] Figures 38A to 38B illustrates the use of a combination of 2D and 3D predicted trajectories on a single display.

[0049] It should be noted that the appended drawings illustrate only typical aspects of methods, algorithms, and apparatuses within the scope of the technology and are thereby not to be considered limiting of its scope. The drawings can not precisely illustrate any given aspect and, as such, should not be entirely relied upon to determine the specifics of any given aspect. In addition, some aspects can incorporate features from combinations of the drawings. DETAILED DESCRIPTION

[0050] The technology described herein relates to improved displays of predicted roadblocks and own-vehicle trajectories on a user interface. In particular, the technology improves how these trajectories can interact with each other and be adjusted and / or simplified with respect to size, position, and various other details related to their display on the user interface. In this regard, the technology can simplify the trajectory information presented to the user. For example, it can provide an improved trajectory prediction system that provides displays that adequately explain the existence of numerous and / or overlapping trajectories that can be provided on the user interface and can result in less complex views.

[0051] As used herein, the term "trajectory" or "trajectories" can refer to simulated, predicted, or observed past, current, and future trajectories for a given roadblocker, vehicle, ego vehicle, etc. As used herein, the term "roadblocker" generally refers to any object that is capable of moving along or in a manner that intersects a roadway. These objects are not necessarily always in motion. For example, various aspects described herein treat a parked car along a street, a bus, a bicycle, other types of vehicles as roadblockers. In those aspects, the system can track the parked car using sensors of the vehicle along with other objects detected in the environment. The sensor data will generally reveal that the roadblocker (parked car) is stationary, at which point there is no trajectory associated with it that can be predicted. However, in those various aspects, the system can continue to track the parked car because it can start moving at any time. In various aspects, the roadblockers of interest are external to the vehicle (sometimes referred to herein as the "ego vehicle" or "host vehicle") in which aspects of the present technology operate. These roadblockers are sometimes referred to herein as "external roadblockers." Additional non-limiting examples of roadblockers include, but are not limited to, various types of other vehicles (cars, buses, motorcycles, bicycles, trucks, construction equipment, etc.), pedestrians, and animals. In some aspects, roadblockers can be referred to simply as objects.

[0052] In non-limiting aspects, a probabilistic variational trajectory predictor can be used to predict ego and / or roadblocker trajectories, which can be referred to as predicted trajectories. In those aspects, the trajectory probability distribution for any given roadblocker or ego vehicle that is applicable can be sampled to generate one or more specific predicted trajectories. As described above, those predicted trajectories can be cross-fed and iteratively updated between the ego vehicle and one or more roadblockers, and they can also be output to a control module of the ego vehicle that controls, at least to some extent, the output provided to various displays and user interfaces of the ego vehicle, as further described below. In some variational predictor aspects, the statistical parameters of the trajectory probability distribution can be output to the control module of the ego vehicle instead of specific trajectories sampled from the distribution.

[0053] Depending on the particular aspect, the predicted trajectories of the multiple external roadblockers can be considered to create the ego vehicle predicted trajectory in any of a number of possible orders. In one aspect, the roadblocker predicted trajectories are prioritized by their distance from the ego vehicle, with those roadblocker predicted trajectories closer to the ego vehicle receiving a higher priority than those further away. In another aspect, the roadblocker predicted trajectories are prioritized by any uncertainty associated with the roadblocker predicted trajectories, with those roadblocker predicted trajectories having less uncertainty (i.e., greater certainty) receiving a higher priority than those having greater uncertainty (i.e., lower certainty). Moreover, in the case where all possible orders of external roadblockers are considered, the intermediate predicted trajectories for the ego vehicle and / or one or more external roadblockers during the iterative trajectory prediction process can be saved, collected, and aggregated. Saving all of these various hypotheses allows the control module of the ego vehicle to consider all possible actions that a roadblocker can take. This saving approach facilitates the goal of the ego vehicle planning and traversing a safe trajectory.

[0054] Other techniques can be advantageously combined with the above-described iterative trajectory prediction architecture: (1) employing multiple trajectory predictors to predict the future trajectory of the ego vehicle, as well as employing multiple trajectory predictors to predict the future trajectory of one or more roadblockers located outside of the ego vehicle; and (2) generating a confidence estimate for the predicted ego vehicle and roadblocker trajectories, such that their trustworthiness can be assessed. These techniques will be further explained in the following paragraphs.

[0055] To predict the future trajectory of the ego vehicle or a given external roadblocker, some aspects described herein employ two or more trajectory predictors that use different deterministic or probabilistic computational models. For example, in one aspect that includes two trajectory predictors, a first trajectory predictor is a probabilistic variational trajectory predictor that includes a DNN, and a second trajectory predictor is a physics-based (deterministic) model. In various aspects, the trajectory predictors receive as input any of the various vehicle sensor data discussed further below. Depending on the particular aspect, the trajectory predictors can also receive measured past trajectory information for the ego vehicle or roadblocker, depending on which type of trajectory is being predicted.

[0056] With respect to confidence estimates, one important aspect in the disclosed aspects is the temporal (time) horizon within which a vehicle or road user trajectory is predicted. For example, a given predicted trajectory from a particular trajectory predictor can be trustworthy within a relatively short temporal horizon of from about 0.1 seconds to about 3 seconds, but it can not be trustworthy within a longer temporal horizon extending beyond about 3 seconds up to about 10 seconds. In some aspects, a deep neural network (DNN) model is used to compute confidence estimates for the ego and road user predicted trajectories as a continuous function of time within the applicable temporal horizon. Thus, the confidence measure helps the trajectory prediction system decide which ego or road user predicted trajectories are most trustworthy for a particular segment of the overall temporal prediction horizon. In various aspects, the confidence scores associated with the iteratively updated ego and road user predicted trajectories are also iteratively updated as the predicted trajectories themselves are iteratively updated.

[0057] Reference is made to Figure 1 , which illustrates an example of a vehicle 100 (sometimes referred to herein as an “ego vehicle”). As used herein, a “vehicle” is any form of motorized conveyance. In one or more implementations, the vehicle 100 is an automobile. Although arrangements will be described herein with respect to an automobile, it will be understood that the aspects are not limited to automobiles. In some implementations, the vehicle 100 can be any other form of motorized conveyance that is capable of operating at least semi-autonomously, for example.

[0058] The vehicle 100 also includes various elements. It will be understood that, in various aspects, the vehicle 100 can not necessarily have all of the elements illustrated in Figure 1 . The vehicle 100 can have any combination of the various elements illustrated in Figure 1 . In addition, the vehicle 100 can have additional elements beyond those illustrated in Figure 1 . In some arrangements, the vehicle 100 can be implemented without one or more of the elements illustrated in Figure 1 . Although various elements are illustrated in Figure 1 as being located within the vehicle 100, it will be understood that one or more of these elements can be located external to the vehicle 100. In addition, the illustrated elements can be physically separated by large distances. In Figure 1 illustrated and will be described along with subsequent Figure 1 some of the possible elements of the vehicle 100. However, for the sake of brevity of the description, the description of many of the elements in Figure 1 will be provided after the remaining figures have been discussed.

[0059] Reference is made to Figure 2 , which illustrates Figure 1an exemplary trajectory prediction system 170. The trajectory prediction system 170 is implemented to perform methods and other functionality as disclosed herein relating to controlling operations of the vehicle 100 based at least in part on past, current, observed, or predicted future trajectories of the vehicle 100 itself and / or based on past, current, observed, or predicted trajectories of one or more roadblocks located outside of the vehicle 100. In some aspects, the trajectories of the vehicle 100 or roadblocks can be modeled in three-dimensional space.

[0060] The trajectory prediction system 170 is shown to include one or more processors 110 of the vehicle 100 from Figure 1 Depending on the embodiment, the one or more processors 110 can be part of the trajectory prediction system 170, the trajectory prediction system 170 can include one or more processors separate from the one or more processors 110 of the vehicle 100, or the trajectory prediction system 170 can access the one or more processors 110 over a data bus or another communication path. In one aspect, the trajectory prediction system 170 includes a memory 172 that stores at least a trajectory prediction module 174 and a control module 176. The memory 172 can be a random access memory (RAM), read only memory (ROM), hard disk drive, flash memory, or other suitable memory for storing the modules 174, 176. The modules 174, 176 are, for example, computer-readable instructions that, when executed by the one or more processors 110, cause the one or more processors 110 to perform various functionality disclosed herein.

[0061] In connection with predicting trajectories of the vehicle 100, the trajectory prediction system 170 can store various kinds of model-related data 178 in a database 180. As shown in Figure 1 The trajectory prediction system 170 can receive sensor data from the sensor system 120 in the vehicle 100 (ego vehicle), as shown in FIG. 1. For example, in some aspects, the trajectory prediction system 170 receives image data from one or more cameras 126. Depending on the particular embodiment, the trajectory prediction system 170 can also receive LIDAR data from LIDAR sensors 124, radar data from radar sensors 123, and / or sonar data from sonar sensors 125. In some aspects, the trajectory prediction system 170 also receives inputs from the vehicle systems 140. Examples include, but are not limited to, steering wheel angle, throttle pedal (accelerator) position, linear velocity, and angular velocity. The steering wheel angle and throttle pedal position data can be examples of what can be referred to as controller area network (CAN bus) data, and the linear velocity and angular velocity can be examples of what can be referred to as inertial measurement unit (IMU) data. As explained further below, certain of the above types of sensor data are relevant to predicting trajectories of the vehicle 100 (ego vehicle), but not to predicting trajectories of external roadblocks. Additionally, as explained further below, certain of the above types of sensor data are relevant to predicting trajectories of external roadblocks, but not to predicting trajectories of the vehicle 100 (ego vehicle). Figure 1 As indicated, the trajectory prediction system 170, and in particular the control module 176, can communicate with the vehicle systems 140 and / or the autonomous driving module(s) 160 to assist in semi-autonomous or autonomous control of various functions of the vehicle 100. The control module 176 also includes instructions that cause the one or more processors 110 to control operation of the user interface system 182 and coordinate data including predicted trajectories provided to various displays throughout the vehicle 100.

[0062] In some aspects, other or additional kinds of data from the sensor system 120 can be fed to the trajectory prediction system 170, such as radar and / or sonar data. Additionally, more highly structured data such as rasterized map data (e.g., an occupancy grid for the environment surrounding the vehicle 100) can be fed to the variational trajectory predictor. The specific kinds of raw sensor data or structured data fed to the trajectory prediction system 170 can vary depending on the aspect.

[0063] In some aspects described below that include a confidence score, the confidence score can be computed while the predicted ego and roadblocker trajectories are being iteratively updated based at least in part on a number of iterations that occur between the prediction of the ego trajectory and the prediction of the roadblocker trajectory. Generally, a greater number of iterations corresponds to a higher level of confidence in the resulting predicted trajectory, as after enough iterations the predicted trajectories tend to converge to a more stable prediction.

[0064] As described in detail herein, the trajectory prediction module 174 generally includes instructions that cause the one or more processors 110 to produce for display on a user interface one or more predicted trajectories for the vehicle 100 (ego) and one or more predicted trajectories for at least one external roadblocker. Various user interface designs can be useful for displaying trajectory information regarding the present technology, and the description provided herein is not meant to limit the types of displays useful to the present technology.

[0065] Figure 3A partial perspective view of an exemplary vehicle interior compartment 50 is provided, illustrating two front row seats for vehicle passengers and various vehicle control devices. As can be seen, the vehicle includes a navigation display 52 and a heads-up display (HUD) 54 projected on a windshield 56, the HUD 54 having a plurality of panels 58 that can accommodate displays for user interfaces. Multiple information displays (MIDs) 60 located in various areas of the vehicle interior compartment can also be used, such as screens / displays that can be switched between different information displays. In other aspects, personal electronic devices such as telephones 62, tablets (not shown), and the like can also be used for display purposes. Many variations of the architecture just described are possible depending on the particular aspect. In various aspects, the systems and methods provided herein can include using being provided as a car, motorcycle, bicycle, and / or pedestrian; and the vehicle user interface is one of a navigation display, a multiple information display, a heads-up display (HUD), a head-mounted display (HMD), a remote operator display, and a wearable device. Multiple displays can be used in combination with each other and can include different perspectives.

[0066] Figure 4A To Fig. 4E is an image illustrating five example displays of a user interface having an overhead plan view 300 of the ego vehicle 100 and at least one road user such as a vehicle 200 or a pedestrian 204 with corresponding trajectories 202, 206. Figures 5A to 5E is an image illustrating five example displays of a user interface showing overlap of trajectories with the vehicle, the user interface having an overhead plan view 300 of the ego vehicle 100 and at least one road user such as a vehicle 200 or a pedestrian 204 with corresponding trajectories 202, 206. Figure 4A To Fig. 4E is an image illustrating five example displays of a user interface having an overhead plan view 300 of the ego vehicle 100 and at least one road user such as a vehicle 200 or a pedestrian 204 with corresponding trajectories 202, 206.

[0067] For example, Figure 4A illustrates an overhead plan view 300 with two road user vehicles 200. Although in Figure 4A the road user predicted trajectories 202 do not overlap, in Figure 5A the road user predicted trajectories 202 approach each other with a portion of overlap that can cause confusion to the user, in the driver’s perspective 310. Figure 4B To Fig. 4D illustrates an overhead plan view 300 of the ego vehicle 100 with ego predicted trajectories 102 and multiple road user vehicles 200 and their corresponding road user predicted trajectories 202 in different traffic patterns. As shown, the various predicted trajectories 102, 202 not only overlap with each other, but they also overlap with certain of the road user vehicles themselves, potentially causing confusion to the user. Figures 5B to 5DThe illustration shows the driver's view 310 of these vehicles 100, 200 and a similar overlap of predicted trajectories 102, 202, also providing a complex visualization that could potentially interfere with visibility and / or cause confusion for the user. Figure 4E and Figure 5E The illustration shows the predicted trajectory 102 and the vehicle 100 adjacent to a pedestrian crossing, which has multiple pedestrians 204 and their corresponding predicted trajectories 206, with various overlaps.

[0068] Figure 6 This is a flowchart of a method 320 for modifying at least one predicted trajectory based on the overlap of a predicted trajectory with an object, according to an illustrative aspect of the present technology. Method 320 is used to generate a trajectory for a vehicle user interface displaying a driver's view. The method first includes generating a predicted trajectory 102 for the vehicle 100 and generating at least one predicted trajectory 202 for an obstacle located outside the vehicle 100. The obstacle (one or more) can be another vehicle 200, a pedestrian 204, or a combination thereof. As indicated in method step 322, after generating the corresponding predicted trajectories 102, 202, method 320 continues by determining that at least one predicted trajectory 202 for an obstacle exists behind an object when viewed from the driver's perspective, indicating travel behind the object. As indicated in method step 324, the method further determines whether the predicted trajectory 202 overlaps with an object when displayed on the user interface displaying the driver's view. Unless otherwise stated, the term "object" used in conjunction with the methods described herein may broadly include: static objects, such as parked vehicles, buildings, medians, etc.; and / or may also include moving objects, which may include moving vehicles, moving pedestrians, etc. Method 320 then includes modifying at least one predicted obstacle trajectory 202 to eliminate overlap, as indicated in method step 326. The method then continues to update the display of the user interface to include any modified predicted obstacle trajectory(s). (Revisit) Figure 2 In various aspects, the control module 176 can be used to provide instructions to one or more processors 110 and / or user interface system 182 to update the user interface to include the display of any modified roadblock prediction trajectory.

[0069] Figure 7 This is a flowchart of method 328, which modifies at least one predicted trajectory based on the overlap between the predicted trajectory and the object. Method 328 and Figure 6 Method 320 is similar but has additional features. For example... Figure 7 As shown, there is an additional method step 330: determining whether the object is an obstacle with its own predicted trajectory. If so, no modification is made. Otherwise, modification is made.

[0070] Figures 8A to 8C illustrates a modification of at least one predicted trajectory based on overlap with an object according to Figures 6 to 7 the method of Figure 8A is a top plan view 300 providing two blocker vehicles 200A, 200B each having a respective blocker predicted trajectory 202A, 202B, the blocker predicted trajectories 202A, 202B having arrows for indicating a direction of travel. The blocker vehicles 200A, 200B are traveling in opposite directions. From the perspective of the host vehicle driver, one vehicle 200A will travel behind the object 208, and one vehicle 200B will travel in front of the object 208. Figure 8B provides a driver's perspective 310 of the situation as presented in Figure 8A As shown in Figure 8B the blocker predicted trajectory 202A can be confusing because it overlaps with the object 208, and it appears that the vehicle 200A will travel in front of the object 208, when in fact the vehicle 200A will travel behind the object 208. Figure 8C illustrates the resulting modification: the length of the blocker predicted trajectory 202A is shortened so as to no longer overlap with the object 208.

[0071] Figures 9A to 9C illustrates a modification of at least one predicted trajectory based on overlap with two objects according to illustrative aspects of the present technology. Figure 9A is a top plan view 300 providing two blocker vehicles 200A, 200B each having a respective blocker predicted trajectory 202A, 202B, the blocker predicted trajectories 202A, 202B having arrows for indicating a direction of travel. The blocker vehicles 200A, 200B are traveling in opposite directions. From the perspective of the host vehicle driver, one vehicle 200A will travel behind the first object 208, and one vehicle 200B will travel in front of the first object 208 but behind the second object 110. Figure 9B provides a driver's perspective 310 of the situation as presented in Figure 9A As shown in Figure 9B the blocker predicted trajectories 202A, 202B can be confusing because they both overlap with the objects 208, 210, and it appears that both vehicles 200A, 200B will travel in front of the objects 208, 210, when in fact the vehicle 200A will travel behind the first object 208, and the vehicle 200B will travel in front of the first object 208 and behind the second object 210. Figure 9C illustrates the resulting modification: the lengths of the two blocker predicted trajectories 202A, 202B are shortened so as to no longer overlap with the objects 208, 210.

[0072] Figure 10is a flowchart of a method 332 of modifying at least one blocker predicted trajectory based on overlap with an object by hiding, diluting, and / or mixing the predicted trajectory in accordance with various aspects of the technology. Method step 334 determines whether the blocker predicted trajectory overlaps with the object when displayed on the user interface in the driver's perspective. Similar to method 320 of FIG. 8, the method 332 also determines whether the blocker predicted trajectory is partially or completely behind the object, being behind the object indicating travel behind the object, as illustrated in method step 336. If so, the method 332 proceeds to modify the display of the object and the predicted trajectory. For example, a portion or all of the blocker predicted trajectory can be modified so as to be hidden by the object or behind the object, as illustrated in method step 338. If not, the method 332 proceeds to modify a portion or all of the predicted trajectory to dilute, mix, etc., as illustrated in method step 340, in order to better enable the user to understand the precedence in the situation. In other aspects, the method includes instructions to modify the display of the object so that a first portion of the blocker predicted trajectory appears to be hidden behind the object when displayed on the user interface showing the driver's perspective map, and to modify a second portion of the blocker predicted trajectory using at least one technique selected from a group comprising hiding, diluting, and mixing.

[0073] FIGS. 11A-11E illustrate a method 1100 of modifying at least one blocker predicted trajectory based on overlap with an object by hiding, diluting, and / or mixing the predicted trajectory in accordance with various aspects of the technology. Figure 10various modifications of the at least one blocker predicted trajectory of method 332. FIG. 11A is a top plan view 300 providing two blocker vehicles 200A, 200B each having a corresponding blocker predicted trajectory 202A, 202B with an arrow for indicating the direction of travel. The blocker vehicles 200A, 200B are traveling in opposite directions. From the perspective of the host vehicle driver, one vehicle 200A will be traveling behind the object 208 and one vehicle 200B will be traveling in front of the object 208. FIG. 11B provides the driver’s perspective 310 of the situation as presented in FIG. 11A. As shown in FIG. 11B, the blocker predicted trajectory 202A can be confusing because it overlaps the object 208 and it appears that the vehicle 200A will be traveling in front of the object 208 when, in fact, the vehicle 200A will be traveling behind the object 208. FIG. 11C provides a resulting first modification: both blocker predicted trajectories 202A, 202B are modified so that a portion of each is hidden behind the object 208. This provides the user with a clearer view of the object 208, however, there can be ambiguity as to whether the second vehicle 200B is traveling in front of the object 208 or behind the object 208. FIG. 11D and FIG. 11E can provide the user with a clearer understanding of the situation. In FIG. 11D, the blocker predicted trajectory 202A of the blocker vehicle 200A traveling behind the object is partially hidden behind the object 208, while the blocker predicted trajectory 202B of the blocker vehicle 200B traveling in front of the object 208 is partially blended with the object 208. In FIG. 11E, the blocker predicted trajectory 202A of the blocker vehicle 200A traveling behind the object is partially diluted behind the object 208, while the blocker predicted trajectory 202B of the blocker vehicle 200B traveling in front of the object 208 is partially blended with the object 208. Different combinations of hiding, blending, and diluting can be used.

[0074] Figure 12 is a flowchart of a method 342 of different techniques for modifying at least one predicted trajectory based on different overlaps in accordance with various aspects of the present technology. As illustrated in method step 344, the method determines the front-back relationship between various blockers and objects displayed in the user interface display having the driver’s perspective. The method then determines whether the target object overlaps another object as illustrated by method step 346. If there is an overlap, the method determines whether another blocker reaches the point of overlap before the host vehicle reaches the point of overlap as guided by step 350 of the method. If so, the method guides the use of the method as provided in Figure 7 Figure 10

[0075] ​​Figures 13A-13B illustrate modification of at least one of the predicted trajectories according to the method 342 of Figure 12 Figures 13A-13B illustrate modification of at least one of the predicted trajectories according to the method 342 of Figure 12 Figures 13A-13B illustrate modification of at least one of the predicted trajectories according to the method 342 of Figure 7 Figures 13A-13B illustrate modification of at least one of the predicted trajectories according to the method 342 of Figure 10 Figures 13A-13B illustrate modification of at least one of the predicted trajectories according to the method 342 of

[0076] Figure 14is a flowchart of a method 354 of modifying at least one predicted trajectory based on overlap of at least two predicted trajectories in a driver's perspective view map in optional use of a confidence score in accordance with illustrative aspects of the present technology. The method 354 includes generating a host vehicle predicted trajectory for a host vehicle and generating at least one pedestrian predicted trajectory for a pedestrian located outside of the host vehicle. After generating the predicted trajectories, the method continues with determining that the at least two predicted trajectories overlap when displayed on a user interface showing a perspective view map of a driver, as indicated by method step 356. In various aspects, the method 356 can proceed directly to step 364, which includes modifying the at least one pedestrian predicted trajectory to eliminate the overlap. The method then continues with updating the display of the user interface to include any modified pedestrian predicted trajectories. In an optional method as indicated by method step 358, the trajectory prediction module 170 can include instructions for calculating or otherwise obtaining a confidence score that represents a likelihood of a collision between pedestrians due to the presentation of the overlap of the pedestrian predicted trajectories. The method can also include performing a comparison of the confidence score to determine that the confidence score is less than a predetermined threshold, as indicated by method step 360. Once it is determined that the risk of collision is less than the predetermined threshold, the method optionally continues with step 362 of determining whether another pedestrian will reach the point of overlap of the predicted trajectories before the host vehicle will reach the intersection. If so, the method includes modifying the at least one predicted trajectory to eliminate the overlap, as indicated by method step 364. In various aspects, the modification can include shortening a length of the predicted trajectory, providing a separation distance between at least two of the predicted trajectories, and modifying the predicted trajectory using techniques such as hiding, diluting, blending, or similar modification of at least a portion of the predicted trajectory (and / or adjacent pedestrians or objects).

[0077] Figure 15is a flowchart of a method 366 of modifying at least one predicted trajectory based on a calculation that the shortest distance between two adjacent predicted trajectories is below a threshold value, in accordance with illustrative aspects of the present technology. For example, the trajectory prediction module 170 can include instructions to generate a host vehicle predicted trajectory for the host vehicle and to generate at least one pedestrian predicted trajectory for a pedestrian located outside of the host vehicle. As shown in method steps 368 and 370, the instructions can include steps to determine that the distance between two adjacent predicted trajectories, when displayed on a user interface showing a driver’s perspective view map, is below a predetermined threshold value. Thereafter, as shown in method step 372, the instructions can include steps to perform at least one modification selected from a group including: (1) changing a color of at least one of the pedestrian predicted trajectories; (2) changing a spacing position of at least one of the pedestrian predicted trajectories; (3) changing a thickness of at least one of the pedestrian predicted trajectories; and (4) determining a priority pedestrian predicted trajectory based on a closest proximity to the host vehicle and displaying only the priority pedestrian predicted trajectory. The control module 176 can also provide instructions that, when executed by the one or more processors 110, cause the one or more processors 110 or the user interface system 182 to update the user interface to include any modified pedestrian predicted trajectories.

[0078] FIGS. 16A to Figure 16C illustrate modifications of at least one pedestrian and host vehicle predicted trajectories in accordance with Figure 14 FIG. 16A illustrates a top plan view 300 of a host vehicle 100 with its predicted trajectory 102 and two pedestrian vehicles 200A, 200B and their respective predicted trajectories 202A, 202B. As shown, the host vehicle predicted trajectory 102 overlaps the pedestrian predicted trajectory 202B and the other pedestrian predicted trajectory 202A overlaps the host vehicle 100. FIG. 16B provides a driver’s perspective 310 that completely hides (eliminates) the pedestrian vehicle 200A and its predicted trajectory 202A and shortens the length of the host vehicle predicted trajectory 102 to eliminate the overlap of the predicted trajectories 102, 202B. Figure 16C A driver’s perspective 310 is provided that completely hides (eliminates) the pedestrian vehicle 200A and its predicted trajectory 202A and blends the color of the host vehicle predicted trajectory 102 to minimize the existence of the overlap of the predicted trajectories 102, 202B.

[0079] Figure 17 illustrates a user interface display from a driver’s perspective 310 with the pedestrian 200A, 200B overlapping the predicted trajectories 202A, 202B. Figures 18A to 18E illustrates a user interface display from a driver’s perspective 310 with the pedestrian 200A, 200B overlapping the predicted trajectories 202A, 202B. Figures 14 to 15the method of Figure 17 the modification of at least one predicted trajectory. For example, in Figures 18A to 18B the color, gradient, or pattern of one blocker predicted trajectory 202A can be changed to make it appear different from another blocker predicted trajectory 202B. In Figure 18C the blocker predicted trajectories 202A, 202B can be provided with different thicknesses. In Figure 18D one of the predicted trajectories 202A can be shifted or separated by a distance "a" to provide a greater separation between adjacent predicted trajectories 202A, 202B. In Figure 18E one of the predicted trajectories 202A can be completely eliminated (hidden) from view.

[0080] Figure 19 is a flowchart of a method 374 of modifying at least one predicted trajectory based on overlap of at least two predicted trajectories or one or more predicted trajectories with one or more objects in the case of a priority intersection in accordance with illustrative aspects of the present technology. After the desired predicted trajectories are generated, the method includes determining whether at least two of the predicted trajectories (or trajectories and objects) overlap at a first intersection (as shown by method step 376), and determining whether at least two of the predicted trajectories (or trajectories and objects) overlap at a second intersection (as shown by method step 378). When at least two overlaps are located, method step 380 provides for a determination of a priority intersection. The determination is based on a calculation of which blockers will reach one of the first and second intersections first. Once a priority intersection is located, the method includes modifying at least one predicted trajectory that overlaps the priority intersection as shown by method step 382.

[0081] To further explain the method 374 of Figure 19 , FIG. 20 is provided to illustrate a set of four vehicles with predicted trajectories that overlap at four intersections with various intersection times. FIG. 20 includes four intersections labeled A, B, C, and D. Assume that the vehicles are traveling at the same speed, the priority intersection here is at intersection B which will occur first in time. Intersection C should occur last in time, with intersections A and D occurring at some time between B and C.

[0082] FIGS. 21A through Figure 21B further illustrate the method 374 of Figure 19of the method of FIG. 21A. FIG. 21A is a top plan view 300 of the host vehicle 100, the blocker vehicle 200, and two pedestrians 204 each having a respective predicted trajectory 102, 202, 206A, 206B. Two stop signs 214 are provided at the intersection. Because the blocker vehicle 200 is at the stop sign, depending on the speed of the pedestrians, the priority intersection is between the host vehicle 100 and one of the pedestrians 204A, 204B. The last in time intersection will be between the host vehicle 100 and the blocker vehicle 200. Figure 21B A driver's perspective 310 of the scenario presented in FIG. 21A is provided and the host vehicle predicted trajectory 102 has been modified by shortening the length of the host vehicle predicted trajectory 102 because the host vehicle predicted trajectory 102 involves the priority intersection.

[0083] Figure 22 is a flowchart of a method 384 of modifying at least one predicted trajectory based on overlap of the predicted trajectory with an object in the case where the object is a partially or fully hidden blocker in accordance with illustrative aspects of the present technology. After generating the desired predicted trajectories, the method includes a step 386 of determining that a blocker predicted trajectory exists behind the object when considering the driver's perspective, existing behind the object indicating travel behind the object. As shown in method step 388, the method determines that the blocker predicted trajectory overlaps the object when displayed on the user interface. If it is determined in method step 390 that at least one hidden blocker is located behind the object when displayed on the user interface showing the driver's perspective map, step 392 provides for performing at least one modification selected from the group consisting of: eliminating both the hidden blocker and the corresponding blocker predicted trajectory; hiding, diluting, or mixing the blocker predicted trajectory with the object; and superimposing the hidden blocker over the object.

[0084] Figures 23A to 23D An example scenario requiring modification of at least one predicted trajectory in accordance with Figure 22 is provided. For example, Figure 23A A top plan view 300 is provided including a smaller vehicle 200A positioned adjacent to a larger vehicle 200B such as a truck. Figure 23B A top plan view 300 is provided including a vehicle 200 adjacent to an object such as a building 212. Figure 23C and Figure 23D A driver's perspective 310 of the scenario presented in Figure 23A and Figure 23B is provided, where the blocker vehicle is not seen. Figures 24A to 24D An additional modification of at least one predicted trajectory in accordance with Figure 23C and Figure 23D is illustrated when the scenario of Figure 22 is presented. Figure 24A Both the hidden blocker vehicle and the corresponding blocker predicted trajectory are eliminated for both cases. Figure 24B The hidden blocker is eliminated, but the predicted trajectory is still rendered. Figure 24C The predicted trajectory is blended with the blocker vehicle 200B and the building 212. Figure 24D The hidden blocker 200, 200A is respectively overlaid with the building 212 and the blocker vehicle 200B.

[0085] Figure 25 is a flowchart of a method 394 of modifying at least one predicted trajectory based on the presence of a sloped terrain in a roadway in accordance with illustrative aspects of the present technology. The method 394 is for generating trajectories for a vehicle user interface showing a driver’s perspective view of the vehicle, and includes generating a host vehicle predicted trajectory for the host vehicle, and generating at least one blocker predicted trajectory for a blocker located outside of the host vehicle. As indicated by method step 396, after generating the respective predicted trajectories, the method 394 continues with determining that the at least one blocker predicted trajectory is present behind an object when viewed in the driver’s perspective, present behind an object indicating travel behind the object. The method also determines that the blocker predicted trajectory overlaps with the object when displayed in the driver’s perspective. In particular, in this method aspect, the object is determined to be a sloped terrain, and the predicted trajectory is at least partially hidden, as indicated by method step 400. As indicated by method step 402, the method continues with modifying the at least one blocker predicted trajectory for display as a curved trajectory extending over the sloped terrain, such that it is visible when displayed on the user interface showing the driver’s perspective. The method then continues with updating the display of the user interface to include any modified blocker predicted trajectory(s). Figure 26A As shown in FIG. 4, the sloped terrain 214 can be an uphill or downhill slope. As indicated by method step 402, the method continues with modifying the at least one blocker predicted trajectory for display as a curved trajectory extending over the sloped terrain, such that it is visible when displayed on the user interface showing the driver’s perspective. The method then continues with updating the display of the user interface to include any modified blocker predicted trajectory(s).

[0086] Figures 26A to 26D and Figures 27A to 27D illustrates the modification of at least one predicted trajectory in the case of a sloped terrain interfering with the field of view, in accordance with the method of Figure 25 Figure 26A is a partial perspective view of an example scenario with a host vehicle 100 and a blocker vehicle 200 with its predicted trajectory 202. Figure 26B a side view planar view is provided of the elevation difference, showing in particular the sloped terrain 214, and where the blocker predicted trajectory 202 is pointing in an upward direction due to the slope, which is undesirable. Figure 26C a top view planar view 300 is provided, while Figure 26D ​A driver's perspective 310 is provided, which has the blocker vehicle 200 hidden and only a portion of the predicted trajectory 202 shown, potentially leading to confusion for the user both as to the blocker's position and its direction of travel. Figure 27A and Figure 27B It is illustrated how the predicted trajectory of the blocker vehicle 200 can be transformed from a straight line 202 into a curved or at least partially curved trajectory 203, which can extend at a substantially fixed distance from the sloped terrain (over which) such that it does not indicate a direction that appears to be upwards to the sky or downwards to the road. For example, if the distance between the trajectory 202 and the road / terrain becomes increasingly larger than a predetermined threshold (indicating travel upwards), at least a portion of the predicted trajectory can be curved 203 in a downwards direction, as illustrated in Figure 27A In another example, if the distance between the trajectory 202 and the road / terrain becomes increasingly smaller than another predetermined threshold, at least a portion of the predicted trajectory can be curved 203 in an upwards direction, as illustrated in Figure 27B Figure 27C It is illustrated a driver's perspective 310 of a scene with a curved predicted trajectory 203. Figure 27A Figure 27D It is also provided an overlaid representation of the blocker vehicle 200, which can be provided with a color and / or shape to indicate being hidden, located on the other side of the slope, etc.

[0087] Figure 28 is a flowchart of a method 404 of modifying at least one predicted trajectory based on at least one blocker predicted trajectory that would otherwise provide a user with an unknown direction of travel of a blocker extending out of a display area, in accordance with illustrative aspects of the present technology. The method 404 is for generating trajectories for a vehicle user interface showing a driver's perspective view, and comprises: generating a host vehicle predicted trajectory for a host vehicle; and generating at least one blocker predicted trajectory for a blocker located outside the host vehicle. As indicated by method step 406, after generating the respective predicted trajectories, the method 404 can use a trajectory prediction module having instructions for determining that the at least one blocker predicted trajectory extends out of a display area when displayed on the user interface showing the driver's perspective view and further has an unknown direction of travel, as shown by method step 408. The method can comprise a step 410 for modifying the at least one blocker predicted trajectory to provide an indication of the direction in the display area.

[0088] Figures 29A to 29E It is illustrated a modification of at least one predicted trajectory in accordance with the method of Figure 28 Figure 29A It is illustrated a top plan view 300 with a host vehicle 100 and a blocker vehicle 200. Both the blocker vehicle 200 and the arrow of direction extend a certain distance out of the display.​​​ Figure 29B The scenario is illustrated from the perspective 310 of the driver. As shown, only a portion of the blocker predicted trajectory 202 is present, which causes confusion to the user about the direction of travel. Figure 29C An icon representing at least a partial view of the blocker vehicle 200 is added, however this can still cause confusion to the user due to the lack of a directional arrow. Figure 29D And Figure 29E The length of the blocker predicted trajectory 202 is modified and a directional arrow is added to the appropriate end of the predicted trajectory in order to provide additional information to the user when viewed from the perspective of the driver.

[0089] Figure 30 is a flowchart of a method 412 of modifying at least one predicted trajectory based on overlap of the predicted trajectory with an object or another predicted trajectory in the event that the predicted trajectory overlaps with one or more static objects in accordance with illustrative aspects of the present technology. The method 412 is used to generate trajectories for a vehicle user interface showing a perspective view of the driver and includes generating a host vehicle predicted trajectory for a host vehicle and generating at least one blocker predicted trajectory for a blocker located outside of the host vehicle. As indicated by method step 414, after the respective predicted trajectories are generated, the method 412 continues with determining that the at least one blocker predicted trajectory overlaps with an object or another blocker predicted trajectory. The method continues to step 416 to determine whether the blocker predicted trajectory overlaps with one or more static objects, in particular static objects that do not reside on a roadway or a sidewalk. Non-limiting examples of such static objects can include a row of trees, a series of buildings or structures, etc. Depending on the size and location of the plurality of static objects, the plurality of static objects can be considered as a group or individually. As shown in method step 418, the method includes modifying the blocker predicted trajectory by obscuring, diluting, or blending the blocker predicted trajectory at one or more of the locations where the predicted trajectory overlaps with one or more of the static objects.

[0090] Figures 31A to 31C illustrates the modification of at least one predicted trajectory adjacent to a plurality of static objects in accordance with the method of Figure 30 . Figure 31A A top plan view 300 of the host vehicle 100 and the blocker vehicle 200 is provided. The predicted trajectory 202 is in front of a plurality of static objects 216. When presented in the perspective 310 of the driver as shown in Figure 31B , the relationship between the blocker predicted trajectory 202 and the static objects 216 can appear confusing. Accordingly, as shown in Figure 31CAs shown in the middle, the roadblocker predicted trajectory 202 is modified such that the area / location of the predicted trajectory 202 that overlaps with the object 216 is provided as blended with the object 216. In other aspects, depending on the type of static object and optionally other factors, those portions can be diluted and / or hidden.

[0091] Figure 32 is a flowchart of a method 420 of modifying at least one predicted trajectory based on overlap of the predicted trajectory with an object or another predicted trajectory in the case where the predicted trajectory is a past-time roadblocker trajectory in accordance with illustrative aspects of the present technology. Typically, the roadblocker predicted trajectory is a current and / or future trajectory. However, in various aspects, it can be beneficial or desirable to provide an indication of at least one past-time trajectory or a roadblocker trajectory that has traversed a portion of the roadway and no longer poses a threat of collision with the host vehicle. The method 420 is for generating trajectories for a vehicle user interface showing a driver's perspective map, and includes generating a host vehicle predicted trajectory for the host vehicle, and generating at least one roadblocker predicted trajectory for a roadblocker located outside of the host vehicle. As indicated by method step 422, after generating the respective predicted trajectories, the method 420 continues with determining that the at least one roadblocker predicted trajectory overlaps with an object or another roadblocker predicted trajectory. The method continues to step 424 to determine that the roadblocker predicted trajectory is in fact a past-time roadblocker predicted trajectory. In other words, traversal across the roadway has already occurred. As shown in method step 426, the method includes modifying the past-time roadblocker predicted trajectory, e.g., by modifying or altering the shape or thickness of the past-time roadblocker predicted trajectory.

[0092] Figures 33A to 33C illustrates modification of at least one predicted trajectory in accordance with Figure 32 the method of Figure 33A A top plan view 300 is provided of the host vehicle 100 and a first roadblocker vehicle 200A having a current or future trajectory 202A and a second roadblocker vehicle 200B having a past-time roadblocker predicted trajectory 202B. Figure 33B A driver's perspective 310 is provided of the scenario as provided in Figure 33A The host vehicle predicted trajectory is omitted for clarity. However, the presence of the past-time roadblocker predicted trajectory 202B can cause confusion to the user. Accordingly, Figure 33C Modification of the shape and / or thickness, etc., of the past-time roadblocker predicted trajectory is provided.

[0093] Figure 34is a flowchart of a method 428 of selecting a type of display to be provided in a user interface based on complexity in accordance with illustrative aspects of the present technology. The method 428 is used to generate a trajectory for a vehicle user interface showing a driver's perspective view, and includes: generating a host vehicle predicted trajectory for the host vehicle; and generating at least one roadblocker predicted trajectory for a roadblocker located outside of the host vehicle. As indicated by method step 430, after generating the respective predicted trajectories, the method 428 continues with determining that the at least one roadblocker predicted trajectory overlaps with an object or another roadblocker predicted trajectory. The method continues to step 432 in which the trajectory prediction module determines that the display in the user interface showing the perspective view when presented / displayed with the driver's perspective can be complex or confusing. In various aspects, the determination of complexity can be based on one or more of a number of factors, including: a threshold number and / or type of roadblockers and objects present in the display; the location of the roadblockers, objects, and predicted trajectories; the time of day; traffic congestion; weather; the user's experience; the duration of travel; and the like. In various aspects, the determination of complexity can be based on one or more calculations or confidence scores, which can be based on, for example, the density of predicted trajectories, the number of roadblockers and / or objects, and the number of overlapping points. As provided in method step 434, the method includes generating a display in a user interface showing a top-down plan view, in contrast to the driver's perspective. The use of a top-down plan view is intended to simplify the display, and provide the user with a more complete display of the surrounding environment and scene. In various aspects, the method can include requesting instructions to obtain a selection request from the user and allow the user to preview different display options and freely switch between different views. In various other aspects, the method can include providing the user with both types of displays, for example, providing a top-down plan view in a first display, and a driver's perspective display in a second display. In other aspects, a single display can be generated providing both a top-down plan view and a driver's perspective, in a side-by-side arrangement, or the like.

[0094] With respect to the display of icons and depictions of predicted trajectories, roads, objects, and the like, it should be understood that the present technology should not be limited to the particular types and styles specifically described herein, and they can be customized as desired. In this regard, Figures 35A to 35C different non-limiting variations of predicted trajectories with different lines, 2D patterns, and 3D shapes are illustrated. For example, Figure 35A different types of line and arrow combinations and designs are provided, which can include an indication of a separate segment that can indicate a change in direction of travel (shown as a dot in Figure 35A ). Figure 35B different types of line and arrow combinations and designs are provided, which can include an indication of a separate segment that can indicate a change in direction of travel (shown as a dot in Figure 35A ). Figure 35ADifferent types of two-dimensional shapes for lines. Two-dimensional shapes can be designed with different gradients or colors to better indicate driving direction, speed, confidence level, etc. Figure 35C Provides a replacement Figure 35A Different types of 3D shapes for lines. 3D shapes can be similarly designed with different gradients or colors to better indicate direction of travel, speed, confidence level, etc. 3D shapes can be provided with various levels of detail, ranging from using lines and simple shapes to displaying detailed 3D objects.

[0095] Figures 36A to 36C The illustration shows the 2D and 3D predicted trajectories when the two predicted trajectories overlap. Figure 36A The illustration shows the previous figures relative to Figure 16A to Figure 16A. Figure 16C The scenario under discussion involves the predicted trajectory 102 of the vehicle intersecting and overlapping with the predicted trajectory 202 of the roadblock. Figure 36B Two sets of two-dimensional shapes with varying colors and gradients are provided to represent the predicted trajectories 102 and 202. Figure 36C Two sets of three-dimensional shapes with varying colors and gradients are provided to represent the predicted trajectories 102 and 202.

[0096] Figures 37A to 37C The illustration shows 2D and 3D predicted trajectories when the predicted trajectory overlaps with at least one static object. Figure 37A The illustration shows the previous relative to Figures 31A to 31C The scenario discussed involves a vehicle 200 blocking the road while traveling adjacent to multiple static objects 216. Figure 37B A two-dimensional shape for the predicted trajectory 202 is provided, which has varying colors and gradients. Figure 37C It provides variations in blending, dilution, and hiding to represent the three-dimensional shape of the predicted trajectory 202.

[0097] Figures 38A to 38B The illustration shows the use of a combination of 2D and 3D predicted trajectories on a single display. Figure 38A A top-down plan of 300 is provided, and Figure 38B Provided in Figure 38A The scene presented is from the driver's perspective 310. Specifically, Figure 38B A vehicle prediction trajectory 102 with a simple line pattern is provided, a first obstacle prediction trajectory 202A with a three-dimensional shape is provided, and a second obstacle prediction trajectory 202B with a two-dimensional shape is provided. It should be understood that various modifications and combinations can be used with this technology. In various aspects, users can customize and change the display type, and the system and method may include changing the shape and / or size of the prediction trajectory based on predetermined thresholds and requirements.

[0098] Each of the various methods described herein can be provided as part of a system that can include one or more processors and memory, the memory can include a trajectory prediction module that includes instructions that, when executed by the one or more processors, cause the processors to perform actions of the steps described in various portions of the methods for performing. Similarly, each of the methods described herein can be stored as instructions on a non-transitory computer readable medium.

[0099] Now, Figure 1 An example vehicle environment within which the systems and methods disclosed herein can operate is discussed in sufficient detail. In some instances, the vehicle 100 is configured to selectively switch between an autonomous mode, one or more semi-autonomous operation modes, and / or a manual mode. Such switching, when transitioning to the manual mode, also referred to as handover, can be implemented in a suitable manner now known or later developed. The "manual mode" means that all or a substantial portion of the navigation and / or maneuvering of the vehicle is performed in accordance with input received from a user (e.g., a human driver / operator).

[0100] In one or more aspects, the vehicle 100 is an autonomous vehicle. As used herein, an "autonomous vehicle" refers to a vehicle that operates in an autonomous mode. The "autonomous mode" refers to the use of one or more computing systems to control the vehicle 100 to navigate and / or maneuver the vehicle 100 along a travel route with minimal or no input from a human driver / operator. In one or more aspects, the vehicle 100 is highly automated or fully automated. In one aspect, the vehicle 100 is configured with one or more semi-autonomous operation modes in which the one or more computing systems perform a portion of the navigation and / or maneuvering of the vehicle along a travel route and a vehicle operator (i.e., driver) provides input to the vehicle to perform a portion of the navigation and / or maneuvering of the vehicle 100 along a travel route. Thus, in one or more aspects, the vehicle 100 operates autonomously according to a particular defined level of autonomy. For example, the vehicle 100 can operate according to the Society of Automotive Engineers (SAE) Automated Vehicle Classification 0-5. In one aspect, the vehicle 100 operates according to SAE Level 2, which provides that the autonomous driving module 160 controls the vehicle 100 through braking, acceleration, and steering without operator input, but the driver / operator will monitor the driving and remain alert and ready to take control of the vehicle 100 if the autonomous module 160 is unable to properly respond or otherwise is unable to adequately control the vehicle 100.

[0101] The vehicle 100 can include one or more processors 110. In one or more arrangements, the processor(s) 110 can be a main processor of the vehicle 100. For example, the processor(s) 110 can be an electronic control unit (ECU). The vehicle 100 can include one or more data storage devices 115 for storing one or more types of data. The data storage devices 115 can include volatile and / or nonvolatile memory. Examples of suitable data storage devices 115 include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, magnetic disks, optical disks, hard drives, or any other suitable type of storage medium or any combination thereof. The data storage devices 115 can be a component of the processor(s) 110 or the data storage devices 115 can be operatively connected to the processor(s) 110 for use in connection therewith. The term “operatively connected” as used throughout this description can include direct or indirect connection among the components, including connection without direct physical contact.

[0102] In one or more arrangements, the one or more data storage devices 115 can include map data 116. The map data 116 can include a map of one or more geographic regions. In some instances, the map data 116 can include information or data regarding roads, traffic control devices, road markings, structures, features, and / or landmarks in one or more geographic regions. The map data 116 can take any suitable form. In some instances, the map data 116 can include an aerial view of a region. In some instances, the map data 116 can include a ground view of a region, including a 360-degree ground view. The map data 116 can include measurements, dimensions, distances, and / or information for one or more items included in the map data 116 and / or relative to other items included in the map data 116. The map data 116 can include a digital map with information about road geometry. The map data 116 can be of high quality and / or highly detailed.

[0103] In one or more arrangements, the map data 116 can include one or more terrain maps 117. The terrain map(s) 117 can include information about the ground, terrain, roads, surfaces, and / or other features of one or more geographic regions. The terrain map(s) 117 can include elevation data in one or more geographic regions. The terrain map(s) 117 can be high quality and / or highly detailed. The terrain map(s) 117 can define one or more ground surfaces, which can include paved roads, unpaved roads, land, and other things that define a ground surface.

[0104] In one or more arrangements, the map data 116 can include one or more static obstacle maps 118. The static obstacle map(s) 118 can include information about one or more static obstacles located within one or more geographic regions. A “static obstacle” is a physical object whose location does not change or substantially change over a period of time and / or whose size does not change or substantially change over a period of time. Examples of static obstacles include trees, buildings, curbs, fences, railings, medians, utility poles, statues, monuments, signs, benches, furniture, mailboxes, large rocks, slopes. A static obstacle can be an object that extends above the level of the ground. One or more static obstacles included in the static obstacle map(s) 118 can have location data, size data, dimension data, material data, and / or other data associated therewith. The static obstacle map(s) 118 can include measurements, dimensions, distances, and / or information for one or more static obstacles. The static obstacle map(s) 118 can be high quality and / or highly detailed. The static obstacle map(s) 118 can be updated to reflect changes within the area being mapped.

[0105] The one or more data stores 115 can include sensor data 119. In this context, “sensor data” means any information about sensors equipped by the vehicle 100, including information about capabilities of such sensors and other information. As will be explained below, the vehicle 100 can include a sensor system 120. The sensor data 119 can relate to one or more sensors of the sensor system 120. As an example, in one or more arrangements, the sensor data 119 can include information about one or more LIDAR sensors 124 of the sensor system 120.

[0106] In some instances, at least a portion of the map data 116 and / or the sensor data 119 can be located in one or more data stores 115 positioned on the vehicle 100. Alternatively or additionally, at least a portion of the map data 116 and / or the sensor data 119 can be located in one or more data stores 115 positioned remotely with respect to the vehicle 100.

[0107] As noted above, the vehicle 100 can include a sensor system 120. The sensor system 120 can include one or more sensors. By "sensor" is meant any device, component, and / or system that can detect and / or sense something. The one or more sensors can be configured to detect and / or sense in real-time. As used herein, the term "real-time" means a level of processing responsiveness that a user or system senses as sufficiently immediate for a particular processing or determination to be made or such that a processor is able to keep up with some external process.

[0108] In arrangements in which the sensor system 120 includes multiple sensors, the sensors can function independently of one another. Alternatively, two or more of the sensors can function in combination with one another. In this case, the two or more sensors can form a sensor network. The sensor system 120 and / or the one or more sensors can be operatively connected to the processor(s) 110, the data store(s) 115, and / or another element of the vehicle 100, including any of the elements shown in FIG. 1. The sensor system 120 can acquire data of at least a portion of an external environment of the vehicle 100 (e.g., nearby vehicles). Figure 1

[0109] ​The sensor system 120 can include any suitable type of sensor. Various examples of different types of sensors will be described herein. It will be appreciated, however, that these aspects are not limited to the particular sensors described. The sensor system 120 can include one or more vehicle sensors 121. The vehicle sensor(s) 121 can detect, determine, and / or sense information about the vehicle 100 itself. In one or more arrangements, the vehicle sensor(s) 121 can be configured to detect and / or sense changes in position and orientation of the vehicle 100, such as for example based on inertial acceleration. In one or more arrangements, the vehicle sensor(s) 121 can include one or more accelerometers, one or more gyroscopes, an inertial measurement unit (IMU), a dead reckoning system, a global navigation satellite system (GNSS), a global positioning system (GPS), a navigation system 147, and / or other suitable sensors. The vehicle sensor(s) 121 can be configured to detect and / or sense one or more characteristics of the vehicle 100. In one or more arrangements, the vehicle sensor(s) 121 can include a speedometer for determining a current speed of the vehicle 100.

[0110] Alternatively or additionally, the sensor system 120 can include one or more environment sensors 122 configured to acquire and / or sense driving environment data. “Driving environment data” includes data or information about an external environment in which the autonomous vehicle is located, or one or more portions thereof. For example, the environment sensor(s) 122 can be configured to detect, quantify, and / or sense obstacles and / or information / data about such obstacles in at least a portion of an external environment of the vehicle 100. Such obstacles can be stationary objects and / or dynamic objects. The environment sensor(s) 122 can be configured to detect, measure, quantify, and / or sense other things in an external environment of the vehicle 100, such as for example lane markings, signs, traffic signal lights, traffic signs, lane lines, crosswalks, a curb proximate to the vehicle 100, objects outside of a roadway, and the like.

[0111] Various examples of sensors of the sensor system 120 will be described herein. The example sensors can be part of the environment sensor(s) 122 and / or the vehicle sensor(s) 121. In addition, the sensor system 120 can include operator sensors for tracking or otherwise monitoring aspects related to a driver / operator of the vehicle 100. It will be appreciated, however, that these aspects are not limited to the particular sensors described.

[0112] As an example, in one or more arrangements, the sensor system 120 can include one or more radar sensors 123, one or more LIDAR sensors 124, one or more sonar sensors 125, and / or one or more cameras 126. In one or more arrangements, the one or more cameras 126 can be a high dynamic range (HDR) camera, an infrared (IR) camera, and / or the like. In one aspect, the camera 126 includes one or more cameras disposed within a passenger compartment of the vehicle for performing eye tracking of an operator / driver in order to determine a line of sight of the operator / driver, a gaze trace of the operator / driver, and / or the like.

[0113] The vehicle 100 can include an input system 130. An "input system" includes any device, component, system, element, or arrangement, or group thereof, that enables information / data to be input into a machine. The input system 130 can receive input from a vehicle passenger (e.g., a driver or passenger). The vehicle 100 can include an output system 135. An "output system" includes any device, component, or arrangement, or group thereof, that enables information / data to be presented to a vehicle passenger (e.g., a human, a vehicle passenger, and / or the like).

[0114] The vehicle 100 can include one or more vehicle systems 140. In Figure 1 Various examples of the one or more vehicle systems 140 are illustrated in FIG. 1. However, the vehicle 100 can include more, fewer, or different vehicle systems. It should be recognized that while particular vehicle systems are defined separately, each or any of the systems or portions thereof can be combined or separated in other ways via hardware and / or software within the vehicle 100. The vehicle 100 can include a propulsion system 141, a braking system 142, a steering system 143, a throttle system 144, a transmission system 145, a signaling system 146, and / or a navigation system 147. Each of these systems can include one or more devices, components, and / or combinations thereof, now known or later developed.

[0115] The navigation system 147 can include one or more devices, sensors, applications, and / or combinations thereof, now known or later developed, configured to determine a geographic location of the vehicle 100 and / or determine a travel route for the vehicle 100. The navigation system 147 can include one or more mapping applications to determine a travel route for the vehicle 100. The navigation system 147 can include a global positioning system, a local positioning system, or a geolocation system.

[0116] The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can be operably connected to communicate with various vehicle systems 140 and / or various components thereof. For example, returning to FIG. 1, the trajectory prediction system 170 can be operably connected to the propulsion system 141, the braking system 142, the steering system 143, the throttle system 144, the transmission system 145, the signaling system 146, and / or the navigation system 147. Figure 1The processor(s) 110 and / or the autonomous driving module(s) 160 can be in communication to send and / or receive information from various vehicle systems 140 to control movement, speed, steering, heading, direction, etc. of the vehicle 100. The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can control some or all of these vehicle systems 140 and thus can be partially or fully autonomous.

[0117] The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can be operatively connected to communicate with various vehicle systems 140 and / or components thereof. For example, returning to FIG. 1, the processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can be in communication with the engine 142, the brakes 144, the steering 146, the suspension 148, the navigation system 150, the radar system 152, the lidar system 154, the camera system 156, and / or the ultrasonic sensor system 158. Figure 1 The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can be in communication to send and / or receive information from various vehicle systems 140 to control movement, speed, steering, heading, direction, etc. of the vehicle 100. The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can control some or all of these vehicle systems 140.

[0118] The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can be operable to control navigation and / or steering of the vehicle 100 by controlling one or more of the vehicle systems 140 and / or components thereof. For example, when operating in an autonomous mode, the processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can control the direction and / or speed of the vehicle 100. The processor(s) 110, the trajectory prediction system 170, and / or the autonomous driving module(s) 160 can cause the vehicle 100 to accelerate (e.g., by increasing a supply of fuel provided to an engine), decelerate (e.g., by decreasing a supply of fuel to an engine and / or by applying brakes), and / or change direction (e.g., by turning two front wheels). As used herein, “cause” or “causing” means to make, force, compel, direct, command, instruct, and / or enable an event or action to occur or at least be in a state in which such event or action can occur, either directly or indirectly.

[0119] The vehicle 100 can include one or more actuators 150. The actuators 150 can be any element or combination of elements operable to modify, adjust, and / or alter one or more of the vehicle systems 140 or components thereof in response to receiving a signal or other input from the processor(s) 110 and / or the autonomous driving module(s) 160. Any suitable actuator can be used. For example, the one or more actuators 150 can include a motor, a pneumatic actuator, a hydraulic piston, a relay, a solenoid, and / or a piezoelectric actuator, to name just a few possibilities.

[0120] The vehicle 100 can include one or more modules, at least some of which are described herein. The modules can be implemented as computer-readable program code that, when executed by the processor 110, implements one or more of the various processes described herein. One or more of the modules can be components of the processor(s) 110, or one or more of the modules can execute on and / or be distributed among other processing systems to which the processor(s) 110 are operatively connected. The modules can include instructions (e.g., program logic) executable by the processor(s) 110. Alternatively or additionally, one or more data stores 115 can contain such instructions. Generally, as used herein, the term module includes routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular data types. In other aspects, memory generally stores the modules mentioned. The memory associated with a module can be a buffer or cache embedded within a processor, RAM, ROM, flash memory, or another suitable electronic storage medium. In other aspects, modules contemplated by the present disclosure are implemented as application-specific integrated circuits (ASICs), hardware components of a system-on-a-chip (SoC), programmable logic arrays (PLAs), or another suitable hardware component that has a defined set of configurations (e.g., instructions) embedded therein for performing the disclosed functions.

[0121] In one or more arrangements, one or more of the modules described herein can include artificial intelligence or computational intelligence elements, e.g., neural networks, fuzzy logic, or other machine learning algorithms. Additionally, in one or more arrangements, one or more of the modules can be distributed among multiple modules described herein. In one or more arrangements, two or more of the modules described herein can be combined into a single module.

[0122] The vehicle 100 can include one or more autonomous driving modules 160. The autonomous driving module(s) 160 can be configured to receive data from the sensor system 120 and / or any other type of system capable of capturing information related to the vehicle 100 and / or the external environment of the vehicle 100. In one or more arrangements, the autonomous driving module(s) 160 can use such data to generate one or more driving scene models. The autonomous driving module(s) 160 can determine the location and speed of the vehicle 100. The autonomous driving module(s) 160 can determine the location of obstacles or other environmental features including traffic signs, trees, bushes, nearby vehicles, pedestrians, etc.

[0123] The autonomous driving module(s) 160 can be configured to receive and / or determine location information for obstacles within the external environment of the vehicle 100 used by the processor(s) 110 and / or one or more of the modules described herein to estimate the location and orientation of the vehicle 100, the vehicle location in global coordinates based on signals from multiple satellites, or any other data and / or signals that can be used to determine the current state of the vehicle 100 or determine the location of the vehicle 100 relative to its environment for creating a map or determining the location of the vehicle 100 relative to map data.

[0124] The autonomous driving module(s) 160 can be configured, independently or in combination with the trajectory prediction system 170, to determine the driving path(s), the current autonomous driving maneuver of the vehicle 100, the future autonomous driving maneuver, and / or the modification to the current autonomous driving maneuver based on data acquired by the sensor system 120, a driving scene model, and / or data from any other suitable source. A “driving maneuver” means one or more actions that affect the movement of the vehicle. Examples of driving maneuvers include accelerating, decelerating, braking, steering, moving in the lateral direction of the vehicle 100, changing lanes, merging into a lane, and / or reversing, to name just a few possibilities. The autonomous driving module(s) 160 can be configured to implement the determined driving maneuver. The autonomous driving module(s) 160 can cause such autonomous driving maneuver to be implemented directly or indirectly. As used herein, “cause” or “causing” means to make, command, instruct, and / or enable an event or action to occur, or at least be in a state in which such event or action can occur, in a direct or indirect manner. The autonomous driving module(s) 160 can be configured to perform various vehicle functions, and / or send data to, receive data from, interact with, and / or control the vehicle 100 or one or more systems thereof (e.g., one or more of the vehicle systems 140).

[0125] Detailed aspects are disclosed herein. It is to be understood, however, that the disclosed aspects are merely exemplary. As such, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the aspects disclosed herein in virtually any appropriately detailed structure. In addition, the terms and phrases used herein are not intended to be limiting, but rather to provide an understandable description of possible implementations. Various aspects are shown in collective figures, but are not limited to the illustrated structures or applications.

[0126] The flow and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods and computer program products according to various aspects. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved.

[0127] The systems, components, and / or processes described above can be implemented in hardware, or a combination of hardware and software, and can be implemented in a centralized fashion in one processing system or in a distributed fashion where different elements are spread across several interconnected processing systems. Any kind of processing system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software can be a processing system with computer-usable program code that was written in a computer language, which controls the processing system such that it carries out the methods described herein when the computer program code is loaded and executed. The systems, components, and / or processes can also be embedded in a computer-readable storage medium such as a computer program product or other data program storage device, which is readable by a machine, tangibly embodying a program of instructions executable by the machine to perform the methods and processes described herein. The elements can also be embedded in an application product including all features enabling the implementation of the methods described herein and the carrying out of the methods when loaded in a processing system.

[0128] Furthermore, the arrangements described herein can take the form of a computer program product having a computer readable medium having embodied thereon a computer program comprising program instructions. Any combination of one or more computer readable medium can be utilized. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The phrase "computer readable storage medium" means a non-transitory storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a solid state drive, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0129] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present arrangements can be written in any combination of one or more programming languages, including an object oriented programming language such as Java™, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0130] The foregoing description is provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure, its application, or uses. It is not intended to be exhaustive or to limit the disclosure, its application, or uses. Various elements or features of a particular aspect are generally not limited to that particular aspect, but can be used in a different aspect, if desired. Likewise, a variety of permutations and

[0131] As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A OR B OR C) using the non-exclusive logical OR. It will be understood that various steps within a method, whether manual, electronic, mechanical, or otherwise, can be performed in

[0132] The headings provided herein, such as "BACKGROUND OF THE INVENTION" and "SUMMARY," are provided for the convenience of the reader and are not intended to limit the disclosure, its application, or use. The recitation of multiple aspects with stated features is not intended to exclude other aspects of the disclosure having additional features, or other aspects integrating features of the stated aspects.

[0133] As used herein, the terms "comprise" and "comprising" and variations thereof are intended to be non-limiting, such that recitation of items in a list does not imply that there are not additional items that can be included in the devices and methods of the present technology. Similarly, the terms "may" and "might" and variations thereof are intended to be non-limiting, such that recitation of aspects that can or might include certain elements or features does not exclude other aspects of the present technology that do not contain those elements or features. The terms "a" and "an" as used herein are defined as one or more. The term "plurality" as used herein is defined as two or more. The term "another" as used herein is defined as at least a second or more.

[0134] The broad teachings of the present disclosure can be implemented in various forms. Therefore, while this disclosure includes particular examples, the true scope of the present disclosure should not be so limited since other modifications will become apparent to the skilled practitioner after a study of the specification and following claims. Reference herein to one or more aspects means that a particular feature, structure, or characteristic described in connection with the embodiment or a particular system is included in at least one embodiment or aspect. The appearance of the phrase "in one aspect" (or variations thereof) in various places in the specification does not necessarily refer to the same aspect or embodiment. It should also be understood that various method steps described herein need not be performed in the same order as described, and that not every method step is required to be performed in every aspect or embodiment.

Claims

1. A system for generating a trajectory for a vehicle user interface displaying a driver's viewpoint, the system comprising: One or more processors; as well as A memory, communicatively coupled to the one or more processors and storing: A trajectory prediction module, the trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: Generate at least one roadblock prediction trajectory for roadblocks located outside the vehicle, wherein the at least one roadblock prediction trajectory does not extend from the display area and has a known direction of travel when displayed on a user interface showing a top-down plan view; The predicted trajectory of the at least one roadblock is determined to extend from the display area and have an unknown direction of travel when displayed on the user interface showing the driver's view. Modify the predicted trajectory of the at least one roadblock by controlling the operation of the user interface to perform at least one of the following to provide directional indication: Modify the length of the predicted trajectory of the at least one roadblock and add directional arrows to the appropriate ends of the predicted trajectory of the at least one roadblock so that they are visible when displayed on the user interface showing the driver's viewpoint; as well as An icon representing the roadblocker is added to the appropriate end of the predicted trajectory of the at least one roadblocker so that it is visible when displayed on the user interface showing the driver's view. as well as A control module, the control module including, when executed by the one or more processors, instructions to cause the one or more processors to update the user interface such that the user interface displaying the driver's view includes any modified roadblock prediction trajectory.

2. A system for generating a trajectory for a vehicle user interface displaying a driver's viewpoint, the system comprising: One or more processors; as well as A memory, communicatively coupled to the one or more processors and storing: A trajectory prediction module, the trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: Generate a predicted trajectory for this vehicle; Generate at least one roadblock prediction trajectory for a roadblock located outside the vehicle; It is determined that the predicted trajectory of the at least one roadblock overlaps with an object or another predicted trajectory when displayed on a user interface showing the driver's view. The predicted trajectory of the at least one roadblock overlaps with one or more static objects not residing on the road or sidewalk when displayed on the user interface showing a driver's view, and the predicted trajectory of the at least one roadblock does not overlap with the one or more static objects when displayed on the user interface showing a top-down plan. as well as The operation of the user interface is controlled to modify the at least one predicted trajectory of the roadblock by diluting or blending the at least one predicted trajectory of the roadblock at each location where the at least one predicted trajectory of the roadblock overlaps with one of the static objects; as well as A control module, the control module including, when executed by the one or more processors, instructions to cause the one or more processors to update the user interface such that the user interface displaying the driver's view includes any modified roadblock prediction trajectory.

3. A system for generating a trajectory for a vehicle user interface displaying a driver's viewpoint, the system comprising: One or more processors; as well as A memory, communicatively coupled to the one or more processors and storing: A trajectory prediction module, the trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: Generate a predicted trajectory for this vehicle; Generate at least one roadblock prediction trajectory for a roadblock located outside the vehicle; It is determined that the predicted trajectory of the at least one roadblock overlaps with an object or another predicted trajectory when displayed on a user interface showing the driver's view. It is determined that the predicted trajectory of the at least one roadblock is a predicted trajectory of a past roadblock; as well as The operation of the user interface is controlled to modify the at least one predictive trajectory of a roadblock by changing one or both of the shape and thickness of the predicted trajectory of the past roadblock; as well as A control module, the control module including, when executed by the one or more processors, instructions to cause the one or more processors to update the user interface such that the user interface displaying the driver's view includes any modified roadblock prediction trajectory.

4. The system according to claim 3, wherein, The shape and thickness of the predicted trajectory of at least one roadblock are changed to be different from the shape and thickness of the predicted trajectory of the vehicle itself.

5. A system for generating a trajectory for a vehicle user interface displaying a driver's viewpoint, the system comprising: One or more processors; as well as A memory, communicatively coupled to the one or more processors and storing: A trajectory prediction module, the trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: Generate a predicted trajectory for this vehicle; Generate at least one roadblock prediction trajectory for a roadblock located outside the vehicle; It is determined that the predicted trajectory of the at least one roadblock overlaps with an object or another predicted trajectory when displayed on a user interface showing the driver's view. The display in the user interface that shows the driver's view is complex; as well as Modify the predicted trajectory of at least one roadblock; as well as A control module, comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the following operations: Generate a display in the user interface that shows the top-down plan; and Update the user interface to display the top-down plan and include any modified roadblock prediction trajectories.

6. The system according to claim 5, wherein, The control module also includes instructions for receiving a selection request from the user to display one of a top-down plan view and a driver's view.

7. The system according to claim 5, wherein, The control module also includes instructions for generating displays in a user interface that shows both a top-down plan view and a driver's view.

8. A method for generating a trajectory for a vehicle user interface displaying a driver's viewpoint, the method comprising: Generate a predicted trajectory for this vehicle; Generate at least one roadblock prediction trajectory for a roadblock located outside the vehicle; It is determined that the predicted trajectory of the at least one roadblock overlaps with the predicted trajectory of an object or another roadblock when displayed on a user interface showing the driver's view. It is determined that the predicted trajectory of the at least one roadblock is a predicted trajectory of a past roadblock; as well as The at least one predictive trajectory of a roadblock is modified by changing one or both of the shape and thickness of the predicted trajectory of the past roadblock; as well as Update the user interface so that the user interface showing the driver's view includes any modified roadblock prediction trajectory.

Images