Autonomous vehicle user interface with predicted trajectories

Abstract

The present disclosure relates to autonomous vehicle user interfaces with predicted trajectories. Systems and methods are provided for generating trajectories for a vehicle user interface showing a driver's perspective view. The method includes generating a host vehicle predicted trajectory for a host vehicle and generating at least one roadblocker predicted trajectory for a roadblocker located outside the host vehicle. After generating the predicted trajectories, the method continues with determining that the at least two predicted trajectories overlap when displayed on a user interface showing a driver's perspective view. The method includes modifying the at least one roadblocker predicted trajectory to eliminate the overlap. The method then continues with updating the display of the user interface to include any modified roadblocker predicted trajectories. The system includes a trajectory prediction module for performing the method.

Description

Technical Field

[0001] This disclosure generally relates to user interfaces for autonomous vehicles, and more particularly to systems and methods for modifying and displaying predicted trajectories on a user interface that provides a driver's viewpoint. Background Technology

[0002] The background description provided herein is for the purpose of generally presenting the context of this disclosure. To the extent that may be described in this background section, the work of the inventors currently named and aspects described that would otherwise qualify as prior art at the time of filing are neither expressly nor impliedly acknowledged as prior art to this invention.

[0003] In various applications, vehicle systems can predict the trajectory of a vehicle (sometimes referred to herein as the "ego-vehicle"). For example, parallel autonomous vehicles, such as those incorporating Advanced Driver Assistance Systems (ADAS), are vehicles where control can be shared between a human driver and the autonomous driving system. The human driver can maintain control over certain aspects of driving such a vehicle (e.g., steering) while the ADAS monitors the driver's actions and intervenes when necessary to prevent accidents. Therefore, predicting the ego-vehicle's trajectory is a crucial aspect of such ADAS. The vehicle system can then display the ego-vehicle's trajectory on a user interface.

[0004] The vehicle system can also predict the trajectories of one or more road agents located outside the vehicle and display these trajectories on the user interface display. Examples of road agents include various types of other vehicles (e.g., cars, motorcycles, or bicycles) and pedestrians. One objective of autonomous or parallel autonomous vehicles is to travel a route without colliding with road agents encountered along the way. Since the driver of an autonomous or parallel autonomous vehicle typically does not know for certain the intentions of road agents or their drivers, predicting the trajectories of road agents can facilitate this objective.

[0005] However, when numerous obstacle trajectories are presented on the user interface, it quickly becomes overly complex, especially when the display is presented as a driver's view (e.g., compared to a top-down plan view). Therefore, it would be desirable to provide an improved trajectory prediction system that adequately accounts for the presence of numerous and / or overlapping trajectories that can be presented on the user interface, resulting in a less complex display. Summary of the Invention

[0006] This section provides a summary of the disclosure and is not a complete disclosure of the full scope or all features of the disclosure.

[0007] In various aspects, this teaching provides a system for generating trajectories for a vehicle user interface displaying a driver's viewpoint. The system includes one or more processors and a memory communicatively coupled to the one or more processors. The memory stores a trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform a series of steps. For example, the trajectory prediction module may include instructions for performing: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. The instructions may include steps for determining that at least two predicted trajectories overlap when displayed on the user interface displaying the driver's viewpoint. The instructions may include steps for modifying at least one predicted roadblock trajectory to eliminate overlap. A control module may also be provided, including instructions that, when executed by the one or more processors, cause the one or more processors to update the user interface to include any modified predicted roadblock trajectories.

[0008] In other aspects, a system for generating trajectories for a vehicle user interface displaying a driver's view includes one or more processors and a memory communicatively coupled to the one or more processors. The memory stores a trajectory prediction module including instructions that, when executed by the one or more processors, cause the one or more processors to perform a series of steps. For example, the trajectory prediction module may include instructions for performing: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. The instructions may include a step for determining that the distance between two adjacent predicted trajectories is less than a predetermined threshold when displayed on the user interface displaying the driver's view. Thereafter, the instructions may include a step for performing at least one modification selected from the group consisting of: (1) changing the color of at least one predicted trajectory among the predicted trajectories; (2) changing the spacing of at least one predicted trajectory among the predicted trajectories; (3) changing the thickness of at least one predicted trajectory among the predicted trajectories; and (4) determining a priority predicted trajectory based on proximity to the vehicle and displaying only the priority predicted trajectory. A control module may also be provided, which includes instructions that, when executed by one or more processors, cause one or more processors to update the user interface to include any modified roadblock prediction trajectory.

[0009] In other aspects, this teaching provides a method for generating trajectories for a vehicle user interface displaying a driver's viewpoint. The method includes: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. After generating the predicted trajectories, the method continues by determining that at least two predicted trajectories overlap when displayed on the user interface displaying the driver's viewpoint. The method includes modifying at least one roadblock predicted trajectory to eliminate the overlap. The method then continues to update the display of the user interface to include any modified roadblock predicted trajectories.

[0010] Other applicable fields and various methods for enhancing the above technology will become clear from the description provided herein. The description and specific examples in this invention are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0011] This teaching will be more fully understood based on the specific description and accompanying drawings, wherein:

[0012] Figure 1 This is a schematic diagram illustrating exemplary aspects of a vehicle in which the systems and methods disclosed herein according to the present technology can be implemented;

[0013] Figure 2 It's a diagram. Figure 1 A schematic diagram illustrating an exemplary aspect of the trajectory prediction system provided in the diagram;

[0014] Figure 3 The illustration shows a partial perspective view of an interior compartment of an exemplary vehicle interior compartment provided with multiple display systems according to various aspects of the present technology. The multiple display systems can be used individually or in combination to provide one or more user interface displays.

[0015] Figures 4A to 4E Five examples of user interfaces are shown, which have images representing the vehicle itself and at least one obstructing vehicle or pedestrian with corresponding trajectories.

[0016] Figures 5A to 5E The five illustrated user interface examples show the overlap of the trajectory and the vehicle, with the user interface displaying the relationship between the trajectory and the vehicle. Figures 4A to 4E Images of the vehicle and one or more obstructing vehicles or pedestrians in the same scene provided in the image, either in a frontal perspective view or a driver's view, and their corresponding trajectories.

[0017] Figure 6 This is a flowchart of a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object, according to the illustrative aspects of this technology.

[0018] Figure 7This is another flowchart of a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object, according to the illustrative aspects of this technology;

[0019] Figures 8A to 8C The illustration depicts a modification of at least one predicted trajectory based on the overlap with an object, according to an illustrative aspect of the present technology.

[0020] Figures 9A to 9C The illustration depicts a modification of at least one predicted trajectory based on the overlap of two objects, according to an illustrative aspect of the present technology.

[0021] Figure 10 This is a flowchart illustrating a method for modifying at least one predicted trajectory based on overlap with an object by hiding, diluting, and / or blending predicted trajectories, according to various aspects of the present technology.

[0022] Figures 11A to 11E The diagram illustrates the following: Figure 10 Various modifications to at least one predicted trajectory method;

[0023] Figure 12 This is a flowchart of different techniques for modifying at least one predicted trajectory based on different overlap selections, according to various aspects of this technology;

[0024] Figures 13A to 13B The diagram illustrates the following: Figure 12 At least one predicted trajectory modification of the method;

[0025] Figure 14 This is a flowchart illustrating a method for modifying at least one predicted trajectory based on the overlap of at least two predicted trajectories, with the optional use of confidence scores, according to an illustrative aspect of the present technology.

[0026] Figure 15 This is a flowchart of a method for modifying at least one predicted trajectory based on the shortest distance between two adjacent predicted trajectories, according to the illustrative aspects of this technology.

[0027] Figures 16A to 16C The diagram illustrates the following: Figure 14 At least one predicted trajectory modification of the method;

[0028] Figure 17 The illustration shows the user interface display from the driver's perspective when the roadblock overlaps with the predicted trajectory;

[0029] Figures 18A to 18E The diagram illustrates the following: Figures 14 to 15 The method Figure 17 At least one predicted trajectory modification;

[0030] Figure 19This is a flowchart of a method for modifying at least one predicted trajectory based on the overlap of at least two predicted trajectories, in accordance with the illustrative aspects of this technology, when a preferred intersection point is determined.

[0031] Figure 20 The illustration shows a set of four vehicles with predicted trajectories, where the four intersections have various priorities;

[0032] Figures 21A to 21B The diagram illustrates the following: Figure 19 At least one predicted trajectory modification of the method;

[0033] Figure 22 This is a flowchart illustrating a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with the object when it is determined that the object is a partially or completely hidden obstacle, according to the illustrative aspects of this technology.

[0034] Figures 23A to 23D The diagram illustrates the following: Figure 22 At least one predicted trajectory modification of the method;

[0035] Figures 24A to 24D The diagram illustrates the following: Figure 22 At least one additional modification to the method for predicting the trajectory;

[0036] Figure 25 This is a flowchart of a method for modifying at least one predicted trajectory based on the presence of sloping terrain, according to the illustrative aspects of this technology;

[0037] Figures 26A to 26D and Figures 27A to 27D The diagram illustrates the following: Figure 25 At least one predicted trajectory modification of the method;

[0038] Figure 28 This is a flowchart of a method for modifying at least one predicted trajectory based on at least one trajectory extending from a display area with an unknown direction of travel, according to an illustrative aspect of the present technology.

[0039] Figures 29A to 29E The diagram illustrates the following: Figure 28 At least one predicted trajectory modification of the method;

[0040] Figure 30 This is a flowchart of a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object or another predicted trajectory, in accordance with the illustrative aspects of the present technology, when it is determined that the predicted trajectory overlaps with multiple static objects.

[0041] Figures 31A to 31C The diagram illustrates the following: Figure 30 At least one predicted trajectory modification of the method;

[0042] Figure 32 This is a flowchart illustrating a method for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object or another predicted trajectory in the case where the predicted trajectory is determined to be a past tense obstacle trajectory, according to the illustrative aspects of this technology.

[0043] Figures 33A to 33C The diagram illustrates the following: Figure 32 At least one predicted trajectory modification of the method;

[0044] Figure 34 This is a flowchart illustrating a method for selecting the type of display to be provided in the user interface based on complexity, according to the illustrative aspects of this technology.

[0045] Figures 35A to 35C The illustration shows variations of the predicted trajectory with different lines, 2D patterns, and 3D shapes;

[0046] Figures 36A to 36C The illustration shows 2D and 3D predicted trajectories when the two predicted trajectories overlap.

[0047] Figures 37A to 37C The illustrations show 2D and 3D predicted trajectories when the predicted trajectory overlaps with at least one static object; and

[0048] Figures 38A to 38B The illustration shows the use of a combination of 2D and 3D predicted trajectories on a single display.

[0049] It should be noted that, for the purpose of describing certain aspects, the accompanying drawings are intended to illustrate the general characteristics of the methods, algorithms, and apparatuses of this technology. These drawings may not precisely reflect the characteristics of any given aspect and are not necessarily intended to define or limit specific aspects within the scope of this technology. Furthermore, certain aspects may integrate features derived from the accompanying drawings. Detailed Implementation

[0050] The technology described herein relates to an improved display of predicted roadblocks and vehicle trajectories on a user interface. Specifically, this technology improves and / or simplifies how these trajectories can interact with each other, and adjusts for and / or simplifies their size, position, and various other details related to their display on the user interface. In this respect, the technology can simplify the trajectory information presented to the user. For example, it can provide an improved trajectory prediction system and a less complex view that provides a display that adequately explains the presence of numerous and / or overlapping trajectories that can be presented on the user interface.

[0051] As used herein, the term "trajectory" or "multiple trajectories" can refer to simulated, predicted, or observed past, present, and future trajectories for a given roadblock, vehicle, or main vehicle. As used herein, the term "roadblock" generally refers to any object capable of moving along or intersecting with a road. These objects are not necessarily always in motion. For example, the aspects described herein consider cars, buses, bicycles, and other types of vehicles parked along a street as roadblocks. In those aspects, the system can use the vehicle's sensors to track the parked car along with other objects detected in the environment. Sensor data will typically reveal that the roadblock (the parked car) is stationary, at which point there is no predictable trajectory associated with it. However, in those aspects, the system may continue to track the parked car as it may begin to move at any time. In the aspects, the roadblock of interest is located outside the vehicle (sometimes referred to herein as "main vehicle" or "main vehicle") in which the aspects of this technology operate. These roadblocks are sometimes referred to herein as "external roadblocks." Additional, non-limiting examples of roadblocks include, but are not limited to, various types of other vehicles (cars, buses, motorcycles, bicycles, trucks, construction equipment, etc.), pedestrians, and animals. In some respects, a roadblock may be simply referred to as an object.

[0052] In a non-restrictive aspect, a probabilistic variational trajectory predictor can be used to predict the trajectory of the vehicle and / or an obstacle, which can be referred to as a predicted trajectory. In those aspects, the trajectory probability distribution of any applicable given obstacle or vehicle can be sampled to generate one or more specific predicted trajectories. As mentioned above, those predicted trajectories can be cross-fed and iteratively updated between the vehicle and one or more obstacles, and they can also be output to the vehicle's control module, which at least partially controls the outputs provided to various displays and user interfaces of the vehicle, as further described below. In some variational predictor aspects, statistical parameters of the trajectory probability distribution can be output to the vehicle's control module instead of the specific trajectory sampled from the distribution.

[0053] Depending on specific aspects, the predicted trajectory of the vehicle can be created by considering the predicted trajectories of multiple external roadblocks in any of a variety of possible orderings. In one aspect, the predicted trajectories are prioritized based on their distance from the vehicle, with those closer to the vehicle receiving higher priority than those farther away. In another aspect, the predicted trajectories are prioritized based on any uncertainty associated with them, with those having lower uncertainty (i.e., higher certainty) receiving higher priority than those having higher uncertainty (i.e., lower certainty). Furthermore, considering all possible orderings of external roadblocks, intermediate predicted trajectories for the vehicle and / or one or more external roadblocks can be retained, collected, and aggregated during iterative trajectory prediction processing. Preserving all these various assumptions allows the vehicle's control module to consider all possible actions that roadblocks might take. This preservation method facilitates the vehicle's planning and safe trajectory traversal objectives.

[0054] Other techniques can be advantageously combined with the iterative trajectory prediction architecture described above: (1) employing multiple trajectory predictors to predict the future trajectory of the vehicle and multiple trajectory predictors to predict the future trajectory of one or more roadblocks located outside the vehicle; and (2) generating confidence estimates for the predicted trajectories of the vehicle and roadblocks, enabling the assessment of their confidence. These techniques will be explained further in the following paragraphs.

[0055] To predict the future trajectory of the vehicle or a given external obstacle, some aspects described herein employ two or more trajectory predictors using different deterministic or probabilistic computational models. For example, in one aspect involving two trajectory predictors, the first trajectory predictor is a probabilistic variational trajectory predictor incorporating a DNN, and the second trajectory predictor is a physics-based (deterministic) model. In each aspect, the trajectory predictor receives any of the various vehicle sensor data discussed further below as input. Depending on the specific aspect, the trajectory predictor may also receive past trajectory information measured for the vehicle or obstacle, depending on the type of trajectory being predicted.

[0056] Regarding confidence estimation, an important aspect disclosed is the temporal (time) range within which the predicted vehicle or obstacle trajectory is defined. For example, a given predicted trajectory from a particular trajectory predictor might be credible within a relatively short temporal range of approximately 0.1 seconds to approximately 3 seconds, but it might not be credible within a longer temporal range extending beyond approximately 3 seconds up to approximately 10 seconds. In some aspects, a deep neural network (DNN) model is used to compute the confidence estimate for the predicted trajectories of the vehicle and obstacle as a continuous time function over the applicable temporal range. Therefore, the confidence measurement helps the trajectory prediction system determine which predicted trajectories of the vehicle or obstacle are most credible for a specific segment of the entire temporal prediction range. In various aspects, the confidence scores associated with the iteratively updated predicted trajectories of the vehicle and obstacle are also iteratively updated as the predicted trajectories themselves are updated.

[0057] refer to Figure 1 The illustration shows an example of vehicle 100 (sometimes referred to herein as "the vehicle"). As used herein, "vehicle" is any form of motorized transport. In one or more implementations, vehicle 100 is an automobile. Although the arrangement will be described herein in relation to an automobile, it will be understood that these aspects are not limited to automobiles. In some implementations, vehicle 100 may be, for example, any other form of motorized transport capable of at least semi-autonomous operation.

[0058] Vehicle 100 also includes various components. It will be understood that vehicle 100 may not necessarily have in each aspect... Figure 1 All components are shown in the diagram. Vehicle 100 may have... Figure 1 Any combination of the various elements shown. In addition, besides Figure 1 In addition to the components shown, vehicle 100 may also have additional components. In some arrangements, vehicle 100 may be without... Figure 1 One or more of the elements shown are implemented in some cases. Although in Figure 1 Various components are shown as being located within vehicle 100, but it will be understood that one or more of these components may be located outside vehicle 100. Furthermore, the components shown may be physically separated by a large distance. Figure 1 The following is shown and will be included in the subsequent Figure 1 This describes some of the possible components of vehicle 100. However, for the sake of brevity in this description, only a few are included. Figure 1 Descriptions of many of the components will be provided after the remaining diagrams have been discussed.

[0059] refer to Figure 2 The illustration shows Figure 1An exemplary trajectory prediction system 170 is provided. The trajectory prediction system 170 is implemented to perform methods and other functions, as disclosed herein, related to controlling the operation of the vehicle 100, based at least in part on the past, current, observed, or predicted future trajectories of the vehicle 100 itself and / or on the past, current, observed, or predicted trajectories of one or more roadblocks located outside the vehicle 100. In some aspects, the trajectories of the vehicle 100 or the roadblocks can be modeled in three-dimensional space.

[0060] The trajectory prediction system 170 is shown as including data from... Figure 1 The vehicle 100 has one or more processors 110. Depending on the embodiment, the one or more processors 110 may be part of a trajectory prediction system 170, which may include one or more processors separate from the one or more processors 110 of the vehicle 100, or the trajectory prediction system 170 may access the one or more processors 110 via a data bus or another communication path. In one aspect, the trajectory prediction system 170 includes a memory 172 that stores at least a trajectory prediction module 174 and a control module 176. The memory 172 may be random access memory (RAM), read-only memory (ROM), hard disk drive, flash memory, or other suitable memory for storing modules 174, 176. Modules 174, 176 are, for example, computer-readable instructions that, when executed by the one or more processors 110, cause the one or more processors 110 to perform the various functions disclosed herein.

[0061] By combining the predicted trajectory of vehicle 100, trajectory prediction system 170 can store various types of model-related data 178 in database 180. For example... Figure 1 As shown, the trajectory prediction system 170 can receive sensor data from the sensor system 120 in the vehicle 100 (the vehicle itself). For example, in some aspects, the trajectory prediction system 170 receives image data from one or more cameras 126. Depending on the specific embodiment, the trajectory prediction system 170 may also receive LiDAR data from LiDAR sensor 124, radar data from radar sensor 123, and / or sonar data from sonar sensor 125. In some aspects, the trajectory prediction system 170 also receives input from vehicle system 140. Examples include, but are not limited to, steering wheel angle, accelerator pedal (accelerator) position, linear velocity, and angular velocity. Steering wheel angle and accelerator pedal position data are examples that may be referred to as Controller Area Network (CAN bus) data, and linear velocity and angular velocity are examples that may be referred to as Inertial Measurement Unit (IMU) data. As further explained below, some of the sensor data of the above types are related to predicting the trajectory of vehicle 100 (the vehicle itself), but not to predicting the trajectory of external roadblocks. Additionally, as Figure 1 As indicated, the trajectory prediction system 170, particularly the control module 176, can communicate with the vehicle system 140 and / or (one or more) autonomous driving modules 160 to assist in semi-autonomous or autonomous control of various functions of the vehicle 100. The control module 176 also includes instructions that cause one or more processors 110 to control the operation of the user interface system 182 and coordinate data including the predicted trajectory provided to various displays throughout the vehicle 100.

[0062] In some aspects, other or additional types of data from sensor system 120, such as radar and / or sonar data, can be fed into trajectory prediction system 170. Additionally, more highly structured data, such as rasterized map data (e.g., an occupancy grid for the environment surrounding vehicle 100), can be fed into the variational trajectory predictor. Depending on the aspect, the specific types of raw sensor data or structured data fed into trajectory prediction system 170 can vary.

[0063] Among the aspects described below, including the confidence score, the confidence score can be calculated at least in part based on the number of iterations occurring between the predicted trajectory of the vehicle and the predicted trajectory of the roadblock, while the predicted trajectories of the vehicle and the roadblock are being iteratively updated. Generally, a larger number of iterations corresponds to a higher confidence level for the resulting predicted trajectory, because after sufficient iterations, the predicted trajectory tends to converge to a more stable prediction.

[0064] As described in detail herein, trajectory prediction module 174 typically includes instructions for one or more processors 110 to generate one or more predicted trajectories for vehicle 100 (the vehicle itself) and one or more predicted trajectories for at least one external roadblock, which are displayed on a user interface. Various user interface designs can be useful for displaying trajectory information related to the present technology, and the description provided herein is not intended to limit the types of displays useful to the present technology.

[0065] Figure 3A partial perspective view of an exemplary vehicle interior compartment 50 is provided, illustrating two front seats for vehicle passengers and various vehicle controls. As can be seen, the vehicle includes a navigation display 52 and a head-up display (HUD) 54 projected onto a windshield 56, the HUD 54 having multiple panels 58 that can accommodate displays for a user interface. Multi-information displays (MIDs) 60, such as screens / displays that can switch between different information displays, can also be used in various areas of the vehicle interior compartment. In other aspects, personal electronic devices such as telephones 62, tablet computers (not shown), etc., can also be used for display purposes. Depending on the specific aspect, many variations of the architecture just described are possible. In various aspects, the systems and methods provided herein can include the use of a roadblock provided as a vehicle, motorcycle, bicycle, and / or pedestrian; and the vehicle user interface is one of a navigation display, multi-information display, head-up display (HUD), head-mounted display (HMD), remote operator display, and wearable device. Multiple displays can be used in combination with each other and can include different viewing angles.

[0066] Figures 4A to 4E Five examples of user interfaces are shown, which have images of a top-down plan view 300 representing the vehicle 100 and at least one obstacle such as vehicle 200 or pedestrian 204 with corresponding tracks 202, 206. Figures 5A to 5E The five illustrated user interface examples show the overlap of the trajectory and the vehicle, with the user interface displaying the relationship between the trajectory and the vehicle. Figures 4A to 4E The image provided is a frontal perspective view 310 or a driver's view of the same vehicle 100 and (one or more) roadblock vehicles 200 or pedestrians 204 and their corresponding trajectories 202, 206 in the same scene.

[0067] For example, Figure 4A The illustration shows a top plan view 300 with two roadblock vehicles 200. Although in Figure 4A The predicted trajectory 202 of the roadblock does not overlap, but in Figure 5A In the driver's perspective 310, the predicted trajectories 202 of the roadblocks are close to each other, and the overlapping parts may cause confusion for the user. Figures 4B to 4D The illustration shows a top view 300 of the vehicle 100 with its predicted trajectory 102 and multiple blocking vehicles 200 and their corresponding blocking predicted trajectories 202 under different traffic modes. As shown, the various predicted trajectories 102, 202 not only overlap with each other, but also overlap with some of the blocking vehicles themselves, which may cause confusion for the user. Figures 5B to 5DThe illustration shows the driver view 310 of these vehicles 100, 200 and a similar overlap of the predicted trajectories 102, 202, which also provides a complex visualization that could potentially interfere with visibility and / or cause confusion for the user. Figure 4E and Figure 5E The illustration shows the predicted trajectory 102 and the vehicle 100 adjacent to a pedestrian crossing, which has multiple pedestrians 204 and their corresponding predicted trajectories 206, with various overlaps.

[0068] Figure 6 This is a flowchart of a method 320 for modifying at least one predicted trajectory based on the overlap of a predicted trajectory with an object, according to an illustrative aspect of the present technology. Method 320 is used to generate a trajectory for a vehicle user interface displaying a driver's view. The method first includes generating a predicted trajectory 102 for the vehicle 100 and generating at least one predicted trajectory 202 for an obstacle located outside the vehicle 100. The obstacle (one or more) can be another vehicle 200, a pedestrian 204, or a combination thereof. As indicated in method step 322, after generating the corresponding predicted trajectories 102, 202, method 320 continues by determining that at least one predicted trajectory 202 for an obstacle exists behind an object when viewed from the driver's perspective, indicating travel behind the object. As indicated in method step 324, the method further determines whether the predicted trajectory 202 overlaps with an object when displayed on the user interface displaying the driver's view. Unless otherwise stated, the term "object" used in conjunction with the methods described herein may broadly include: static objects, such as parked vehicles, buildings, medians, etc.; and / or may also include moving objects, which may include moving vehicles, moving pedestrians, etc. Method 320 then includes modifying at least one predicted obstacle trajectory 202 to eliminate overlap, as indicated in method step 326. The method then continues to update the display of the user interface to include any modified predicted obstacle trajectory(s). (Revisit) Figure 2 In various aspects, the control module 176 can be used to provide instructions to one or more processors 110 and / or user interface system 182 to update the user interface to include the display of any modified roadblock prediction trajectory.

[0069] Figure 7 This is a flowchart of method 328, which modifies at least one predicted trajectory based on the overlap between the predicted trajectory and the object. Method 328 and Figure 6 Method 320 is similar but has additional features. For example... Figure 7 As shown, there is an additional method step 330: determining whether the object is an obstacle with its own predicted trajectory. If so, no modification is made. Otherwise, modification is made.

[0070] Figures 8A to 8C The diagram illustrates the following: Figures 6 to 7 The method is based on the modification of at least one predicted trajectory that overlaps with the object. Figure 8A The plan view 300 shows two roadblock vehicles 200A and 200B, each with corresponding predicted roadblock trajectories 202A and 202B, marked with arrows indicating the direction of travel. The roadblock vehicles 200A and 200B are traveling in opposite directions. From the driver's perspective, vehicle 200A will be traveling behind object 208, and vehicle 200B will be traveling in front of object 208. Figure 8B Provided such as Figure 8A The driver's perspective 310 presents the situation. (For example...) Figure 8B As shown, the predicted trajectory 202A of the roadblock may cause confusion because it overlaps with object 208, and it appears that vehicle 200A will travel in front of object 208, when in fact vehicle 200A will travel behind object 208. Figure 8C The resulting modification is illustrated: the length of the predicted trajectory 202A of the roadblock is shortened so that it no longer overlaps with object 208.

[0071] Figures 9A to 9C The illustration shows a modification of the predicted trajectory of at least one roadblock based on the overlap of two objects, according to an illustrative aspect of the present technology. Figure 9A The plan view 300 shows two roadblock vehicles 200A and 200B, each with corresponding roadblock prediction trajectories 202A and 202B, marked with arrows indicating the direction of travel. The roadblock vehicles 200A and 200B are traveling in opposite directions. From the driver's perspective, vehicle 200A will be traveling behind the first object 208, and vehicle 200B will be traveling in front of the first object 208 but behind the second object 110. Figure 9B Provided such as Figure 9A The driver's perspective 310 presents the situation. (For example...) Figure 9B As shown, the predicted trajectories 202A and 202B of the roadblock may cause confusion because they both overlap with objects 208 and 210, and it appears that vehicles 200A and 200B will both be traveling in front of objects 208 and 210. In reality, vehicle 200A will be traveling behind the first object 208, and vehicle 200B will be traveling in front of the first object 208 and behind the second object 210. Figure 9C The illustration shows the resulting modification: the lengths of the predicted trajectories 202A and 202B of the two roadblocks are shortened so that they no longer overlap with objects 208 and 210.

[0072] Figure 10 This is a flowchart of method 332 for modifying at least one obstruction prediction trajectory based on overlap with an object by hiding, diluting, and / or blending the predicted trajectory, according to various aspects of the present technology. Method step 334 determines whether the obstruction prediction trajectory overlaps with an object when displayed on the user interface from the driver's perspective. Similar to method 320 of Figure 8, method 332 also determines whether the obstruction prediction trajectory exists partially or completely behind the object, indicating travel behind the object, as illustrated in method step 336. If so, method 332 modifies the display of the object and the prediction trajectory. For example, part or all of the obstruction prediction trajectory can be modified to be hidden by the object or behind the object, as illustrated in method step 338. Otherwise, method 332 modifies part or all of the prediction trajectory to dilute, blend, etc., as illustrated in method step 340, to allow the user to better understand the context. In other aspects, the method includes instructions for performing the following operations: modifying the display of an object such that a first portion of the predicted trajectory of the roadblock appears to be hidden behind the object when displayed on a user interface showing a view of the driver's perspective; and modifying a second portion of the predicted trajectory of the roadblock using at least one technique selected from the group consisting of hiding, diluting, and blending.

[0073] Figures 11A to 11E The diagram illustrates the following: Figure 10 Method 332 provides various modifications to at least one roadblock prediction trajectory. Figure 11A The plan view 300 shows two roadblock vehicles 200A and 200B, each with corresponding predicted roadblock trajectories 202A and 202B, marked with arrows indicating the direction of travel. The roadblock vehicles 200A and 200B are traveling in opposite directions. From the driver's perspective, vehicle 200A will be traveling behind object 208, and vehicle 200B will be traveling in front of object 208. Figure 11B Provided such as Figure 11A The driver's perspective 310 presents the situation. (For example...) Figure 11B As shown, the predicted trajectory 202A of the roadblock may cause confusion because it overlaps with object 208 and appears to be traveling in front of object 208, when in fact, vehicle 200A will be traveling behind object 208. Figure 11C The first modification provided is that both the predicted trajectories 202A and 202B of the roadblock are modified so that a portion of each is hidden behind object 208. This provides the user with a clearer view of object 208; however, there may be ambiguity regarding whether the second vehicle 200B is traveling in front of or behind object 208. Figure 11D and Figure 11E This can provide users with a clearer understanding of the situation. Figure 11D In the image, the predicted trajectory 202A of the roadblock vehicle 200A traveling behind the object 208 is partially hidden behind the object 208, while the predicted trajectory 202B of the roadblock vehicle 200B traveling in front of the object 208 is partially blended with the object 208. Figure 11E In the process, the predicted trajectory 202A of the roadblock vehicle 200A traveling behind the object is partially diluted behind the object 208, while the predicted trajectory 202B of the roadblock vehicle 200B traveling in front of the object 208 is partially blended with the object 208. Different combinations of hiding, blending, and dilution can be used.

[0074] Figure 12 This is a flowchart of a method 342 for modifying at least one predicted trajectory based on different overlap selections according to various aspects of this technology. As illustrated in method step 344, the method determines the relative positions of various roadblocks and objects displayed in a user interface display with a driver's perspective. The method then determines whether the target object overlaps with another object, as illustrated in method step 346. If an overlap exists, the method determines whether the other roadblock arrives at the overlap point before the vehicle will arrive at the overlap point. If so, step 350 of the method guides the use of... Figure 7 The method provided in [the document] modifies the predicted trajectory of the roadblock. If not, step 352 of the method guides the use of [method name]. Figure 10 The method provided in the document modifies the predicted trajectory of roadblocks.

[0075] Figures 13A to 13B The diagram illustrates the following: Figure 12 Method 342 modifies at least one predicted trajectory, while ignoring the vehicle's own trajectory for simplicity. Figure 13A The diagram 300 is a top-view plan showing three vehicles 200A, 200B, and 200C that are blocking the road in series. Each of the vehicles 200A, 200B, and 200C has a corresponding predicted trajectory 202A, 202B, and 202C. Two of the predicted trajectories, 202A and 202B, overlap with the adjacent vehicles 200B and 200C that are blocking the road. Figure 13B Provided in implementation Figure 12 Method 342 is the driver's perspective 310 afterward. For example, since there are no obstructions or objects in front of vehicle 200C, therefore Figure 7 The method was applied to the predicted trajectory 202C of the roadblock provided in the display. Since there are other vehicles in front of both vehicles 200A and 200B, Figure 10The method was applied to their respective predicted trajectories 202A and 202B. As a result, predicted trajectory 202A overlapped with vehicle 200B, but predicted trajectory 202B was hidden due to its overlap with vehicle 200C.

[0076] Figure 14 This is a flowchart of a method 354 for modifying at least one predicted trajectory based on the overlap of at least two predicted trajectories in a driver's view, using an optional confidence score, according to an illustrative aspect of the present technology. Method 354 includes generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. After generating the predicted trajectory, the method continues by determining that at least two predicted trajectories overlap when displayed on a user interface showing the driver's view, as shown in method step 356. In various aspects, method 356 may proceed directly to step 364, which includes modifying at least one roadblock predicted trajectory to eliminate the overlap. The method then continues to update the display of the user interface to include any modified roadblock predicted trajectories. In an alternative method as shown in method step 358, the trajectory prediction module 170 may include instructions for calculating or otherwise obtaining a confidence score, which represents the probability of a collision between roadblocks due to the presentation of the overlap of the roadblock predicted trajectories. The method may further include performing a comparison of confidence scores to determine if the confidence score is less than a predetermined threshold, as indicated by method step 360. Once it is determined that the collision risk is less than the predetermined threshold, the method optionally proceeds to step 362: determining whether another obstacle will arrive at the point of overlap of the predicted trajectories before the vehicle will arrive at the intersection. If so, the method includes modifying at least one predicted trajectory to eliminate the overlap, as indicated by method step 364. In various aspects, the modification may include shortening the length of the predicted trajectory, providing a spacing distance between at least two predicted trajectories, and modifying the predicted trajectory using techniques such as hiding, diluting, blending, or similarly modifying at least a portion of the predicted trajectory (and / or adjacent obstacles or objects).

[0077] Figure 15This is a flowchart of a method 366 for modifying at least one predicted trajectory based on a calculation of the shortest distance between two adjacent predicted trajectories being less than a threshold, according to an illustrative aspect of the present technology. For example, the trajectory prediction module 170 may include instructions for performing the following operations: generating a predicted trajectory for the vehicle; and generating at least one predicted trajectory for a roadblock located outside the vehicle. As shown in method steps 368 and 370, the instructions may include a step for determining that the distance between two adjacent predicted trajectories is less than a predetermined threshold when displayed on a user interface showing a view of the driver's perspective. Thereafter, as shown in method step 372, the instructions may include a step for performing at least one modification selected from the group consisting of: (1) changing the color of at least one predicted trajectory among the predicted trajectories; (2) changing the spacing of at least one predicted trajectory among the predicted trajectories; (3) changing the thickness of at least one predicted trajectory among the predicted trajectories; and (4) determining a priority predicted trajectory based on the closest proximity to the vehicle, and displaying only the priority predicted trajectory. Control module 176 may also provide instructions that, when executed by one or more processors 110 or user interface system 182, cause one or more processors 110 or user interface system 182 to update the user interface to include any modified roadblock prediction trajectory.

[0078] Figures 16A to 16C The diagram illustrates the following: Figure 14 Method 354 modifies the predicted trajectory of at least one roadblock and the vehicle itself. Figure 16A The illustration shows a top view 300 of the vehicle 100 with its predicted trajectory 102 and two obstructing vehicles 200A and 200B and their corresponding predicted trajectories 202A and 202B. As shown, the predicted trajectory 102 of the vehicle overlaps with the predicted trajectory 202B of the obstructing vehicle, and the predicted trajectory 202A of the other obstructing vehicle overlaps with the predicted trajectory 202A of the vehicle 100. Figure 16B A driver's perspective 310 is provided, which completely hides (eliminates) the roadblock vehicle 200A and its predicted trajectory 202A, and shortens the length of the vehicle's predicted trajectory 102 to eliminate the overlap of predicted trajectories 102 and 202B. Figure 16C A driver's perspective 310 is provided, which completely hides (eliminates) the roadblock vehicle 200A and its predicted trajectory 202A, and mixes the color of the vehicle's predicted trajectory 102 to minimize the overlap of predicted trajectories 102 and 202B.

[0079] Figure 17 The illustration shows the user interface display from the driver's perspective 310 when roadblocks 200A and 200B overlap with predicted trajectories 202A and 202B. Figures 18A to 18E The diagram illustrates the following: Figures 14 to 15The method Figure 17 At least one predicted trajectory modification. For example, in Figures 18A to 18B In this context, the color, gradient, or pattern of one obstacle's predicted trajectory 202A can be altered to make it appear different from another obstacle's predicted trajectory 202B. Figure 18C In the process, the predicted trajectories 202A and 202B for the roadblock can be provided with different thicknesses. Figure 18D In the predicted trajectory, one of the predicted trajectories, 202A, can be shifted or separated by a distance "a" to provide a larger interval between adjacent predicted trajectories 202A and 202B. Figure 18E In the middle, one of the predicted trajectories 202A can be completely removed (hidden) from the view.

[0080] Figure 19 This is a flowchart of method 374, based on an illustrative aspect of the present technology, for modifying at least one predicted trajectory based on the overlap of at least two predicted trajectories or one or more predicted trajectories with one or more objects, in the case of determining a preferred intersection point. After generating the desired predicted trajectory, the method includes determining whether at least two predicted trajectories (or trajectories with objects) overlap at a first intersection point (as shown in method step 376), and determining whether at least two predicted trajectories (or trajectories with objects) overlap at a second intersection point (as shown in method step 378). When at least two overlaps are located, method step 380 provides the determination of the preferred intersection point. This determination is based on a calculation of which roadblocks will arrive at one of the first and second intersection points first. Once the preferred intersection point is located, the method includes modifying at least one predicted trajectory that overlaps with the preferred intersection point, as shown in method step 382.

[0081] To further explain Figure 19 Method 374 Figure 20 The diagram is provided as a set of four vehicles with predicted trajectories overlapping at four intersections with various intersection times. Figure 20 This includes four intersections labeled A, B, C, and D. Assuming the vehicles are traveling at the same speed, the priority intersection is intersection B, which will occur first in time. Intersection C should occur last in time, with intersections A and D occurring sometime between B and C.

[0082] Figures 21A to 21B Further illustrations are provided based on Figure 19 The method modifies at least one predicted trajectory. Figure 21AThis is a top-down plan 300 showing the vehicle 100, the blocking vehicle 200, and two pedestrians 204, each with their corresponding predicted trajectories 102, 202, 206A, and 206B. Two stop signs 214 are provided at the road intersections. Since the blocking vehicle 200 is at the stop sign, the priority intersection point is between the vehicle 100 and one of the pedestrians 204A or 204B, depending on the pedestrian's speed. The final intersection point in time will be between the vehicle 100 and the blocking vehicle 200. Figure 21B Provided in Figure 21A The driver's perspective 310 presents the situation, and the vehicle's predicted trajectory 102 has been modified by shortening its length, since the vehicle's predicted trajectory 102 involves priority intersections.

[0083] Figure 22 This is a flowchart of a method 384 for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with the object, in accordance with the illustrative aspects of the present technology, when it is determined that the object is a partially or completely hidden obstacle. After generating the desired predicted trajectory, the method includes step 386: determining, when considering the driver's viewpoint, that the obstacle's predicted trajectory exists behind the object, indicating that travel is being made behind the object. As shown in method step 388, the method determines that the obstacle's predicted trajectory overlaps with the object when displayed on the user interface. If, in method step 390, it is determined that at least one hidden obstacle is located behind the object when displayed on the user interface showing the driver's viewpoint, then step 392 provides to perform at least one modification selected from the group consisting of: eliminating both the hidden obstacle and the corresponding obstacle's predicted trajectory; hiding, diluting, or blending a portion of the obstacle's predicted trajectory with the object; and overlaying the hidden obstacle onto the object.

[0084] Figures 23A to 23D The diagram illustrates what needs to be done according to... Figure 22 The method modifies at least one example case of the predicted trajectory. For example, Figure 23A A top plan view 300 is provided, which includes a smaller vehicle 200A positioned adjacent to a larger vehicle 200B, such as a truck. Figure 23B The illustration shows a top plan view 300 including a vehicle 200 adjacent to an object such as building 212. Figure 23C and Figure 23D They provided Figure 23A and Figure 23B The driver's perspective 310 shows the situation presented in the video, in which no vehicle blocking the road is seen. Figures 24A to 24D The diagram illustrates when using Figure 23C and Figure 23D When the situation is presented according to Figure 22 At least one additional modification to the method for predicting trajectories. Figure 24AIn both of these scenarios, the hidden obstructing vehicle and the corresponding obstructing predicted trajectory are completely eliminated. Figure 24B The hidden obstacles have been eliminated, but the predicted trajectory is still presented. Figure 24C The predicted trajectory is mixed with the roadblock vehicle 200B and building 212. Figure 24D Overlay the hidden roadblocks 200 and 200A with building 212 and roadblock vehicle 200B respectively.

[0085] Figure 25 This is a flowchart of a method 394 for modifying at least one predicted trajectory based on the presence of sloping terrain in a road, according to an illustrative aspect of the present technology. Method 394 is used to generate a trajectory for a vehicle user interface showing a driver's view, and includes: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for an obstacle located outside the vehicle. As indicated in method step 396, after generating the corresponding predicted trajectory, method 394 continues by determining that at least one predicted trajectory for an obstacle exists behind an object when viewed from the driver's perspective, the existence behind the object indicating travel behind the object. The method also determines that the predicted trajectory for the obstacle overlaps with the object when displayed from the driver's perspective. Specifically, in this aspect of the method, it is determined that the object is sloping terrain, and the predicted trajectory is at least partially hidden, as indicated in method step 400. Figure 26A As shown, the sloping terrain 214 can be uphill or downhill. As indicated in method step 402, the method continues to modify at least one roadblock prediction trajectory for display as a curved trajectory extending over the sloping terrain, such that it is visible when displayed on a user interface showing the driver's view. The method then continues to update the display of the user interface to include any modified roadblock prediction trajectory(s).

[0086] Figures 26A to 26D and Figures 27A to 27D The diagram illustrates the following: Figure 25 The method modifies at least one predicted trajectory in cases where sloping terrain obstructs the view. Figure 26A This is a partial perspective view of an example scene featuring the vehicle 100 and a roadblock vehicle 200 with its predicted trajectory 202. Figure 26B A side view plan of the elevation difference is provided, specifically showing the sloping terrain 214, and in which the predicted trajectory 202 of the roadblock is pointing upwards due to the slope, which is undesirable. Figure 26C A top-down plan of 300 is provided, and Figure 26D A driver's perspective 310 is provided, which has a hidden roadblock vehicle 200 and only shows a portion of the predicted trajectory 202, which may cause confusion for the user regarding both the location of the roadblock and its direction of travel. Figure 27A and Figure 27B The illustration shows how the predicted trajectory of a roadblock vehicle 200 changes from a straight line 202 to a curved or at least partially curved trajectory 203, which may extend at a substantially fixed distance from sloping terrain (above) such that it does not indicate a direction that appears to be up to the sky or down to the road. For example, if the distance between trajectory 202 and the road / terrain becomes increasingly greater than a predetermined threshold (indicating upward travel), at least a portion of the predicted trajectory may curve downward 203, as... Figure 27A As shown in the example. In another example, if the distance between trajectory 202 and the road / terrain becomes increasingly smaller than another predetermined threshold, then at least a portion of the predicted trajectory may curve upwards 203, as shown in the example. Figure 27B As shown in the image. Figure 27C The illustration shows a curve prediction trajectory 203. Figure 27A The scene is from the driver's perspective 310. Figure 27D An overlay representation of the roadblock vehicle 200 is also provided, which may be provided with color and / or shape to indicate that it is hidden, located on the other side of the slope, etc.

[0087] Figure 28 This is a flowchart of a method 404 for modifying at least one predicted trajectory based on at least one predicted trajectory of a roadblock extending from a display area, which would otherwise provide the user with an unknown direction of travel for the roadblock. Method 404 is used to generate a trajectory for a vehicle user interface showing a driver's view, and includes: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. As indicated in method step 406, after generating the corresponding predicted trajectory, method 404 may use a trajectory prediction module having instructions to determine that at least one predicted trajectory of a roadblock extends from a display area and also has an unknown direction of travel when displayed on the user interface showing a driver's view, as shown in method step 408. The method may include step 410 for modifying at least one predicted trajectory of a roadblock to provide directional indication in the display area.

[0088] Figures 29A to 29E The diagram illustrates the following: Figure 28 The method modifies at least one predicted trajectory. Figure 29A The illustration shows a top view 300 of the vehicle 100 and a roadblock vehicle 200. The roadblock vehicle 200 and the directional arrow are both shown extending a certain distance away from the display. Figure 29B The illustration shows the scene from the driver's perspective 310. As shown, only a portion of the predicted trajectory 202 of the roadblock exists, causing confusion for the user regarding the direction of travel. Figure 29CAn icon representing at least a partial view of the roadblock vehicle 200 has been added; however, the lack of directional arrows may still confuse users. Figure 29D and Figure 29E The length of the roadblock prediction trajectory 202 has been modified, and directional arrows have been added to the appropriate ends of the prediction trajectory to provide additional information to the user from the driver's perspective.

[0089] Figure 30 This is a flowchart of a method 412, based on an illustrative aspect of the present technology, to modify at least one predicted trajectory based on the overlap of the predicted trajectory with one or more static objects. Method 412 is used to generate a trajectory for a vehicle user interface showing a driver's view, and includes: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. As indicated in method step 414, after generating the corresponding predicted trajectory, method 412 continues by determining that at least one roadblock predicted trajectory overlaps with an object or another roadblock predicted trajectory. The method continues to step 416 to determine whether the roadblock predicted trajectory overlaps with one or more static objects (particularly static objects not located on a road or sidewalk). Non-limiting examples of such static objects may include a row of trees, a series of buildings or structures, etc. Depending on the size and location of the multiple static objects, the multiple static objects may be considered as a group or individually. As shown in method step 418, the method includes modifying the roadblock predicted trajectory by hiding, diluting, or blending the roadblock predicted trajectory at one or more locations where the predicted trajectory overlaps with one or more static objects.

[0090] Figures 31A to 31C The diagram illustrates the following: Figure 30 The method modifies at least one predicted trajectory adjacent to multiple static objects. Figure 31A A top-down plan view 300 is provided for the vehicle 100 and the blocking vehicle 200. The predicted trajectory 202 is in front of multiple static objects 216. When... Figure 31B When presented from the driver's perspective 310, the relationship between the predicted trajectory 202 of the roadblock and the static object 216 would appear chaotic. Therefore, as shown... Figure 31C As shown, the predicted trajectory 202 for the roadblock is modified such that the area / location of the predicted trajectory 202 that overlaps with object 216 is provided as blended with object 216. In other respects, depending on the type of static object and alternatively other factors, those portions may be diluted and / or hidden.

[0091] Figure 32This is a flowchart of a method 420 for modifying at least one predicted trajectory based on the overlap of the predicted trajectory with an object or another predicted trajectory, according to an illustrative aspect of the present technology, when it is determined that the predicted trajectory is a past roadblock trajectory. Typically, the roadblock predicted trajectory is a current and / or future trajectory. However, in various aspects, providing an indication of at least one past trajectory or a roadblock trajectory that has already crossed a portion of the road and no longer poses a threat of collision with the vehicle may be beneficial or desirable. Method 420 is used to generate a trajectory for a vehicle user interface showing a driver's view and includes: generating a vehicle predicted trajectory for the vehicle; and generating at least one roadblock predicted trajectory for a roadblock located outside the vehicle. As indicated in method step 422, after generating the corresponding predicted trajectory, method 420 continues by determining that at least one roadblock predicted trajectory overlaps with an object or another roadblock predicted trajectory. The method continues to step 424 to determine that the roadblock predicted trajectory is indeed a past roadblock predicted trajectory. In other words, travel across the road has occurred. As shown in method step 426, the method includes, for example, modifying the predicted trajectory of the past time roadblock by altering or changing the shape or thickness of the predicted trajectory.

[0092] Figures 33A to 33C The diagram illustrates the following: Figure 32 The method modifies at least one predicted trajectory. For example, Figure 33A A top-view plan 300 is provided, showing the vehicle 100, a first roadblock vehicle 200A with a current or future trajectory 202A, and a second roadblock vehicle 200B with a past roadblock predicted trajectory 202B. Figure 33B Provided such as Figure 33A The scene provided is from the driver's perspective 310. For clarity, the predicted trajectory of the vehicle itself has been omitted. However, the presence of the predicted trajectory 202B for the past roadblock may confuse the user. Therefore, Figure 33C It provides modifications to the shape and / or thickness of the predicted trajectory of the past roadblock.

[0093] Figure 34This is a flowchart of method 428, which selects the type of display to be provided in a user interface based on complexity according to an illustrative aspect of the present technology. Method 428 is used to generate a trajectory for a vehicle user interface displaying a driver's viewpoint and includes: generating a predicted trajectory for the vehicle itself; and generating at least one predicted trajectory for a roadblock located outside the vehicle. As indicated in method step 430, after generating the corresponding predicted trajectory, method 428 continues by determining that at least one predicted trajectory for a roadblock overlaps with an object or another predicted trajectory for a roadblock. The method continues to step 432, in which the trajectory prediction module determines that the display in the user interface displaying the viewpoint, when presented / displayed from the driver's perspective, may be complex or cluttered. In various aspects, the determination of complexity may be based on one or more of a number of factors, including: the threshold number and / or type of roadblocks and objects present in the display; the location of the roadblocks, objects, and predicted trajectories; the time of day; traffic congestion; weather; the user's experience; the duration of the journey; and so on. In various aspects, the determination of complexity can be based on one or more calculations or confidence scores, which can be based on, for example, the density of the predicted trajectory, the number of roadblocks and / or objects, and the number of overlapping points. As provided in method step 434, the method includes generating a display in a user interface showing a top-down plan, contrasting it with the driver's perspective. The use of the top-down plan is intended to simplify the display and provide the user with a more complete view of the surrounding environment and scene. In various aspects, the method may include request instructions to obtain selection requests from the user and allow the user to preview different display options and freely switch between different views. In various other aspects, the method may include providing the user with two types of displays, for example, providing a top-down plan in a first display and a driver's perspective display in a second display. In other aspects, a single display can be generated that provides both the top-down plan and the driver's perspective in a side-by-side arrangement, etc.

[0094] Regarding the display of icons and depictions of predicted trajectories, roads, objects, etc., it should be understood that this technique should not be limited to the specific types and styles described herein, and they can be customized as desired. In this regard, Figures 35A to 35C The illustrations show various non-limiting variations of predicted trajectories with different lines, 2D patterns, and 3D shapes. For example, Figure 35A Different types of line and arrow combinations and designs are provided, which can include separate sections of indicators that can indicate changes in driving direction (in...). Figure 35A (The point is shown in the middle). Figure 35B Provides a replacement Figure 35ADifferent types of two-dimensional shapes for lines. Two-dimensional shapes can be designed with different gradients or colors to better indicate driving direction, speed, confidence level, etc. Figure 35C Provides a replacement Figure 35A Different types of 3D shapes for lines. 3D shapes can be similarly designed with different gradients or colors to better indicate direction of travel, speed, confidence level, etc. 3D shapes can be provided with various levels of detail, ranging from using lines and simple shapes to displaying detailed 3D objects.

[0095] Figures 36A to 36C The illustration shows the 2D and 3D predicted trajectories when the two predicted trajectories overlap. Figure 36A The illustration shows the previous relative to Figures 16A to 16C The scenario under discussion involves the predicted trajectory 102 of the vehicle intersecting and overlapping with the predicted trajectory 202 of the roadblock. Figure 36B Two sets of two-dimensional shapes with varying colors and gradients are provided to represent the predicted trajectories 102 and 202. Figure 36C Two sets of three-dimensional shapes with varying colors and gradients are provided to represent the predicted trajectories 102 and 202.

[0096] Figures 37A to 37C The illustration shows 2D and 3D predicted trajectories when the predicted trajectory overlaps with at least one static object. Figure 37A The illustration shows the previous relative to Figures 31A to 31C The scenario discussed involves a vehicle 200 blocking the road while traveling adjacent to multiple static objects 216. Figure 37B A two-dimensional shape for the predicted trajectory 202 is provided, which has varying colors and gradients. Figure 37C It provides variations in blending, dilution, and hiding to represent the three-dimensional shape of the predicted trajectory 202.

[0097] Figures 38A to 38B The illustration shows the use of a combination of 2D and 3D predicted trajectories on a single display. Figure 38A A top-down plan of 300 is provided, and Figure 38B Provided in Figure 38A The scene presented is from the driver's perspective 310. Specifically, Figure 38B A vehicle prediction trajectory 102 with a simple line pattern is provided, a first obstacle prediction trajectory 202A with a three-dimensional shape is provided, and a second obstacle prediction trajectory 202B with a two-dimensional shape is provided. It should be understood that various modifications and combinations can be used with this technology. In various aspects, users can customize and change the display type, and the system and method may include changing the shape and / or size of the prediction trajectory based on predetermined thresholds and requirements.

[0098] Each of the various methods described herein can be provided as part of a system that may include one or more processors and memory, the memory including a trajectory prediction module that, when executed by one or more processors, causes the processors to perform actions for executing the steps described in the various parts of the method. Similarly, each of the methods described herein can be stored as instructions on a non-transitory computer-readable medium.

[0099] Now, Figure 1 The example vehicle environment in which the systems and methods disclosed herein can operate is discussed in full detail. In some instances, vehicle 100 is configured to selectively switch between autonomous mode, one or more semi-autonomous operating modes, and / or manual mode. This switching, also known as handover when transitioning to manual mode, can be implemented in a suitable manner now known or developed later. “Manual mode” means performing all or most of the navigation and / or manipulation of the vehicle based on input received from a user (e.g., a human driver / operator).

[0100] In one or more aspects, vehicle 100 is an autonomous vehicle. As used herein, “autonomous vehicle” means a vehicle operating in an autonomous mode. “Autonomous mode” means using one or more computing systems to control vehicle 100 to navigate and / or maneuver vehicle 100 along a driving route with minimal or no input from a human driver / operator. In one or more aspects, vehicle 100 is highly automated or fully automated. In one aspect, vehicle 100 is configured with one or more semi-autonomous operating modes, in which one or more computing systems perform a portion of the navigation and / or maneuvering of the vehicle along the driving route, and a vehicle operator (i.e., driver) provides input to the vehicle to perform a portion of the navigation and / or maneuvering of vehicle 100 along the driving route. Thus, in one or more aspects, vehicle 100 operates autonomously according to a particularly defined level of autonomy. For example, vehicle 100 may operate according to SAE Autonomous Vehicle Classification 0-5. In one aspect, vehicle 100 operates according to SAE Level 2, which specifies that autonomous driving module 160 controls vehicle 100 by braking, acceleration, and steering in the absence of operator input, but the driver / operator will monitor driving and remain vigilant and prepared to intervene in control of vehicle 100 if autonomous module 160 fails to respond properly or otherwise fails to adequately control vehicle 100.

[0101] Vehicle 100 may include one or more processors 110. In one or more arrangements, the processor(s)110 may be the main processor of vehicle 100. For example, the processor(s)110 may be an electronic control unit (ECU). Vehicle 100 may include one or more data storage devices 115 for storing one or more types of data. Data storage devices 115 may include volatile and / or non-volatile memory. Examples of suitable data storage devices 115 include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, disks, optical disks, hard disks, or any other suitable storage media or any combination thereof. Data storage device 115 may be a component of the processor(s)110, or data storage device 115 may be operatively connected to the processor(s)110 for use therewith. The term “operatively connected” as used throughout this specification may include direct or indirect connections, including connections without direct physical contact.

[0102] In one or more arrangements, one or more data storage devices 115 may include map data 116. Map data 116 may include maps of one or more geographic areas. In some instances, map data 116 may include information or data about roads, traffic control facilities, road markings, structures, features, and / or landmarks within one or more geographic areas. Map data 116 may take any suitable form. In some instances, map data 116 may include an aerial view of the area. In some instances, map data 116 may include a ground map of the area, including a 360-degree ground map. Map data 116 may include measurements, dimensions, distances, and / or information for one or more items included in map data 116 and / or relative to other items included in map data 116. Map data 116 may include digital maps with information about road geometry. Map data 116 may be of high quality and / or highly detailed.

[0103] In one or more arrangements, map data 116 may include one or more topographic maps 117. The topographic maps 117 may include information about the ground, topography, roads, surfaces, and / or other features of one or more geographic areas. The topographic maps 117 may include elevation data for one or more geographic areas. The topographic maps 117 may be of high quality and / or highly detailed. The topographic maps 117 may define one or more ground surfaces, which may include paved roads, unpaved roads, land, and other things that define ground surfaces.

[0104] In one or more arrangements, map data 116 may include one or more static obstacle maps 118. The static obstacle maps 118 may include information about one or more static obstacles located within one or more geographic areas. A “static obstacle” is a physical object whose location does not change or substantially does not change over a period of time and / or whose size does not change or substantially does not change over a period of time. Examples of static obstacles include trees, buildings, curbs, fences, railings, medians, utility poles, statues, monuments, signs, benches, furniture, mailboxes, large rocks, and slopes. Static obstacles may be objects extending above ground level. The one or more static obstacles included in the static obstacle maps 118 may have location data, size data, dimension data, material data, and / or other data associated with them. The static obstacle maps 118 may include measurements, dimensions, distances, and / or information for one or more static obstacles. The static obstacle maps 118 may be of high quality and / or highly detailed. The static obstacle maps 118 may be updated to reflect changes within the area where the map is drawn.

[0105] One or more data storage devices 115 may include sensor data 119. In this context, "sensor data" means any information about the sensors equipped on vehicle 100, including information about the capabilities of such sensors and other information. As will be explained below, vehicle 100 may include sensor system 120. Sensor data 119 may relate to one or more sensors of sensor system 120. As an example, in one or more arrangements, sensor data 119 may include information about one or more LIDAR sensors 124 of sensor system 120.

[0106] In some instances, at least a portion of the map data 116 and / or sensor data 119 may be located in one or more data storage devices 115 positioned on the vehicle 100. Alternatively or additionally, at least a portion of the map data 116 and / or sensor data 119 may be located in one or more data storage devices 115 remotely positioned relative to the vehicle 100.

[0107] As described above, vehicle 100 may include sensor system 120. Sensor system 120 may include one or more sensors. “Sensor” means any device, component, and / or system capable of detecting and / or sensing something. One or more sensors may be configured to detect and / or sense in real time. As used herein, the term “real time” means a level of processing response that is sufficiently immediate for a particular processing or determination to be performed by the user or system, or that enables the processor to keep up with the processing response of an external process.

[0108] In the sensor system 120, which includes an arrangement of multiple sensors, the sensors can operate independently of each other. Alternatively, two or more sensors can operate in combination with each other. In this case, the two or more sensors can form a sensor network. The sensor system 120 and / or one or more sensors can be operatively connected to one or more processors 110, one or more data storage devices 115, and / or another element of the vehicle 100 (including...). Figure 1 (Any of the elements shown). The sensor system 120 can acquire data on at least a portion of the external environment of the vehicle 100 (e.g., nearby vehicles).

[0109] Sensor system 120 may include any suitable type of sensor. Various examples of different types of sensors will be described herein. However, it will be understood that these aspects are not limited to the specific sensor described. Sensor system 120 may include one or more vehicle sensors 121. The vehicle sensors 121 may detect, determine, and / or sense information about vehicle 100 itself. In one or more arrangements, the vehicle sensors 121 may be configured to detect and / or sense changes in the position and orientation of vehicle 100, such as based on inertial acceleration. In one or more arrangements, the vehicle sensors 121 may include one or more accelerometers, one or more gyroscopes, inertial measurement units (IMUs), dead reckoning systems, global navigation satellite systems (GNSS), global positioning systems (GPS), navigation systems 147, and / or other suitable sensors. The vehicle sensors 121 may be configured to detect and / or sense one or more characteristics of vehicle 100. In one or more arrangements, the vehicle sensors 121 may include a speedometer for determining the current speed of vehicle 100.

[0110] Alternatively or additionally, sensor system 120 may include one or more environmental sensors 122 configured to acquire and / or sense driving environment data. "Driving environment data" includes data or information about the external environment in which the autonomous vehicle is located, or one or more portions thereof. For example, one or more environmental sensors 122 may be configured to detect, quantify, and / or sense obstacles and / or information / data about such obstacles in at least a portion of the external environment of vehicle 100. Such obstacles may be stationary objects and / or moving objects. One or more environmental sensors 122 may be configured to detect, measure, quantify, and / or sense other things in the external environment of vehicle 100, such as, for example, lane markings, signs, traffic lights, traffic signs, lane lines, pedestrian crossings, curbs near vehicle 100, objects outside the road, etc.

[0111] This document will describe various examples of sensors for sensor system 120. Example sensors may be portions of one or more environmental sensors 122 and / or one or more vehicle sensors 121. Furthermore, sensor system 120 may include operator sensors for tracking or otherwise monitoring aspects relevant to the driver / operator of vehicle 100. However, it will be understood that these aspects are not limited to the specific sensors described.

[0112] As an example, in one or more arrangements, sensor system 120 may include one or more radar sensors 123, one or more LiDAR sensors 124, one or more sonar sensors 125, and / or one or more cameras 126. In one or more arrangements, the one or more cameras 126 may be high dynamic range (HDR) cameras, infrared (IR) cameras, etc. In one aspect, camera 126 includes one or more cameras disposed in the passenger compartment of the vehicle for performing eye tracking on the operator / driver to determine the operator / driver's gaze, eye trails, etc.

[0113] Vehicle 100 may include an input system 130. An "input system" includes any device, component, system, element, or arrangement, or group thereof, that enables information / data to be input into the machine. Input system 130 may receive input from vehicle passengers (e.g., a driver or passenger). Vehicle 100 may include an output system 135. An "output system" includes any device, component, or arrangement, or group thereof, that enables information / data to be presented to vehicle passengers (e.g., people, vehicle passengers, etc.).

[0114] Vehicle 100 may include one or more vehicle systems 140. Figure 1 Various examples of one or more vehicle systems 140 are shown. However, vehicle 100 may include more, fewer, or different vehicle systems. It should be appreciated that while specific vehicle systems are defined separately, each or any of these systems or parts thereof may be combined or separated in other ways via hardware and / or software within vehicle 100. Vehicle 100 may include a propulsion system 141, a braking system 142, a steering system 143, a throttle system 144, a transmission system 145, a signaling system 146, and / or a navigation system 147. Each of these systems may include one or more devices, components, and / or combinations thereof now known or developed hereafter.

[0115] Navigation system 147 may include one or more devices, sensors, applications, and / or combinations thereof, now known or later developed, configured to determine the geographic location of vehicle 100 and / or determine a route for vehicle 100. Navigation system 147 may include one or more mapping applications to determine a route for vehicle 100. Navigation system 147 may include a Global Positioning System, a Local Positioning System, or a geographic location system.

[0116] One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 may be operatively connected to communicate with various vehicle systems 140 and / or their respective components. For example, return Figure 1One or more processors 110 and / or one or more autonomous driving modules 160 can communicate to send and / or receive information from various vehicle systems 140 to control the movement, speed, handling, heading, direction, etc. of vehicle 100. One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 can control some or all of these vehicle systems 140, and therefore can be partially or fully autonomous.

[0117] One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 may be operatively connected to communicate with various vehicle systems 140 and / or their respective components. For example, return Figure 1 One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 can communicate to send and / or receive information from various vehicle systems 140 to control the movement, speed, handling, heading, direction, etc. of vehicle 100. One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 can control some or all of these vehicle systems 140.

[0118] One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 may be operable to control the navigation and / or maneuvering of vehicle 100 by controlling vehicle system 140 and / or one or more of its components. For example, when operating in autonomous mode, one or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 may control the direction and / or speed of vehicle 100. One or more processors 110, trajectory prediction system 170, and / or one or more autonomous driving modules 160 may cause vehicle 100 to accelerate (e.g., by increasing the supply of fuel to the engine), decelerate (e.g., by reducing the supply of fuel to the engine and / or by applying brakes), and / or change direction (e.g., by steering the two front wheels). As used herein, “make” or “cause” means to directly or indirectly make, force, compel, instruct, command, instruct, and / or enable an event or action to occur or at least be in a state where such an event or action can occur.

[0119] Vehicle 100 may include one or more actuators 150. Actuator 150 may be any element or combination of elements operable to modify, adjust, and / or alter one or more components of vehicle system 140 or its components in response to receiving signals or other inputs from processor(s) 110 and / or autonomous driving module(s) 160. Any suitable actuator may be used. For example, one or more actuators 150 may include motors, pneumatic actuators, hydraulic pistons, relays, solenoids, and / or piezoelectric actuators, to name just a few possibilities.

[0120] Vehicle 100 may include one or more modules, at least some of which are described herein. Modules may be implemented as computer-readable program code that, when executed by processor 110, implements one or more of the various processes described herein. One or more modules may be components of processor(s)110, or one or more modules may execute on and / or be distributed among other processing systems to which processor(s)110 is operatively connected. Modules may include instructions (e.g., program logic) executable by processor(s)110. Alternatively or additionally, one or more data storage devices 115 may contain such instructions. Generally, as used herein, the term module includes routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular data type. In other respects, memory typically stores the aforementioned modules. Memory associated with a module may be a buffer or cache embedded within the processor, RAM, ROM, flash memory, or other suitable electronic storage medium. In other respects, the modules contemplated in this disclosure are implemented as application-specific integrated circuits (ASICs), system-on-a-chip (SoC) hardware components, programmable logic arrays (PLAs), or other suitable hardware components embedded with a set of defined configurations (e.g., instructions) for performing the disclosed functions.

[0121] In one or more arrangements, one or more of the modules described herein may include artificial intelligence or computational intelligence elements, such as neural networks, fuzzy logic, or other machine learning algorithms. Additionally, in one or more arrangements, one or more of the modules may be distributed among multiple modules described herein. In one or more arrangements, two or more of the modules described herein may be combined into a single module.

[0122] Vehicle 100 may include one or more autonomous driving modules 160. The autonomous driving modules 160 may be configured to receive data from sensor system 120 and / or any other type of system capable of capturing information related to vehicle 100 and / or its external environment. In one or more arrangements, the autonomous driving modules 160 may use such data to generate one or more driving scenario models. The autonomous driving modules 160 may determine the position and speed of vehicle 100. The autonomous driving modules 160 may determine the position of obstacles or other environmental features, including traffic signs, trees, bushes, nearby vehicles, pedestrians, etc.

[0123] One or more autonomous driving modules 160 may be configured to receive and / or determine location information of obstacles in the external environment of the vehicle 100 used by one or more processors 110 and / or one or more modules described herein to estimate the position and orientation of the vehicle 100, the vehicle's position in global coordinates based on signals from multiple satellites, or any other data and / or signals that can be used to determine the current state of the vehicle 100 or to determine the position of the vehicle 100 relative to its environment for the purpose of creating a map or determining the position of the vehicle 100 relative to map data.

[0124] One or more autonomous driving modules 160 may be configured independently or in combination with trajectory prediction system 170 to determine one or more driving paths, current autonomous driving maneuvers of vehicle 100, future autonomous driving maneuvers, and / or modifications to current autonomous driving maneuvers based on data acquired by sensor system 120, driving scenario models, and / or data from any other suitable source. “Driving maneuver” means one or more actions that affect the movement of the vehicle. Examples of driving maneuvers include: acceleration, deceleration, braking, steering, moving in the lateral direction of vehicle 100, changing lanes, merging into lanes, and / or reversing, to name just a few possibilities. One or more autonomous driving modules 160 may be configured to perform the determined driving maneuvers. One or more autonomous driving modules 160 may directly or indirectly cause such autonomous driving maneuvers to be performed. As used herein, “cause” or “make” means to make, command, instruct, and / or enable an event or action to occur or at least be in a state where such event or action can occur, directly or indirectly. One or more autonomous driving modules 160 may be configured to perform various vehicle functions and / or send data to vehicle 100 or one or more of its systems (e.g., one or more of vehicle systems 140), receive data from vehicle 100 or one or more of its systems (e.g., one or more of vehicle systems 140), interact with vehicle 100 or one or more of its systems (e.g., one or more of vehicle systems 140), and / or control vehicle 100 or one or more of its systems (e.g., one or more of vehicle systems 140).

[0125] Detailed aspects are disclosed herein. However, it is to be understood that the disclosed aspects are intended only as examples. Therefore, the specific structural and functional details disclosed herein are not to be construed as limiting, but merely as the basis for the claims and as a representative basis for teaching those skilled in the art to adopt the aspects herein in various ways in any suitable detailed structure in fact. Furthermore, the terminology and phrases used herein are not intended to be limiting, but rather to provide an understandable description of possible implementations. Various aspects are illustrated in the collective figures, but these aspects are not limited to the illustrated structures or applications.

[0126] The flowcharts and block diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products, depending on various aspects. In this regard, each box in a flowchart or block diagram may represent a section of code comprising one or more executable instructions for implementing a specified logical function(s). It should also be noted that in some alternative implementations, the functions indicated in the boxes may not occur in the order shown in the diagram. For example, depending on the functions involved, two boxes shown consecutively may actually execute substantially simultaneously, or these boxes may sometimes execute in reverse order.

[0127] The systems, components, and / or processes described above can be implemented in hardware or a combination of hardware and software, and can be implemented in a centralized manner in a single processing system or in a distributed manner in which different elements are distributed across several interconnected processing systems. Any kind of processing system or other apparatus suitable for performing the methods described herein is appropriate. A typical combination of hardware and software can be a processing system having computer-usable program code that, when loaded and executed, controls the processing system to perform the methods described herein. Systems, components, and / or processes can also be embedded in a machine-readable storage device such as a computer program product or other data program storage device, thereby tangibly implementing a program that is machine-executable to perform the methods and processes described herein. These elements can also be embedded in an application product that includes all the features enabling the implementation of the methods described herein and, when loaded into a processing system, enables the execution of these methods.

[0128] Furthermore, the arrangements described herein can take the form of a computer program product implemented thereon (e.g., stored thereon) on one or more computer-readable media having computer-readable program code embodied thereon. Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The phrase “computer-readable storage medium” means a non-transitory storage medium. A computer-readable storage medium may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include the following: portable computer disks, hard disk drives (HDDs), solid-state drives (SSDs), read-only memory (ROMs), erasable programmable read-only memory (EPROMs or flash memory), portable compact disc read-only memory (CD-ROMs), digital versatile discs (DVDs), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium may be any tangible medium capable of containing or storing a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0129] Program code implemented on a computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic, cable, RF, or any suitable combination thereof. Computer program code for performing operations of aspects of this arrangement can be written in any combination of one or more programming languages, including languages ​​such as Java. TMObject-oriented programming languages ​​such as Smalltalk and C++, as well as traditional procedural programming languages ​​such as "C" or similar languages, can be used. Program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The foregoing description is provided for illustrative and descriptive purposes and is in no way intended to limit this disclosure, its application, or its use. It is not intended to be exhaustive or restrictive of this disclosure. Elements or features of a particular aspect are generally not limited to that particular aspect, but are interchangeable where appropriate and can be used in chosen aspects, even if not specifically shown or described. These can also vary in many ways. Such variations should not be considered a departure from this disclosure, and all such modifications are intended to be included within the scope of this disclosure.

[0131] As used herein, at least one of the phrases A, B, and C should be interpreted as referring to logic using the non-exclusive logic "OR" (A or B or C). It should be understood that various steps within the method may be performed in different orders without altering the principles of this disclosure. Scope disclosure includes disclosures of the entire scope as well as disclosures of subdivisions (including endpoints) within the entire scope.

[0132] The headings (such as “Background Art” and “Summary of the Invention”) and subheadings used herein are intended only for the general organization of the subject matter within this disclosure and are not intended to limit the disclosure of this art or any aspect thereof. The description of multiple aspects having the stated features is not intended to exclude other aspects having additional features or other aspects incorporating different combinations of the stated features.

[0133] As used herein, the terms “comprising” and “including” and variations thereof are intended to be non-limiting, such that a description of consecutive items or a list is not intended to exclude other similar items that may also be used in the apparatus and methods of the present art. Similarly, the terms “may” and “can” and variations thereof are intended to be non-limiting, such that a description of an aspect that may or may include certain elements or features does not exclude other aspects of the present art that do not include those elements or features. The terms “a” and “an” as used herein are defined as one or more. The term “a plurality” as used herein is defined as two or more. The term “another” as used herein is defined as at least a second or more.

[0134] The extensive teachings of this disclosure can be implemented in various forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become apparent to those skilled in the art after studying the specification and the following claims. Reference herein to an aspect or aspect means that a particular feature, structure, or characteristic described in connection with an embodiment or a particular system is included in at least one embodiment or aspect. The appearance of the phrase “in an aspect” (or a variation thereof) does not necessarily refer to the same aspect or embodiment. It should also be understood that the various method steps discussed herein need not be performed in the same order as depicted, and not every method step is required in every aspect or embodiment.

Claims

1. A system for modifying the display of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising: one or more processors; and a memory communicably coupled to the one or more processors and storing a trajectory prediction module including computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: generate a host vehicle predicted trajectory for a host vehicle; generate at least one pedestrian predicted trajectory for a pedestrian located outside the host vehicle; determine that at least two predicted trajectories overlap at a first intersection when displayed on a user interface showing a driver's perspective view; determine that at least two pedestrian predicted trajectories overlap at a second intersection; determine a priority intersection based on a calculation of which pedestrians will reach one of the first intersection and the second intersection first; and control operation of the user interface to modify the at least one pedestrian predicted trajectory that overlaps at the priority intersection by eliminating the overlap from the display, the memory further storing a control module including computer readable instructions that, when executed by the one or more processors, cause the one or more processors to control operation of the user interface to update the user interface to display any modified pedestrian predicted trajectories.

2. A system for modifying the display of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising: one or more processors; and a memory communicably coupled to the one or more processors and storing a trajectory prediction module including computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: generate a host vehicle predicted trajectory for a host vehicle; generate at least one pedestrian predicted trajectory for a pedestrian located outside the host vehicle; determine that a first pedestrian predicted trajectory overlaps with a first object at a first intersection when displayed on a user interface showing a driver's perspective view; determine that a second pedestrian predicted trajectory overlaps with one of a second object and a third pedestrian predicted trajectory at a second intersection; determine a priority intersection based on a calculation of which pedestrians will reach one of the first intersection and the second intersection first; and control operation of the user interface to modify the display of the at least one pedestrian predicted trajectory that overlaps at the priority intersection by eliminating the overlap from the display, the memory further storing a control module including computer readable instructions that, when executed by the one or more processors, cause the one or more processors to control operation of the user interface to update the user interface to display any modified pedestrian predicted trajectories.

3. The system of claim 1 or 2, wherein: the pedestrian is one of a car, a motorcycle, a bicycle, and a pedestrian; and the first intersection and the second intersection are one of a crosswalk, a stop sign, a yield sign, a traffic light, and a pedestrian crossing. ​ ​ ​ The vehicle user interface is one of a navigation display, a multi-information display, a head-up display (HUD), a head-mounted display (HMD), a remote operator display, and a wearable device.

Images