Reference trajectory verification and collision check management

Abstract

Techniques are discussed herein for determining that a reference trajectory does not cross or potential collide with an object. A trajectory generated by a host computing device of a vehicle may be used to select or determine a reference trajectory followed by the vehicle. An auxiliary computing device of the vehicle may identify a current offset from the reference trajectory, and utilize the offset and a kinematic model to determine a trajectory that the vehicle is predicted to drive back to the reference trajectory. The verification of the reference trajectory may be based on predicted collision data determined using the tracker trajectory. The predicted collision data may be used to control the vehicle to follow a reference trajectory or a stop trajectory.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to U.S. utility patent application serial number 17 / 875,700, filed on July 28, 2022. Application serial number 17 / 875,700 is incorporated herein by reference in its entirety. Background Art

[0003] Certain systems, such as autonomous vehicles, may include a computing system for generating a trajectory and controlling the vehicle based on the trajectory, as well as error and state data associated with the vehicle. For example, a vehicle trajectory generated by a vehicle computing system may indicate a desired state of the vehicle and / or controls issued to the vehicle at discrete locations and / or times. The vehicle computing system may perform collision checks based on the vehicle trajectory. However, differences between the vehicle position used to generate the trajectory and the actual vehicle position may result in inaccuracies in collision checks, which may, in some cases, result in unsafe operation of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The detailed description is described with reference to the accompanying drawings. In the drawings, the leftmost digit of a reference number identifies the drawing in which the reference number first appears. The use of the same reference number in different drawings indicates similar or identical components or features.

[0005] Figure 1 is a pictorial flow chart of an example process for reference trajectory validation and collision check management according to examples of the present disclosure.

[0006] Figure 2 is a schematic diagram of tracker trajectory generation according to an example of the present disclosure.

[0007] Figure 3 Depicted is a block diagram of vehicle system components for implementing the techniques described herein.

[0008] Figure 4 Depicted is a block diagram of an example system for implementing the techniques described herein.

[0009] Figure 5 An example process for reference trajectory validation and collision check management according to examples of the present disclosure is depicted. DETAILED DESCRIPTION

[0010] This article discusses techniques for determining that a reference trajectory has no intersection or potential collision. For example, a vehicle may generate a trajectory (e.g., referred to as a reference trajectory or a first trajectory) using a main computing device, and verify the reference trajectory using a second computing device. The main computing device may generate a trajectory, select / determine an optimized trajectory from the trajectory as a reference trajectory to be followed by the vehicle, and provide the reference trajectory to an auxiliary computing device. The auxiliary computing device may identify a point (e.g., a waypoint, a location, etc.) associated with the reference trajectory and offset information associated with the point. The offset information (which may include a lateral offset and a longitudinal offset) may be used with a motion model (e.g., a kinematic and / or dynamic model) to determine a trajectory for collision checking. The trajectory for collision checking may be determined as a tracker trajectory (e.g., a second trajectory) that the vehicle is predicted to drive (or follow) to converge to the reference trajectory. The auxiliary computing device may verify the reference trajectory based on predicted collision data associated with the estimated position of the vehicle along the tracker trajectory. The predicted collision data may be used to control the vehicle to follow a reference trajectory or a stop trajectory (or a safe stop trajectory, a stop trajectory, etc.). By utilizing the tracker trajectory to identify predicted collision data (eg, corresponding collision likelihood), the reference trajectory may be utilized to control the vehicle based on accurate (eg, real-time or near real-time) current and predicted information about the vehicle.

[0011] In some cases, a vehicle traveling in an environment (e.g., an autonomous vehicle) may include a vehicle computing system for performing trajectory generation and preliminary collision checks. The vehicle computing system may include a main computing device having a planning component to generate a trajectory associated with the vehicle. Trajectories may be generated so that the vehicle can be safely and legally controlled while avoiding any possible interaction (e.g., collision or attempted collision) with objects in the environment. The planning component may select or otherwise determine an optimized trajectory based on characteristics associated with the optimized trajectory, which is more closely aligned with corresponding priorities and / or corresponding preferences than one or more remaining trajectories. The planning component may output the optimized trajectory as a reference trajectory. Additional examples of detecting objects and determining the trajectory followed by the vehicle may be found, for example, in U.S. patent application No. 15 / 843,512, entitled "Trajectory Generation Using Curvature Segments," filed on December 15, 2017, the entire contents of which are incorporated herein by reference for all purposes.

[0012] The vehicle computing system may include an auxiliary computing device with a tracker trajectory component to simulate a trajectory that the vehicle is expected to travel based on an offset from a reference trajectory (which may be referred to as a tracker trajectory, an estimated trajectory, a simulated trajectory, a predicted trajectory, etc.). The tracker trajectory component may simulate the trajectory that the vehicle is expected to follow based on the reference trajectory and current data of the vehicle (e.g., a current position) and points associated with the reference trajectory. The current data used to generate the predicted trajectory may include, for example, current lateral data and current longitudinal data. A simulated trajectory may be generated based on the current data and a vehicle motion model (e.g., a kinematic model and / or a dynamic model), and the simulated trajectory may be identified as a predicted trajectory associated with the vehicle.

[0013] For example, a kinematic model can be used to determine a predicted trajectory based on a lateral offset and a longitudinal offset relative to a position (e.g., a current position) (or "current position") associated with a reference trajectory (e.g., a point having lateral and longitudinal coordinates, such as a waypoint). The point associated with the reference trajectory can be received from a host computing device. The kinematic model (which can use the kinematic model over a constant time period over a fixed horizon) can be integrated to project (e.g., orthogonally project) current lateral data and current longitudinal data onto the reference trajectory as projected lateral data (e.g., a lateral component of a point) and projected longitudinal data (e.g., a longitudinal component of a point), respectively.

[0014] The tracker trajectory component of the auxiliary computing device can use a motion model (e.g., a kinematic model) and a lane-keeping tracker and / or a least squares model to determine the next position (e.g., the next point) of the vehicle along the tracker trajectory. For example, based on the current position (e.g., the current point) of the vehicle along the tracker trajectory, the motion model can be used to determine the tracker trajectory that the vehicle is predicted to follow to return to the reference trajectory. In some examples, the lane keeping tracker can be used with the kinematic model to determine the next point associated with the tracker trajectory based on the current lateral data. In these or other examples, the kinematic model can be used with the least squares model to determine the next point based on the control command input into the kinematic model (e.g., the control command determined using the current lateral data and / or the current longitudinal data). The lateral error can be determined based on the difference between the current lateral data and the projected lateral data. The lateral error can be identified as a lateral offset. The longitudinal error can be determined based on the difference between the current longitudinal data (e.g., the current longitudinal speed and acceleration) and the projected longitudinal data (e.g., the expected speed and acceleration along the reference trajectory). The longitudinal error may be identified as a longitudinal offset.This estimated trajectory may be generated iteratively using the same process for each time step (eg, determine control, propagate the vehicle forward using the model, determine the difference between the new state and the desired state, etc.).

[0015] Based on the lateral offset and the longitudinal offset, the tracker trajectory component can identify a predicted trajectory as a tracker trajectory that the autonomous vehicle is predicted to follow to converge to the reference trajectory. As described above, the kinematic model can be used iteratively until a tracker trajectory point (e.g., a point on the tracker trajectory determined based on the kinematic model) i) projects onto a stopping point on the reference trajectory, or ii) aligns with a fixed time horizon.

[0016] The vehicle computing system may include a collision check component for performing collision checks based on a tracker trajectory. The collision check component may perform collision checks by generating a vehicle bounding box associated with the vehicle and an object bounding box associated with an object in the environment. The vehicle bounding box may be propagated along the tracker trajectory, and a collision probability associated with the object may be determined based on the object bounding box propagated along the object trajectory of the object. A collision probability of a tracker trajectory point may be determined, the tracker trajectory point being identified by the tracker trajectory component based on a lateral offset and a longitudinal offset. The collision check component may use the tracker trajectory to verify a reference trajectory as a verified reference trajectory, and based on determining that there is no collision probability associated with the tracker trajectory, output the verified reference trajectory as a control trajectory. In some examples, the collision check component may output an alternative trajectory as a control trajectory, such as a safe stop trajectory, based on determining that there is a collision probability associated with the tracker trajectory. The tracker component may use the control trajectory to control the vehicle.

[0017] As shown in these examples, the techniques described herein can improve the functionality, safety, and efficiency of vehicles (e.g., driver-controlled vehicles with safety systems, semi-autonomous vehicles, autonomous vehicles, robotic platforms, systems, etc.) traversing an environment. In various examples, a tracker trajectory component of an auxiliary computing device can receive reference trajectory information from a planning component of a main computing device. In these examples, the tracker trajectory component can determine the tracker trajectory information using a kinematic model based on lateral data and longitudinal data of points relative to a reference trajectory. The tracker trajectory information can include a tracker trajectory associated with current and latest vehicle position information. The tracker trajectory component can provide the tracker trajectory information to a collision checking component, and the collision checking component can use the tracker trajectory instead of the reference trajectory to verify the reference trajectory.

[0018] Even in those examples where only a reference trajectory is used to incorporate lateral and / or longitudinal errors into collision checking by expanding one or more dimensions of a bounding box (or polygon) representing the vehicle, such techniques may result in overly conservative estimates for collision checking, which may lead to problems with brake checks or planning or traversing narrow areas. By planning control based on a reference trajectory and utilizing a tracker trajectory for collision checking, the vehicle may avoid errors caused by outdated data and / or inconsistencies that may occur due to differences (e.g., due to errors, drift, etc.) between the current vehicle position and the vehicle position used to generate the reference trajectory. For example, the techniques described herein use the results of collision checks performed based on a tracker trajectory to determine an output (or selected) trajectory, which may provide improvements in safety and efficiency over conventional systems.

[0019] The technology described herein can be implemented in a variety of ways. Example implementations are provided below with reference to the following figures. Although applicable to vehicles, such as autonomous vehicles, the methods, devices and systems described herein can be applied to various systems and are not limited to autonomous vehicles. In another example, these technologies can be used in aviation or navigation scenarios, or in any system configured to input data to determine the motion associated with objects in the environment. In addition, the technology described herein can be used with real data (e.g., captured using sensors), simulated data (e.g., generated by a simulator), or any combination of the two.

[0020] Figure 1 is a flow chart of an example process 100 for determining that a reference trajectory does not intersect or potentially collide with an object according to an example of the present disclosure. In an example, one or more operations of process 100 may be implemented by a vehicle computing system, such as by using Figure 4 One or more components and systems shown in and described below. For example, one or more components and systems may include Figure 4 In an example, one or more operations of process 100 may be performed by a remote system in communication with the vehicle, such as a first computing device 404 and a second computing device 418. Figure 4 452 shown in FIG. 4. In another example, one or more operations of process 100 may be performed by a combination of a remote system and a vehicle computing system. However, process 100 is not limited to being performed by such components and systems, and Figure 4 The components and systems are not limited to performing process 100 .

[0021] Operation 102 may include receiving a reference trajectory associated with a vehicle (e.g., an autonomous vehicle). Receiving the reference trajectory may include receiving, by an auxiliary computing device (or auxiliary computing component), an optimized trajectory for the autonomous vehicle to be controlled to drive (or follow). In some examples, the reference trajectory may include steering and / or acceleration data, a location and associated time point at which the vehicle is expected to be, and the like.

[0022] Based on multiple candidate trajectories generated by the main computing device, an optimized trajectory can be received from the main computing device (or main computing component). The candidate trajectory can be determined based on the detection of objects in the environment in which the vehicle is traveling, and in some examples, can be determined based on the predicted trajectory of the moving object in the environment. The main computing device can select an optimized trajectory from the candidate trajectories using the candidate trajectories generated by the main computing device. The optimized trajectory can be sent by the main computing device as a reference trajectory and received by the auxiliary computing device. Such an optimized trajectory may include a nominal trajectory determined according to various systems for traversing the environment, for example, as described in U.S. Patent No. 11,048,260 entitled "Adaptive Scaling in Trajectory Generation", the entire contents of which are hereby incorporated by reference herein.

[0023] Objects and predicted object trajectories can be determined by a host computing device based on sensor data associated with the environment. Sensor data, such as image data, radar data, lidar data, etc., can be received by the host computing device and can be received from sensors used to capture sensor data. In some examples, such sensors can be local to the vehicle or far away from the vehicle. Sensor data can be associated with various objects within the environment and these objects can be identified. Objects encountered within the environment can include dynamic objects that are moving or capable of moving (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and / or static objects (e.g., buildings, road surfaces, trees, signs, obstacles, parked vehicles, etc.). The host computing device can determine information about objects detected in the environment. Object information can include bounding boxes, object segmentation information, classification information, etc.

[0024] The initial starting position of the vehicle can be used to determine a position (e.g., a point) associated with the vehicle. The initial starting position, which can be identified as a point, can be determined based on various types of information associated with the vehicle. In some examples, the point can be received by an auxiliary computing system, and can be received from a primary computing device. As described below, the point can be associated with a reference trajectory 108.

[0025] Example 104 shows a vehicle 106 that may be located in an environment. The vehicle 106 may be configured with and / or communicate with a vehicle computing system, including a primary computing device and an auxiliary computing device. The primary computing device of the vehicle 106 may receive and / or determine a trajectory (e.g., a reference trajectory) 108 to be used, for example, to navigate the vehicle 106 in the environment.

[0026] The initial starting position of the vehicle 106 can be used to determine a position (or current position) (e.g., point) 110 associated with the vehicle 106. The initial starting position, which can be identified as a point, can be determined based on position data (e.g., geographic location), orientation data (e.g., attitude, tilt, etc.), motion data (e.g., speed, acceleration, etc.), and various other data types associated with the vehicle. The point 110 can be associated with at least one of a vehicle point (e.g., a center point of a front bumper), a metric (e.g., a predetermined distance from a vehicle point, such as a predetermined longitudinal distance), and / or any other type of characteristic and / or metric. In some examples, the point 110 can be a point at the center of the vehicle 106, where the point 110 is the same lateral distance from both sides of the vehicle 106 and the same longitudinal distance from both ends of the vehicle 106. The point 110 can be received from a host computing device, which can be associated with and / or determined based on a current trajectory (e.g., a reference trajectory 108). In some examples, vehicle 106 (and point 110 ) may not be exactly located on reference trajectory 108 due to errors caused by actual obstacles or conditions (eg, ice, snow, rain, potholes, tire slip, etc.), vehicle drift or skidding, etc.

[0027] Operation 112 may include determining a lateral offset and a longitudinal offset relative to the reference trajectory. The lateral offset and the longitudinal offset may be determined relative to point 110. The lateral offset and the longitudinal offset may be determined based on determining a distance between an actual position of vehicle 106 (or a predicted position of vehicle 106) and a corresponding point on reference trajectory 108, respectively.

[0028] In some examples, the lateral offset and the longitudinal offset may be determined based on errors associated with the predicted position of the vehicle 106. The lateral offset may be determined based on a lateral error, including one or more lateral errors associated with the yaw and / or orientation of the vehicle 106, the orientation indicating whether the vehicle 106 (e.g., a vehicle point) is at a position (e.g., a point) on the reference trajectory or at a position adjacent to the reference trajectory. For example, the lateral offset may include a yaw offset and / or an orientation offset. The lateral error may be determined based on one or more differences between the yaw (e.g., expected yaw) and / or orientation (e.g., expected orientation) associated with the reference trajectory 108 and the current yaw and / or current orientation of the vehicle 106, respectively. The longitudinal offset may be determined based on a longitudinal error, including one or more longitudinal errors associated with the speed and / or acceleration of the vehicle 106. The longitudinal error may be determined based on one or more differences between the speed (e.g., expected speed) and / or acceleration (e.g., expected acceleration) associated with the reference trajectory 108 and the current speed and / or current acceleration of the vehicle 106, respectively. The lateral offset and the longitudinal offset may be determined by an auxiliary computing component. The lateral and longitudinal offsets relative to point 110 at the initial starting position may be determined.

[0029] The host computing device of vehicle 106 may also or alternatively receive and / or determine vehicle data, such as a vehicle bounding box representing the location of vehicle 106 in an environment.

[0030] Example 114 shows a bounding box 116 associated with vehicle 106. Bounding box 116 can be identified and / or generated based on vehicle 106. Characteristics associated with vehicle 106 can be used to generate bounding box 116. Characteristics used to generate bounding box 116 may include positioning data (e.g., position), orientation data (e.g., attitude, yaw, tilt, roll, etc.), and / or other types of characteristics associated with vehicle 106. Bounding box 116 may include data representing (e.g., approximated, estimated, and / or detected) boundaries and / or other data associated with vehicle 106. Bounding box 116 may include data representing discrete top-down views of vehicle 106. Such data may include data representing specific parts or aspects of vehicle 106.

[0031] The bounding box 116 can represent the location of the vehicle 106 in the environment. In some examples, the bounding box 116 can be generated at a location associated with any portion of the vehicle 106, such as a portion of the front bumper (e.g., a side point) (e.g., the front bumper side point is used to identify the outside front corner of the bounding box 116), a portion of the rear axle (e.g., a center portion) (e.g., the rear axle center point is used to identify a center point of the bounding box 116 that is closer to one end of the bounding box 116 than another end of the bounding box 116), and / or any other portion of the vehicle 106.

[0032] The vehicle tracker data may be identified and / or generated by the vehicle 106. The vehicle tracker data may be generated by an auxiliary computing device of the vehicle 106. The vehicle tracker data may include lateral data 118 and longitudinal data 120. The offset data may indicate a distance from a reference trajectory and may include lateral offset data (or lateral data) 118 (e.g., a current lateral offset) and longitudinal offset data (or longitudinal data) 120 (e.g., a current longitudinal offset). The lateral data 118 and the longitudinal data 120 relative to the point 110 may be identified. A vehicle motion model (e.g., a kinematic model, a dynamic model, a machine learning (ML) model, etc.) may be used to determine a predicted trajectory (e.g., a tracker trajectory as described below) based on the lateral data 118 and the longitudinal data 120 relative to the point 110.

[0033] Although various techniques may be implemented using the kinematic models discussed in this disclosure, they are not limited thereto. Any technique discussed herein may be implemented using any type of motion model in a manner similar to a kinematic model.

[0034] Operation 122 may include determining a tracker trajectory based on the lateral offset and the longitudinal offset. In some examples, expected lateral offset data and / or expected longitudinal offset data may be determined for portions of the reference trajectory, and the expected lateral offset data and / or expected longitudinal offset data may be used to generate the tracker trajectory. The tracker trajectory that may be associated with the vehicle 106 may be generated by an auxiliary computing device of the vehicle 106. The tracker trajectory may simulate a trajectory that the vehicle 106 is expected to travel based on the bounding box 116, the current data of the vehicle 106, and / or the point 110. For example, the current data may include current lateral data and current longitudinal data. A simulated trajectory may be generated based on the current data and a motion model (e.g., a kinematic model), which may be identified as a predicted trajectory associated with the vehicle. The kinematic model (which may be used iteratively over a constant time period over a fixed range) may be integrated to project (e.g., orthogonally) the current lateral data and the current longitudinal data onto the reference trajectory, respectively, as projected lateral data (e.g., a lateral component of a point) and projected longitudinal data (e.g., a longitudinal component of a point). For example, the kinematic model may be used with a lane keeping tracker and / or a least squares model as described below. Figure 2 As discussed in further detail, a next location (eg, next point) associated with the bounding box 116 may be determined. A predicted trajectory may be identified as a tracker trajectory that the vehicle 106 is predicted to follow to converge to the reference trajectory 108 .

[0035] Example 124 illustrates a tracker track 126 associated with the vehicle 106. The tracker track 126 may be generated based on the bounding box 116, current data of the vehicle 106 (eg, lateral offset and longitudinal offset), and / or the points 110 using a kinematic model.

[0036] Operation 128 may include determining collision check data. The collision check data may be generated by a second computing device of the vehicle 106 based on the bounding box 116 (and / or other geometric representations of the vehicle traveling along the tracker trajectory) in combination with the detected and / or predicted motion of the detected object (as may be determined by the main computer system and / or the auxiliary computer system). The bounding box 116 may be propagated along the tracker trajectory 126 and used to determine the likelihood of a collision associated with an object in the environment. The likelihood of a collision associated with an object may be determined based on an object bounding box propagated along the object trajectory of the object. Additional examples of determining perceptual information for trajectory selection may be found, for example, in U.S. patent application Ser. No. 17 / 514,542, entitled “CollisionAvoidance and Mitigation in Autonomous Vehicles,” filed on October 29, 2021, the entire contents of which are incorporated herein by reference for all purposes.

[0037] Operation 130 may include controlling the vehicle 106. The vehicle 106 may be controlled based on the collision probability associated with the tracker trajectory 126 and the object trajectory of the objects in the environment. The collision probability and the object trajectory associated with the tracker trajectory 126 may be used to verify the reference trajectory 108 as a verified reference trajectory (or a valid reference trajectory). In some examples, the reference trajectory 108 may be verified as a verified reference trajectory based on a determination that there is no collision probability associated with the tracker trajectory. The verified reference trajectory may be output as a control trajectory for controlling the vehicle 106. Additional examples of verifying and selecting trajectories using state and / or error data identified by safety system components may be found, for example, in U.S. Patent Application No. 17 / 842,469, entitled “Vehicle Safety System,” filed on June 16, 2022, the entire contents of which are incorporated herein by reference for all purposes.

[0038] In some examples, the collision checking component can output an alternative trajectory as a control trajectory, such as a safe stop trajectory. The safe stop trajectory can be identified from one or more safe stop trajectories (e.g., one or more of a first safe stop trajectory, a second safe stop trajectory, etc.) having different longitudinal deceleration profiles. Based on determining that there is a collision probability associated with the tracker trajectory, the alternative trajectory can be output as a control trajectory. The alternative trajectory output as a control trajectory can be used to control the vehicle 106 to reduce the collision probability, and / or control the vehicle 106 to avoid a potential collision with an object.

[0039] Figure 2 2 is a schematic diagram of a tracker trajectory generation according to an example of the present disclosure. The tracker trajectory generation may include generating a tracker trajectory 202, such as Figure 2 , to indicate that generation of the tracker trajectory 202 is being performed. The tracker trajectory 202 may be implemented as the tracker trajectory 126, as described above with reference to Figure 1 As shown, and represented by the solid line, the generation of the tracker trajectory 126 has been completed. A predicted trajectory can be generated as the tracker trajectory 202 that the vehicle 106 is predicted to follow to converge to the reference trajectory 108, as described above with reference to Figure 1 described.

[0040] The optimized trajectory can be identified as a reference trajectory 108, which can be generated from a plurality of candidate trajectories. Any number of multiple candidate trajectories can be generated and analyzed to determine (e.g., select) the optimized trajectory. Characteristics used to select the optimized trajectory can include characteristics associated with current and / or predicted motion (e.g., vehicle speed, acceleration, posture, position, etc.), safety (e.g., current and / or predicted distance to objects in the environment), comfort (e.g., level and speed of motion changes), efficiency (e.g., energy consumption level), drivability (e.g., the ability of vehicle components to perform maneuvers), resource conservation (e.g., the amount of resources required to perform an operation, such as computing, network and / or memory resources), and / or various types of other characteristics.

[0041] Tracker trajectory 202 may be generated based on lateral data 118 and / or longitudinal data 120, which may be identified relative to point 110 (and / or several points along a reference trajectory), as described above with reference to FIG. Figure 1 A simulated trajectory that can be identified as a predicted trajectory associated with the vehicle can be generated as the tracker trajectory 202 based on current data (e.g., one or more of the lateral data 118, the longitudinal data 120, etc.) and a vehicle motion model (e.g., a kinematic model) (e.g., a set of equations that define how the vehicle 106 behaves based on one or more constraints).

[0042] In some examples, lateral data 118 and longitudinal data 120 (which may be determined based on point 110) may be determined. For example, lateral data 118 and longitudinal data 120 may be used to determine a steering direction associated with lateral data 118. Lateral data 118 and / or longitudinal data 120 may be determined as a closed-form solution and used to determine a predicted steering command that the vehicle may use to return to reference trajectory 108. The process may then be repeated to generate tracker trajectory 108.

[0043] A lateral error may be determined based on a difference between current lateral data (e.g., current yaw and / or heading) and projected lateral data (e.g., expected yaw and / or heading). For example, a lateral error may be determined based on a difference between the current position of the vehicle 106 (and / or an estimated position of a future state of the vehicle) and the reference trajectory 108. The lateral error may be identified as a lateral offset of the lateral data 118. A longitudinal error may be determined based on a difference between current longitudinal data (e.g., current longitudinal velocity and acceleration) and projected longitudinal data (e.g., expected velocity and acceleration along the reference trajectory). For example, a longitudinal error may be determined based on a difference between the current position of the vehicle 106 (and / or an estimated position of a future state of the vehicle) and the reference trajectory 108. The difference may include a distance between a point of the vehicle 106 (e.g., a point at the center of the vehicle 106) and a point on the reference trajectory 108 measured vertically relative to the reference trajectory 108. The longitudinal error may be identified as a longitudinal offset of the longitudinal data 120. A predicted trajectory may be determined based on the lateral offset and the longitudinal offset, such as the tracker trajectory 202.

[0044] The current lateral data and the current longitudinal data may be projected (e.g., orthogonally) onto the reference track 108 as projected lateral data (e.g., a lateral component of a point) and projected longitudinal data (e.g., a longitudinal component of a point), respectively. For example, the projected data may be identified by projecting (e.g., orthogonally) a point of the vehicle 106 (e.g., a point at the center of the vehicle 106) onto the reference track 108. Based on an orthogonal projection (e.g., an orthogonal projection similar to the orthogonal projection 208 discussed below), the current lateral data and the current longitudinal data may be orthogonally projected onto the reference track 108. In some examples, the offset data (which may include one or more offsets (e.g., lateral offsets and / or longitudinal offsets)) may be determined as an orthogonal projection of the current position of the vehicle 106 on the reference track 108.

[0045] The offset data and reference trajectory associated with the vehicle 106 at a current position (e.g., a current point at a current time (e.g., time t0)) can be used to determine one or more control commands (e.g., steering and / or speed commands) for the vehicle 106. The control commands can be input into the kinematic model to determine a next position of the vehicle 106 along the tracker trajectory 108. In some examples, the next position of the vehicle 106 along the tracker trajectory 108 can include a next point at time t1 (e.g., a point at the next time step). For example, the next point (e.g., a center point of the bounding box 206 at time t1) can be determined using the orthogonally projected points and the kinematic model. In some examples, the next point (e.g., a point at the center of the bounding box 206) can be orthogonally projected, such as Figure 2as shown, and is used together with the kinematic model to determine another next point.

[0046] The kinematic model may be used with a control model (e.g., a model utilizing a lane keeping tracker and / or a least squares model) to determine tracker trajectory 202. In some examples, the kinematic model may be used with a lane keeping tracker to determine the next lateral data based on lateral data 118 (e.g., lateral offset).

[0047] The kinematic model can be used with one or more models of various types (e.g., a least squares model) to determine the next lateral data and / or the next longitudinal data. The kinematic model can be integrated and used to determine the next point associated with the tracker trajectory 202 based on the current lateral data and the current longitudinal data.

[0048] The next point, the next lateral data, and / or the next longitudinal data may be determined by propagating the bounding box 204 at time t0 to the bounding box 206 at time t1 (or a second time) associated with the next point. At time t1, the next point may be associated with the next lateral data and the next longitudinal data. At time t1, the next lateral data and the next longitudinal data may be orthogonally projected onto the reference trajectory 108 as an orthogonal projection 208.

[0049] Although the kinematic model can be used, for example, to identify a next point based on a current point, as described above in the present disclosure, it is not limited thereto. In some examples, since the reference trajectory 108 can be determined based on a series of points (e.g., waypoints) and / or can include a series of points that may or may not be used to map the current position of the vehicle 106 to the reference trajectory 108, various models can be used to implement the kinematic model. In these examples, the kinematic model can be implemented as one or more models, including a lateral kinematic model and / or a longitudinal kinematic model.

[0050] As described above, the kinematic model can be used iteratively to determine other points associated with the tracker trajectory 202 based on other lateral data (e.g., additional lateral data) and / or other longitudinal data (e.g., additional longitudinal data). In some examples, the projections described above (e.g., orthogonal projections perpendicular to the reference trajectory 108) and predictions using the kinematic model can be repeatedly and iteratively used to determine the tracker trajectory 202 (e.g., the next point, such as a point at the center of the bounding box 206, can be orthogonally projected, such as Figure 2, and used together with the kinematic model to determine another next point). The kinematic model can be integrated any number of times to identify any amount of lateral data and / or any amount of longitudinal data until a tracker trajectory point (e.g., a point on the tracker trajectory determined based on the kinematic model) i) is projected onto a stopping point on a reference trajectory, and / or ii) is aligned with (or reached) a fixed time range, as discussed in further detail below.

[0051] Because the current position of the bounding box can be projected onto the reference track 108 using an iterative process, other positions (e.g., next positions) of the bounding box can be determined. The process can be repeated until the tracker track 202 converges with the reference track 108 with respect to the lateral data and the longitudinal data, or until a stopping point is reached, as discussed in more detail below. The iterative process can be used to simulate the processing to be performed by the tracker component 314 of the vehicle 106.

[0052] By verifying the reference trajectory 108 with the tracker trajectory, a potential collision with an object (e.g., object 210) can be avoided, which accounts for potential deviations from the reference trajectory 108. For example, the object 210 can be located in the environment in which the vehicle 106 is traveling. For example, the tracker trajectory 202 can be utilized to iteratively propagate bounding boxes starting from the bounding box 204 along the tracker trajectory 202 until the tracker trajectory bounding box 212 at time tN is determined. Because the tracker trajectory 202 accurately represents the predicted position of the vehicle 106 at various times, the potential collision with the object 210 can be avoided by controlling the vehicle 106 to utilize, for example, a safety stop trajectory instead of the reference trajectory 108. Based on utilizing the tracker trajectory bounding box 212, potential collisions that may remain undetected if the bounding box propagated along the reference trajectory 108 (e.g., the reference trajectory bounding box 214 at time tN) is used for collision checking can be avoided.

[0053] The kinematic model may be used at a constant time period over a fixed time range or at a stopping point to identify the next lateral data and the next longitudinal data. In some examples, stopping points may be associated with various portions of the environment in which the vehicle 106 is traveling. For example, a stopping point may be associated with the vehicle 106 approaching a stop sign, a stop light, a stop line, another object, etc.

[0054] The fixed time range may be a predetermined time range for stopping the iterative process using the kinematic model. The fixed time range may be associated with the time of the iteration (e.g., 0.01 seconds, 0.01 seconds, 0.1 seconds, 1 second, etc.) that has passed since the analysis was performed by the initial iteration for the bounding box 206 at the initial time t0. The fixed time range may be used to determine the number of iterations used to propagate the bounding box associated with the vehicle 106 to verify the reference trajectory 108. The control model and the kinematic model may be used in one or more iterations to verify the reference trajectory 108 through various techniques as described above until the time associated with the position of the bounding box undergoing the iteration reaches or exceeds a threshold time associated with the fixed time range.

[0055] By utilizing the tracker trajectory 202 to validate the reference trajectory 108, the level of accuracy associated with the validation of the reference trajectory 108 may be respectively greater than the level of accuracy otherwise utilized for validation of the reference trajectory 108. Because the tracker trajectory 202 is generated based on data associated with the vehicle 106 that is more accurate and / or current than the data used to generate the reference trajectory 108, by utilizing the tracker trajectory 202, the level of accuracy associated with the validation of the reference trajectory 108 may be respectively greater.

[0056] In some examples, multiple trajectories can be verified (e.g., one or more reference trajectories, one or more safety stop trajectories, etc.). For example, reference trajectory 108 can be verified, another trajectory 108 (or a first safety stop trajectory), and / or another trajectory (or a second safety stop trajectory) can be verified. A comparison can be determined between a first collision likelihood determined based on reference trajectory 108, a second collision likelihood determined based on the first safety stop trajectory, and a second collision likelihood determined based on the second safety stop trajectory. In some examples, vehicle 106 can be controlled based on the trajectory associated with the lowest collision likelihood among reference trajectory 108, the first safety stop trajectory, and the second safety stop trajectory. The trajectory associated with the lowest collision likelihood can be sent to tracker component 314, as described below with reference to Figure 3 discussed.

[0057] Figure 3 A block diagram of a vehicle system component 300 for implementing the techniques described herein is depicted. The vehicle system component 300 may include a main computing device 302 and an auxiliary computing device 304. Figure 1 As described above, the main computing device 302 can be used to implement the main computing device of the vehicle 106. The auxiliary computing device 304 can be used to implement the above reference Figure 1 An auxiliary computing device of the vehicle 106 is described.

[0058] The host computing device 302 may include a planning component 306. The planning component 306 may be used to perform the above-referenced Figure 1 Any technology of the host computing device described above. For example, the planning component 306 can generate multiple candidate trajectories and generate an optimized trajectory based on the candidate trajectories. The optimized trajectory can be sent as a reference trajectory (e.g., reference trajectory 108).

[0059] The auxiliary computing device 304 may include a predicted driving trajectory component (or "simulated driving trajectory component") 308 and a collision checking component 310. The auxiliary computing device 304 may receive the reference trajectory 108. In some examples, the auxiliary computing device 304 may receive the reference trajectory via the predicted driving trajectory component 308. The predicted driving trajectory component 308 may generate a tracker trajectory (e.g., the tracker trajectory 126) based on the reference trajectory 108.

[0060] The collision check component 310 can perform collision checking using the tracker trajectory 126. The reference trajectory 108 can be verified using the collision check on the tracker trajectory 126 (which can be generated by the kinematic model). The verified reference trajectory 108 or an alternative trajectory can be output as the command trajectory 312. For example, the verified reference trajectory 108 can be output as the command trajectory 312, or an alternative trajectory (e.g., a first safety stop trajectory, a second safety stop trajectory, etc.) can be output as the command trajectory 312.

[0061] The command trajectory 312 output by the collision checking component 310 may be input by a tracker component 314 . The tracker component 314 may analyze the command trajectory 312 and utilize the command trajectory 312 to generate controls (eg, steering and / or speed controls) 316 to control the vehicle 106 .

[0062] Tracker component 314 can output control 316 to one or more drive systems (e.g., drive systems) 318. Drive systems 318 can be used to control vehicle 106 based on controls (e.g., steering and / or speed control). For example, controlling vehicle 106 can include controlling the speed, acceleration (e.g., increasing or decreasing speed), orientation (e.g., attitude, tilt), etc. of vehicle 106 through any type of vehicle system / component.

[0063] Figure 4 4 is a block diagram of an example system 400 for implementing the techniques described herein. In at least one example, the system 400 can include a vehicle 402. In the example system 400 shown, the vehicle 402 can be an autonomous vehicle; however, the vehicle 402 can be any other type of vehicle. In some examples, the vehicle 402 can be used to implement a vehicle, as described above with respect to Figure 1 described.

[0064] In some examples, vehicle 402 may be an unmanned vehicle, such as an autonomous vehicle configured to operate in accordance with a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout a journey in which a driver (or passenger) is not expected to be in control of the vehicle at any time. In such an example, because vehicle 402 may be configured to control all functions from the start to the completion of a journey, including all parking functions, it may not include a driver and / or controls for driving vehicle 402, such as a steering wheel, accelerator pedal, and / or brake pedal. This is merely an example, and the systems and methods described herein may be incorporated into any ground, air, or water vehicle, including those ranging from vehicles that require manual control by a driver at all times to those that are partially or fully autonomously controlled.

[0065] The vehicle 402 may include one or more first computing devices 404, one or more sensor systems 406, one or more transmitters 408, one or more communication connections 410 (also referred to as communication devices and / or modems), at least one direct connection 412 (e.g., for physically coupling with the vehicle 402 to exchange data and / or provide power), and one or more drive systems 414. By way of example, the first computing device 404 may be considered a master system. In some examples, the first computing device 404 may be used to implement the master computing device 302, as described above with reference to Figure 3 The one or more sensor systems 406 can be configured to capture sensor data associated with the environment.

[0066] Sensor system 406 may include time-of-flight sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement unit (IMU), accelerometer, magnetometer, gyroscope, etc.), lidar sensors, radar sensors, sonar sensors, infrared sensors, cameras (e.g., RGB, IR, intensity, depth, etc.), microphone sensors, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), ultrasonic transducers, wheel encoders, etc. Sensor system 406 may include multiple instances of each of these or other types of sensors. For example, the time-of-flight sensors may include separate time-of-flight sensors located at the corners, front, rear, sides, and / or top of vehicle 402. As another example, camera sensors may include multiple cameras arranged at various locations around the exterior and / or interior of vehicle 402. Sensor system 406 may provide input to first computing device 404.

[0067] The vehicle 402 may also include a transmitter 408 for emitting light and / or sound. The transmitters 408 in this example include internal audio and visual transmitters for communicating with passengers of the vehicle 402. By way of example and not limitation, the internal transmitters may include speakers, lights, signs, display screens, touch screens, tactile transmitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.), and the like. The transmitters 408 in this example may also include external transmitters. By way of example and not limitation, the external transmitters in this example include lights indicating the direction of travel or other indicators of vehicle motion (e.g., indicator lights, signs, light arrays, etc.), and one or more audio transmitters (e.g., speakers, speaker arrays, horns, etc.) to communicate with pedestrians or other nearby vehicles using sound, and one or more of the audio transmitters may include acoustic beam steering technology.

[0068] The vehicle 402 may also include a communication connection 410 that enables communication between the vehicle 402 and one or more other local or remote computing devices (e.g., a remote teleoperation computing device) or remote services. For example, the communication connection 410 may facilitate communication with other local computing devices on the vehicle 402 and / or the drive system 414. In addition, the communication connection 410 may allow the vehicle 402 to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic lights, etc.).

[0069] The communication connection 410 may include a physical interface and / or a logical interface for connecting the first computing device 404 to another computing device or one or more external networks 416 (e.g., the Internet). For example, the communication connection 410 may enable, for example, Wi-Fi-based communications over frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communications, dedicated short-range communications (DSRC), or any suitable wired or wireless communication protocol that enables a respective computing device to interact with other computing devices.

[0070] In at least one example, vehicle 402 may include drive system 414. In some examples, vehicle 402 may have a single drive system 414. In at least one example, if vehicle 402 has multiple drive systems 414, the individual drive systems 414 may be positioned on opposite ends (e.g., front and rear, etc.) of vehicle 402. In at least one example, drive system 414 may include sensor system 406 to detect conditions of drive system 414 and / or the surrounding environment of vehicle 402. By way of example and not limitation, sensor system 406 may include: one or more wheel encoders (e.g., rotary encoders) to sense the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure the orientation and acceleration of the drive system, cameras or other image sensors, ultrasonic sensors to acoustically detect objects in the surrounding environment of the drive system, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be unique to drive system 414. In some cases, sensor system 406 on drive system 414 may overlap or supplement a corresponding system (eg, sensor system 406 ) of vehicle 402 .

[0071] The drive system 414 may include many of the vehicle systems, including: a high-voltage battery; a motor for propelling the vehicle; an inverter for converting direct current from the battery into alternating current for use by other vehicle systems; a steering system including a steering motor and a steering rack (which may be electric); a braking system including a hydraulic actuator or an electric actuator; a suspension system including a hydraulic component and / or a pneumatic component; a stability control system for distributing braking force to mitigate loss of traction and maintain control; an HVAC system; lighting (e.g., lighting such as head / tail lights for illuminating the external surroundings of the vehicle); and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components, such as DC / DC converters, high-voltage connectors, high-voltage cables, charging systems, charging ports, etc.). In addition, the drive system 414 may include a drive system controller that may receive and pre-process data from the sensor system 406 and control the operation of various vehicle systems. In some examples, the drive system controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more components to perform various functions of the drive system 414. In addition, the drive system 414 may also include one or more communication connections that enable the corresponding drive system to communicate with one or more other local or remote computing devices. In some examples, the drive system 414 can be used to implement the drive system 318, as described above with reference to Figure 3 described.

[0072] Vehicle 402 may include one or more second computing devices 418 to provide redundancy, error checking, and / or verification of decisions and / or commands determined by first computing device 404. For example, while first computing device 404 may be considered a primary system, second computing device 418 may be considered an auxiliary system. In some examples, second computing device 418 may be used to implement auxiliary computing device 304, as described above with reference to Figure 3 described.

[0073] The host system may generally perform processing to control how the vehicle maneuvers within the environment. The host system may implement various artificial intelligence (AI) techniques, such as machine learning, to understand the environment around the vehicle and / or direct the vehicle to move within the environment. For example, the host system may implement AI techniques to locate the vehicle, detect objects around the vehicle, segment sensor data, determine the classification of objects, predict object trajectories, generate vehicle trajectories, etc. In an example, the host system processes data from multiple types of sensors on the vehicle, such as light detection and ranging (lidar) sensors, radar sensors, image sensors, depth sensors (time of flight, structured light, etc.), etc.

[0074] The auxiliary system can verify the operation of the primary system and can take over control of the vehicle from the primary system when there is a problem with the primary system. In an example, the auxiliary system processes data from several sensors, such as a subset of the sensor data processed by the primary system. For illustration, the primary system may process lidar data, radar data, image data, depth data, etc., while the auxiliary system may only process lidar data and / or radar data (and / or time of flight data). However, in other examples, the auxiliary system may process sensor data from any number of sensors, such as data from each of the sensors, data from the same number of sensors as the primary system, etc.

[0075] Additional examples of vehicle architectures including primary and auxiliary computing systems can be found, for example, in U.S. Patent Application No. 16 / 189,726, filed on November 13, 2018, entitled “Perception Collision Avoidance,” which is incorporated herein by reference in its entirety for all purposes.

[0076] The first computing device 404 may include one or more processors 420 and a memory 422 communicatively coupled to the one or more processors 420. In the example shown, the memory 422 of the first computing device 404 stores a positioning component 424, a perception component 426, a prediction component 428, a planning component 430, a map component 432, and one or more system controllers 434. Although described as being present in the memory 422 for purposes of illustration, it is contemplated that the positioning component 424, the perception component 426, the prediction component 428, the planning component 430, the map component 432, and the one or more system controllers 434 may additionally or alternatively be accessible to the computing device 404 (e.g., stored in a different component of the vehicle 402 and / or accessible to the vehicle 402 (e.g., stored remotely)).

[0077] In the memory 422 of the first computing device 404, the positioning component 424 may include functionality to receive data from the sensor system 406 to determine the location of the vehicle 402. For example, the positioning component 424 may include and / or request / receive a three-dimensional map of the environment, and may continuously determine the location of the autonomous vehicle within the map. In some cases, the positioning component 424 may use SLAM (simultaneous localization and mapping) or CLAMS (simultaneous calibration, localization and mapping) to receive time-of-flight data, image data, lidar data, radar data, sonar data, IMU data, GPS data, wheel encoder data, or any combination thereof, etc. to accurately determine the location of the autonomous vehicle. In some cases, the positioning component 424 may provide data to various components of the vehicle 402 to determine the initial location of the autonomous vehicle for use in generating a trajectory, as discussed herein.

[0078] The perception component 426 may include functionality to perform object detection, segmentation, and / or classification. In some examples, the perception component 426 may provide processed sensor data indicating the presence of an entity approaching the vehicle 402 and / or an entity classified as an entity type (e.g., a vehicle, a pedestrian, a cyclist, a building, a tree, a pavement, a curb, a sidewalk, an unknown, etc.). In additional or alternative examples, the perception component 426 may provide processed sensor data indicating one or more characteristics associated with the detected entity and / or the environment in which the entity is located. In some examples, the characteristics associated with the entity may include, but are not limited to, x-position (global position), y-position (global position), z-position (global position), orientation, entity type (e.g., classification), speed of the entity, range (size) of the entity, etc. The characteristics associated with the environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, the time of day, the day of the week, the season, the weather condition, an indication of darkness / light, etc.

[0079] As described above, the perception component 426 can use a perception algorithm to determine a perception-based bounding box associated with an object in the environment based on sensor data and / or a predicted trajectory associated with the object in the environment. For example, the perception component 426 can receive image data and classify the image data to determine that the object is represented in the image data. Then, using the detection algorithm, the perception component 426 can generate a two-dimensional bounding box associated with the object and / or a perception-based three-dimensional bounding box. The perception component 426 can also generate a three-dimensional bounding box associated with the object. As described above, the three-dimensional bounding box can provide additional information associated with the object, such as position, orientation, posture, and / or size (e.g., length, width, height, etc.).

[0080] The perception component 426 may include functionality to store perception data generated by the perception component 426. In some cases, the perception component 426 may determine a trail corresponding to an object that has been classified as an object type. For purposes of illustration only, the perception component 426 may capture one or more images of an environment using the sensor system 406. The sensor system 406 may capture images of an environment including an object such as a pedestrian. The pedestrian may be at a first position at time T, and at a second position at time T+t (e.g., movement during a time span t after time T). In other words, the pedestrian may move from a first position to a second position during this time span. Such movement may, for example, be recorded as stored perception data associated with the object.

[0081] In some examples, the stored perception data may include fused perception data captured by the vehicle 402. The fused perception data may include a fusion or other combination of sensor data (e.g., image sensors, lidar sensors, radar sensors, time of flight sensors, sonar sensors, global positioning system sensors, internal sensors, and / or any combination of these sensors) from the sensor system 406. The stored perception data may additionally or alternatively include classification data, which includes semantic classifications of objects (e.g., pedestrians, vehicles, buildings, road surfaces, etc.) represented in the sensor data. The stored perception data may additionally or alternatively include trace data (position, orientation, sensor features, etc.) corresponding to the movement of objects classified as dynamic objects in the environment. The trace data may include multiple traces of multiple different objects over time. The trajectory data may be mined to identify images of certain types of objects (e.g., pedestrians, animals, etc.) when the objects are stationary (e.g., standing still) or moving (e.g., walking, running, etc.). In this example, the computing device determines the trace corresponding to the pedestrian.

[0082] Prediction component 428 can generate one or more probability maps that represent the predicted probabilities of the possible locations of one or more objects in the environment. For example, prediction component 428 can generate one or more probability maps for vehicles, pedestrians, animals, etc. within a threshold distance from vehicle 402. In some cases, prediction component 428 can measure the traces of objects based on observed behavior and predicted behavior and generate discrete predicted probability maps, heat maps, probability distributions, discrete probability distributions, and / or trajectories for objects. In some cases, one or more probability maps can represent the intentions of one or more objects in the environment.

[0083] Planning component 430 can determine a path for vehicle 402 to follow through an environment. For example, planning component 430 can determine various routes and paths and various levels of detail. In some cases, planning component 430 can determine a route for traveling from a first location (e.g., a current location) to a second location (e.g., a target location). For the purposes of this discussion, a route can be a series of waypoints for traveling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. In addition, planning component 430 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In some examples, planning component 430 can be used to implement planning component 306, as described above with reference to Figure 3 described.

[0084] In at least one example, planning component 430 can determine how to guide the autonomous vehicle from a first waypoint in a series of waypoints to a second waypoint in a series of waypoints. In some examples, the instructions can be a path or a portion of a path. In some examples, multiple paths can be generated substantially simultaneously (i.e., within a technical tolerance) according to a receding horizon technique. A single path with the highest confidence level among the multiple paths in the receding data horizon can be selected to operate the vehicle.

[0085] In other examples, planning component 430 can alternatively or additionally use data from perception component 426 and / or prediction component 428 to determine a path for vehicle 402 to follow through the environment. For example, planning component 430 can receive data about objects associated with the environment from perception component 426 and / or prediction component 428. Using this data, planning component 430 can determine a route that avoids objects in the environment to travel from a first location (e.g., a current location) to a second location (e.g., a target location). In at least some examples, such planning component 430 can determine that there is no such collision-free path and, in turn, provide a path that avoids all collisions and / or otherwise mitigates damage to bring vehicle 402 to a safe stop.

[0086] The memory 422 may also include one or more maps 432, which can be used by the vehicle 402 to navigate within the environment. For the purposes of this discussion, a map can be any number of data structures modeled in two, three, or N dimensions, and the map can provide information about the environment, such as, but not limited to, topology (e.g., intersections), streets, mountains, roads, terrain, and the overall environment. In some cases, the map may include, but is not limited to: texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.); spatial information (e.g., image data projected onto a grid, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflection information (e.g., specular reflection information, retroreflection information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional grid of the environment. In some instances, the map may be in a tiled format (tile). 432) such that individual tiles of the map represent discrete portions of the environment and can be loaded into working memory as needed, as discussed herein. In at least one example, one or more maps 432 can include at least one map (e.g., an image and / or a grid). In some examples, vehicle 402 can be controlled based at least in part on map 432. That is, map 432 can be used in conjunction with positioning component 424, perception component 426, prediction component 428, and / or planning component 430 to determine the location of vehicle 402, identify objects in the environment, generate predicted probabilities associated with objects and / or vehicle 402, and / or generate routes and / or trajectories to navigate within the environment.

[0087] In some examples, one or more maps 432 may be stored on a remote computing device (e.g., computing device 452) that may be accessible via network 416. In some examples, multiple maps 432 may be stored based on, for example, characteristics (e.g., type of entity, time of day, day of week, season of year, etc.). Storing multiple maps 432 may have similar memory requirements, but may increase the speed at which data in the maps is accessed.

[0088] In at least one example, the first computing device 404 can include one or more system controllers 434 that can be configured to control steering, propulsion, braking, safety, transmitter, communication, and other systems of the vehicle 402. These system controllers 434 can communicate with and / or control corresponding systems of the drive system 414 and / or other components of the vehicle 402, which can be configured to operate according to the path provided from the planning component 430.

[0089] The second computing device 418 may include one or more processors 436 and memory 438, including components for verifying and / or controlling various aspects of the vehicle 402, as discussed herein. In at least one example, the one or more processors 436 may be similar to the processor 420 and the memory 438 may be similar to the memory 422. However, in some examples, the processor 436 and the memory 438 may include different hardware than the processor 420 and the memory 422 for additional redundancy. In some examples, the memory 438 may include a positioning component 440, a perception / prediction component 442, one or more system controllers 444, and a planning component 446.

[0090] In some examples, the positioning component 440 may receive sensor data from the sensor 406 to determine one or more of the position and / or orientation (collectively referred to as attitude) of the autonomous vehicle 402. Here, the position and / or orientation may be relative to a point and / or object in the environment in which the autonomous vehicle 402 is located. In an example, the orientation may include an indication of the yaw, roll, and / or pitch of the autonomous vehicle 402 relative to a reference plane and / or relative to a point and / or object. In an example, the positioning component 440 may perform less processing (e.g., a higher level of positioning) than the positioning component 424 of the first computing device 404. For example, the positioning component 440 may not determine the attitude of the autonomous vehicle 402 relative to the map, but only determine the attitude of the autonomous vehicle 402 relative to objects and / or surfaces (e.g., local positions rather than global positions) detected around the autonomous vehicle 402. Such a position and / or orientation may be determined, for example, using some or all of the sensor data using a probabilistic filtering technique (e.g., a Bayesian filter (Kalman filter, extended Kalman filter, unscented Kalman filter, etc.)). In some examples, the positioning component 440 can be omitted, and the positioning information can be determined by the first computing device 404 and provided to the second computing device 418 .

[0091] In some examples, perception / prediction component 442 can include functionality for detecting, identifying, classifying, and / or tracking objects represented in sensor data. For example, perception / prediction component 442 can perform clustering operations and operations for estimating or determining heights associated with objects, as discussed herein.

[0092] In some examples, the perception / prediction component 442 may include an M-estimator, but may lack an object classifier, such as a neural network, a decision tree, etc., for classifying objects. In additional or alternative examples, the perception / prediction component 442 may include any type of ML model configured to disambiguate the classification of an object. In contrast, the perception component 426 may include a pipeline of hardware and / or software components that may include one or more machine learning models, Bayesian filters (e.g., Kalman filters), graphics processing units (GPUs), and / or the like. In some examples, the perception data determined by the perception / prediction component 442 (and / or 426) may include object detection (e.g., identification of sensor data associated with objects in the environment surrounding the autonomous vehicle), object classification (e.g., identification of an object type associated with a detected object), object trajectory (e.g., historical, current, and / or predicted object position, velocity, acceleration, and / or orientation), and / or the like.

[0093] The perception / prediction component 442 can also process the input data to determine one or more predicted trajectories for the object. For example, based on the current position of the object and the speed of the object over a period of several seconds, the perception / prediction component 442 can predict the path that the object will move in the next few seconds. In some examples, such a predicted path may include using a linear assumption of motion in the case of a given position, orientation, speed, and / or orientation. In other examples, such a predicted path may include more complex analysis. Additional examples of detecting objects and determining predicted object trajectories can be found in, for example, U.S. patent application No. 17 / 514,542, entitled “Collision Avoidance and Mitigation in Autonomous Vehicles” filed on October 29, 2021, the entire contents of which are incorporated herein by reference for all purposes. Additional examples of detecting object classification for trajectory verification can be found, for example, in U.S. patent application No. 16 / 588,717, entitled “Complex Ground Profile Estimation” filed on September 30, 2019, the entire contents of which are incorporated herein by reference. Additional examples of determining perceptual information for trajectory selection can be found, for example, in U.S. patent application Ser. No. 17 / 116,888, filed on December 9, 2020, entitled “Perception System Velocity Determination,” the entire contents of which are incorporated herein by reference for all purposes.

[0094] In some examples, system controller 444 may include functionality to control safety-critical components of the vehicle (e.g., steering, brakes, motors, etc.). In this manner, second computing device 418 may provide redundancy and / or additional hardware and software layers for vehicle safety. In some examples, system controller 444 may be used to implement tracker component 314, as described above with reference to Figure 3 described.

[0095] In some examples, planning component 446 may include predicted driving trajectory component 448 and collision check component 450. In some examples, predicted driving trajectory component 448 may be used to implement predicted driving trajectory component 308. In these or other examples, collision check component 450 may be used to implement collision check component 310. For example, predicted driving trajectory component 448 may receive a reference trajectory (e.g., reference trajectory 108) from planning component 430, as described in detail above. Predicted driving trajectory component 448 may utilize the reference trajectory to generate a tracker trajectory (e.g., tracker trajectory 202). Collision check component 450 may utilize the tracker trajectory to perform collision checking to verify the reference trajectory. Collision check component 450 may output command trajectory 312 to drive system 414 based on verification of the reference trajectory.

[0096] The vehicle 402 can be connected to a computing device 452 via a network 416 and can include one or more processors 454 and a memory 456 communicatively coupled to the one or more processors 454. In at least one example, the one or more processors 454 can be similar to the processor 420 and the memory 456 can be similar to the memory 422. In the example shown, the memory 456 of the computing device 452 stores components 458 that can correspond to any of the components discussed herein.

[0097] Processors 420, 436, and / or 454 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of example and not limitation, processors 420, 436, and / or 454 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data to convert the electronic data into other electronic data that may be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered to be processors configured to implement encoded instructions.

[0098] Memory 422, 438, and / or 456 is an example of a non-transitory computer-readable medium. Memory 422, 438, and / or 456 can store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions belonging to various systems. In various embodiments, memory 422, 438, and / or 456 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architecture, system, and separate elements described herein may include many other logical components, programmed components, and physical components, of which those components shown in the drawings are merely examples relevant to the discussion herein.

[0099] In some cases, aspects of some or all of the components discussed herein may include any model, algorithm, and / or machine learning algorithm. For example, in some cases, the components in memory 422, 438, and / or 456 may be implemented as a neural network. In some examples, the components in memory 422, 438, and / or 456 may not include a machine learning algorithm to reduce complexity and to verify and / or authenticate from a security perspective.

[0100] As described herein, an exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to produce an output. Each layer in the neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As can be understood in the context of the present disclosure, a neural network may utilize machine learning, which may refer to a broad classification of algorithms in which an output is generated based on learned parameters.

[0101] Although discussed in the context of neural networks, any type of machine learning consistent with the present disclosure may be used. For example, machine learning or machine learning algorithms may include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimated scatter plot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomy 3 (ID3), chi-squared automatic interaction detection (CHAID), decision stumps, conditional decision trees), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, averaged single dependency estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptron, backpropagation, jump field network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), hybrid discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting methods, bootstrap aggregation (Bagging), AdaBoost, stacked generalization (hybrid), gradient boosting machine (GBM), gradient boosted regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.

[0102] Additional examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, etc.

[0103] Figure 5 An example process for reference trajectory validation and collision check management according to examples of the present disclosure is depicted.

[0104] At operation 502 , the example process 500 may include receiving a reference trajectory 108 . The reference trajectory 108 for the vehicle 106 to follow may be received from a first computing component of the vehicle 106 and received by a second computing component of the vehicle 106 .

[0105] At operation 504, the example process 500 may include determining an offset of the vehicle. Offset data including at least one offset may be determined based on the reference trajectory 108. The offset data may include lateral data 118 and / or longitudinal data 120. The lateral data 118 may include a lateral offset. The longitudinal data 120 may include a longitudinal offset. The lateral offset and / or the longitudinal offset may be determined as at least one offset based on the reference trajectory 108.

[0106] At operation 506 , the example process 500 may include determining a predicted tracker trajectory that the vehicle will follow. The predicted tracker trajectory may be determined as a tracker trajectory (eg, the tracker trajectory 126 ).

[0107] At operation 508 , the example process 500 may include determining predicted collision data. The predicted collision check may be performed by utilizing the tracker trajectory 126 to determine a collision likelihood associated with objects in the environment.

[0108] At operation 510, the example process 500 may include validating the reference trajectory 108. The reference trajectory 108 may be validated as a valid reference trajectory based on determining that the collision likelihood associated with the tracker trajectory 126 is less than or equal to a threshold likelihood (including the absence of any collision).

[0109] At operation 512, the example process 500 may include determining whether the reference trajectory has been verified as a valid reference trajectory. Based on determining that the reference trajectory 108 has not been verified as a valid reference trajectory, the example process 500 may continue to operation 514. Based on determining that the reference trajectory 108 has been verified as a valid reference trajectory, the example process 500 may continue to operation 516.

[0110] At operation 514 , the example process 500 may include controlling the vehicle 106 to follow the alternative trajectory. Based on determining that there is a collision potential associated with the tracker trajectory 126 , the vehicle 106 may be controlled using the alternative trajectory by outputting the alternative trajectory as the control trajectory.

[0111] At operation 516 , the example process 500 may include controlling the vehicle 106 to follow the reference trajectory. Based on determining that there is no collision potential associated with the tracker trajectory 126 , the vehicle 106 may be controlled using the effective reference trajectory by outputting the reference trajectory 108 as the control trajectory.

[0112] Sample Clauses

[0113] A: A system, comprising: one or more processors; and one or more computer-readable media storing instructions executable by the one or more processors, wherein when the instructions are executed, the system performs the following operations, including: receiving a reference trajectory to be followed by an autonomous vehicle from a primary computing component; determining, by an auxiliary computing component, a lateral offset and a longitudinal offset of the autonomous vehicle relative to the reference trajectory; determining, based at least in part on the lateral offset, the longitudinal offset, and a kinematic model, a tracker trajectory that the autonomous vehicle is predicted to drive to converge to the reference trajectory; detecting an object approaching the autonomous vehicle; determining a likelihood of a collision between the autonomous vehicle and the object based at least in part on the tracker trajectory and the object; and controlling the autonomous vehicle based at least in part on the likelihood of a collision.

[0114] B: The system of clause A, wherein determining the tracker trajectory comprises using a least squares model, and wherein the operations further comprise determining a longitudinal control for the tracker trajectory based at least in part on the least squares model.

[0115] C: The system of clause A or B, the operations further comprising iteratively determining the tracker trajectory based at least in part on at least one of a fixed time range or reaching a stopping point associated with a reference trajectory.

[0116] D: A system according to any of clauses AC, wherein controlling the autonomous vehicle further comprises: controlling the autonomous vehicle to follow a reference trajectory or a safe stop trajectory.

[0117] E: A system as described in any of clauses AD, wherein determining the tracker trajectory includes: determining an orientation offset associated with the autonomous vehicle relative to a reference trajectory; determining, based at least in part on the orientation offset, orientation data that the autonomous vehicle is predicted to utilize to follow the reference trajectory; determining an acceleration offset associated with the autonomous vehicle relative to the reference trajectory; and determining, based at least in part on the acceleration offset, acceleration data that the autonomous vehicle is predicted to utilize to navigate to the reference trajectory.

[0118] F: One or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein when the instructions are executed, the one or more processors perform operations including: receiving a first trajectory to be followed by a vehicle; determining an offset of the vehicle based at least in part on the first trajectory; determining a second trajectory that the vehicle is predicted to drive to return to the first trajectory based at least in part on the offset; determining a likelihood of collision between the vehicle and an object approaching the vehicle based at least in part on the second trajectory; and controlling the vehicle based at least in part on the likelihood of collision.

[0119] G: One or more non-transitory computer-readable media as described in clause F, wherein the offset includes a lateral offset and a longitudinal offset, and wherein determining the second trajectory further includes: determining the second trajectory based at least in part on the lateral offset, the longitudinal offset, and a kinematic model.

[0120] H: One or more non-transitory computer-readable media according to clause F or G, wherein determining the likelihood of a collision comprises: determining an estimated position of the vehicle along the second trajectory.

[0121] I: One or more non-transitory computer-readable media as described in any of clauses FH, wherein determining the second trajectory includes using a least squares model, and wherein the operations further include determining longitudinal control for the second trajectory based at least in part on the least squares model.

[0122] J: According to one or more non-transitory computer-readable media of any of clauses FI, the operations further comprising: iteratively determining the second trajectory based at least in part on at least one of a fixed time range or reaching a stopping point associated with the second trajectory.

[0123] K: One or more non-transitory computer-readable media as described in any of clauses FJ, wherein the offset is determined based on a current position of the vehicle, and the current position is received from a host computing device.

[0124] L: One or more non-transitory computer-readable media according to any of clauses FK, wherein the offset is determined as an orthogonal projection of the vehicle's current position on the first trajectory.

[0125] M: One or more non-transitory computer-readable media according to any of clauses FL, wherein determining the second trajectory further comprises: performing integration of a kinematic model based at least in part on the offset.

[0126] N: One or more non-transitory computer-readable media as described in any of clauses FM, wherein: the offset includes a lateral offset; and determining the offset includes utilizing a lane keeping algorithm to determine lateral error data associated with at least one of the lateral data associated with the vehicle or the yaw of the vehicle.

[0127] O: One or more non-transitory computer-readable media as described in any of clauses FN, wherein controlling the vehicle includes: controlling the vehicle to follow a first trajectory based on the collision probability being less than or equal to a threshold probability, or controlling the vehicle to execute a safe stop trajectory based on the collision probability being greater than the threshold probability.

[0128] P: One or more non-transitory computer-readable media according to any of clauses FO, wherein the offset comprises a longitudinal offset, and the operation further comprises: determining the longitudinal offset based at least in part on a difference between a measured speed of the vehicle and a reference speed associated with the first trajectory.

[0129] Q: A method comprising: receiving a first trajectory to be followed by a vehicle; determining an offset of the vehicle relative to the first trajectory; determining a second trajectory that the vehicle is predicted to be driven to return to the first trajectory based at least in part on the offset; determining a likelihood of collision between the vehicle and an object approaching the vehicle based at least in part on the second trajectory; and controlling the vehicle based at least in part on the likelihood of collision.

[0130] R: A method according to clause Q, wherein the offset includes a lateral offset and a longitudinal offset, and wherein determining the second trajectory further comprises: determining the second trajectory based at least in part on the lateral offset, the longitudinal offset, and a kinematic model.

[0131] S: A method according to clause Q or R, wherein determining the likelihood of collision includes: determining an estimated position of the vehicle along the second trajectory.

[0132] T: A method according to any of clauses QS, wherein determining the second trajectory includes using a least squares model, and wherein the operations further include determining longitudinal control for the second trajectory based at least in part on the least squares model.

[0133] Although the example clauses described above are described with respect to a specific implementation, it should be understood that in the context of this article, the content of the example clauses can also be implemented via a method, an apparatus, a system, a computer-readable medium, and / or another implementation. In addition, any one of the example ATs can be implemented alone or in combination with any other one or more of the example ATs.

[0134] in conclusion

[0135]

[00136] While one or more examples of the techniques described herein have been described, various modifications, additions, permutations, and equivalents thereof are included within the scope of the techniques described herein.

[0136] In the description of the examples, reference is made to the accompanying drawings forming a part thereof, which illustrate specific examples of the claimed subject matter by way of illustration. It should be understood that other examples may be used, and changes or substitutions such as structural changes may be made. Such examples, changes or substitutions do not necessarily deviate from the scope of the subject matter claimed with respect to the intent. Although the steps herein may be presented in a certain order, in some cases, the ordering may be changed so that certain inputs are provided at different times or in different orders without changing the functionality of the described systems and methods. The disclosed processes may also be performed in different orders. In addition, it is not necessary to perform the various calculations herein in the order disclosed, and other examples using alternative orders of calculations may be easily implemented. In addition to being reordered, calculations may also be decomposed into sub-calculations with the same results.

Claims

1. A system comprising: one or more processors; as well as One or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein when the instructions are executed, the one or more processors are caused to perform the following operations, including: receiving a first trajectory to be followed by a vehicle; determining an offset of the vehicle based at least in part on the first trajectory; determining, based at least in part on the offset, a second trajectory that the vehicle is predicted to drive to return to the first trajectory; determining a likelihood of collision between the vehicle and an object proximate to the vehicle based at least in part on the second trajectory; and The vehicle is controlled based at least in part on the likelihood of a collision.

2. The system according to claim 1, wherein: The first trajectory comprises a reference trajectory, the vehicle comprises an autonomous vehicle, the offset comprises a lateral offset, and the second trajectory comprises a tracker trajectory, Wherein, receiving the first trajectory further comprises: receiving the reference trajectory from a main computing component, Wherein, determining the offset further comprises: determining the lateral offset and the longitudinal offset of the autonomous vehicle relative to the reference trajectory, wherein determining the second trajectory further comprises: determining the tracker trajectory that the autonomous vehicle is predicted to drive to converge to the reference trajectory based at least in part on the lateral offset, the longitudinal offset, and a kinematic model, Wherein, determining the collision likelihood further comprises: detecting an object proximate to the autonomous vehicle and determining the collision likelihood between the autonomous vehicle and the object based at least in part on the tracker trajectory and the object, and Wherein, controlling the vehicle further comprises: controlling the autonomous vehicle based at least in part on the collision likelihood.

3. The system according to claim 1 or 2, wherein: The first trajectory comprises a reference trajectory, the vehicle comprises an autonomous vehicle, the second trajectory comprises a tracker trajectory, and determining the tracker trajectory comprises: determining a heading offset associated with the autonomous vehicle relative to the reference trajectory; determining, based at least in part on the orientation offset, orientation data that the autonomous vehicle is predicted to utilize to follow the reference trajectory; determining an acceleration offset associated with the autonomous vehicle relative to the reference trajectory; and Acceleration data that the autonomous vehicle is predicted to utilize to navigate to the reference trajectory is determined based at least in part on the acceleration offset.

4. The system according to any one of claims 1 to 3, wherein: The first trajectory comprises a reference trajectory, and the second trajectory comprises a tracker trajectory, the operations further comprising: The tracker trajectory is iteratively determined based at least in part on at least one of a fixed time range or reaching a stopping point associated with the reference trajectory.

5. The system according to any one of claims 1 to 4, wherein: The offset is determined based on the current position of the vehicle, and The current location is received from a host computing device.

6. The system according to any one of claims 1 to 5, wherein: The offset is determined as an orthogonal projection of the vehicle's current position on the first trajectory.

7. The system according to any one of claims 1 to 6, wherein: Determining the second trajectory further includes performing integration of a kinematic model based at least in part on the offset.

8. The system according to any one of claims 1 to 7, wherein: The offset comprises a lateral offset; and Determining the offset includes utilizing a lane keeping algorithm to determine lateral error data associated with at least one of lateral data associated with the vehicle or a yaw of the vehicle.

9. The system according to any one of claims 1 to 8, wherein: The first trajectory comprises a reference trajectory, and controlling the vehicle comprises: controlling the vehicle to follow the reference trajectory based on the collision probability being less than or equal to a threshold probability, or The vehicle is controlled to execute a safety stop trajectory based on the collision likelihood being greater than the threshold likelihood.

10. The system according to any one of claims 1 to 9, wherein: The offset includes a longitudinal offset, and the operation further includes: The longitudinal offset is determined based at least in part on a difference between a measured speed of the vehicle and a reference speed associated with the first trajectory.

11. A method comprising: receiving a first trajectory to be followed by a vehicle; determining an offset of the vehicle relative to the first trajectory; determining, based at least in part on the offset, a second trajectory that the vehicle is predicted to drive to return to the first trajectory; determining a likelihood of collision between the vehicle and an object proximate to the vehicle based at least in part on the second trajectory; as well as The vehicle is controlled based at least in part on the likelihood of a collision.

12. The method according to claim 11, wherein: The offset includes a lateral offset and a longitudinal offset, and Wherein, determining the second trajectory further comprises: determining the second trajectory based at least in part on the lateral offset, the longitudinal offset and a kinematic model.

13. The method according to claim 11 or 12, wherein: Determining the likelihood of collision includes determining an estimated position of the vehicle along the second trajectory.

14. The method according to any one of claims 11 to 13, wherein: Determining the second trajectory includes using a least squares model, and The method further includes determining longitudinal control for the second trajectory based at least in part on the least squares model.

15. A computer program product comprising coded instructions which, when executed on a computer, implement the method according to any one of claims 11 to 14.

Images