Autonomous vehicle blind spot management

Abstract

Various examples relate to systems and methods for controlling autonomous vehicles including towing vehicles and trailers. For example, the system may determine that a line from a location of a first sensor on the autonomous vehicle to a location of a first actor in an environment of the autonomous vehicle intersects the trailer. The system may determine that the first actor is in a blind spot of the autonomous vehicle, generate a motion plan for the autonomous vehicle, and control the autonomous vehicle according to the motion plan.

Description

Claim for priority

[0001] This application claims the benefit of priority to U.S. application serial number 18 / 046,839, filed on October 14, 2022, the entire contents of which are incorporated herein by reference. Background Art

[0002] The automotive industry is currently developing autonomous features for controlling vehicles in certain situations. According to the Society of Automotive Engineers (SAE) International Standard J3016, there are 6 levels of autonomy ranging from Level 0 (no autonomy) to Level 5 (a vehicle capable of operating without operator input under all conditions). Vehicles with autonomous features utilize sensors to sense the environment through which the vehicle is navigating. Acquiring and processing data from the sensors allows the vehicle to navigate through its environment. Summary of the invention

[0003] An autonomous vehicle may sometimes have one or more blind spots. A blind spot may be a portion of the autonomous vehicle's environment that is outside the field of view of at least one sensor of the autonomous vehicle. For example, a blind spot may present a challenge to an autonomous vehicle when an actor may enter and / or exit the blind spot. When an actor is present in a blind spot, the sensor data may not provide sufficient information about the actor.

[0004] Various examples described herein relate to systems and methods for controlling an autonomous vehicle while taking into account at least one vehicle blind spot. For example, an autonomous vehicle (e.g., an autonomous system thereof) may access sensor location data describing the location of at least one sensor on the autonomous vehicle and actor location data describing the location of an actor in an environment of the autonomous vehicle. Using the location of the at least one sensor and the location of the actor, the autonomous vehicle may determine whether the actor is present in a blind spot of the autonomous vehicle.

[0005] In some embodiments, a method is provided that includes accessing sensor position data describing a position of a first sensor on an autonomous vehicle. The method may also include accessing actor position data describing a position of a first actor in an environment of the autonomous vehicle, and determining that a line from the position of the first sensor to the position of the first actor intersects a trailer. The method may also include determining that the first actor is in a blind spot of the autonomous vehicle based at least in part on determining that the line from the position of the first sensor to the position of the first actor intersects the trailer, and generating a motion plan for the autonomous vehicle based at least in part on determining that the first actor is in the blind spot of the autonomous vehicle. The method may also include controlling the autonomous vehicle according to the motion plan.

[0006] These and other implementations of the disclosure may include one or more of the following features.

[0007] In some embodiments, the sensor position data further describes a position of a second sensor on the autonomous vehicle. The method may also include determining that a second line from the position of the second sensor to the position of the first actor intersects the trailer. Determining that the first actor is in the blind spot of the autonomous vehicle may also be based at least in part on determining that the second line from the position of the second sensor to the position of the first actor intersects the trailer.

[0008] In some embodiments, the sensor position data may describe positions of a plurality of sensors, wherein the plurality of sensors include a first sensor. The method may also include determining that a line from a corresponding position of each of the plurality of sensors intersects the trailer. Determining that the first actor is in a blind spot of the autonomous vehicle may also be based at least in part on determining that a line from a corresponding position of each of the plurality of sensors intersects the trailer.

[0009] In some embodiments, the sensor position data may further describe a position of a second actor in the environment of the autonomous vehicle, and the sensor position data may further describe a position of a second sensor on the autonomous vehicle. The method may further include determining that a line from the position of the first sensor to the position of the second actor intersects the trailer, and determining that a line from the position of the second sensor to the position of the second actor does not intersect the trailer, and the method may further include determining that the second actor is outside a blind spot of the autonomous vehicle.

[0010] In some embodiments, the method may further include determining a pose of the trailer. The pose of the trailer may describe the position of the trailer. The method may further include using the pose of the trailer to determine that a line from the position of the first sensor to the position of the first actor intersects the trailer.

[0011] In some implementations, the method may further include determining a location of the first actor based at least in part on last known state data describing a last known state of the first actor.

[0012] In some embodiments, the last known state data may include at least one of a last known location of the first actor, a last known speed of the first actor, a last known acceleration of the first actor, a last known heading of the first actor, or a last known orientation of the first actor.

[0013] In some implementations, the method may further include setting a blind spot flag to indicate that at least one actor is in a blind spot of the autonomous vehicle. Generating the motion plan may be based at least in part on the blind spot flag.

[0014] In some implementations, the method may further include generating an estimated trajectory of the first actor in the blind spot of the autonomous vehicle using last known state data describing a last known state of the first actor. Generating the motion plan may be based at least in part on the estimated trajectory of the first actor.

[0015] In some implementations, generating a motion plan may include modifying a cost associated with a first candidate motion plan in the plurality of candidate motion plans to generate a modified cost, and determining a cost associated with the first candidate motion plan using the modified cost. Generating a motion plan may further include selecting a motion plan from the plurality of candidate motion plans, wherein the selection may be based at least in part on the cost associated with the first candidate motion plan.

[0016] In some implementations, generating the motion plan may include modifying the first candidate motion plan to generate a modified first motion plan. The modification may be based at least in part on determining that the first actor is in a blind spot of the autonomous vehicle. Generating the motion plan may further include selecting a motion plan from a plurality of candidate motion plans, wherein the plurality of candidate motion plans may include the modified first motion plan.

[0017] In some implementations, modifying the first candidate motion plan may include at least one of: modifying an acceleration associated with the first candidate motion plan; or modifying a lateral velocity associated with the first candidate motion plan.

[0018] In some implementations, the method may also include accessing blind spot actor data describing at least one actor in a blind spot of the autonomous vehicle, and determining that sensor data generated by at least one sensor on the autonomous vehicle indicates that the actor has exited the blind spot of the autonomous vehicle. The method may also include decrementing the number of actors in the blind spot.

[0019] In some implementations, the method can further include accessing blind spot actor data describing at least one actor in a blind spot of the autonomous vehicle, the at least one actor including a first actor, and determining that more than a threshold time period has elapsed since determining that the first actor is in the blind spot of the autonomous vehicle. The method can also include decrementing the number of actors in the blind spot.

[0020] In some implementations, the method may further include determining that there are no remaining actors in the blind spot of the autonomous vehicle, and generating a second motion plan for the autonomous vehicle based at least in part on determining that there are no remaining actors in the blind spot of the autonomous vehicle. The method may further include controlling the autonomous vehicle according to the second motion plan.

[0021] In some embodiments, an autonomous vehicle is provided, comprising: a tractor; a trailer; and at least one processor programmed to perform operations. The operations may include accessing sensor position data describing a position of a first sensor on the autonomous vehicle, and accessing actor position data describing a position of a first actor in an environment of the autonomous vehicle. The operations may also include determining that a line from the position of the first sensor to the position of the first actor intersects the trailer, and determining that the first actor is in a blind spot of the autonomous vehicle based at least in part on determining that the line from the position of the first sensor to the position of the first actor intersects the trailer. The operations may additionally include generating a motion plan for the autonomous vehicle based at least in part on determining that the first actor is in a blind spot of the autonomous vehicle, and controlling the autonomous vehicle according to the motion plan.

[0022] These and other implementations of the disclosure may include one or more of the following features.

[0023] In some embodiments, generating a motion plan may include modifying a cost associated with a first candidate motion plan in a plurality of candidate motion plans to generate a modified cost, and using the modified cost to determine a cost associated with the first candidate motion plan. Generating a motion plan may further include selecting a motion plan from the plurality of candidate motion plans. The selection may be based at least in part on the cost associated with the first candidate motion plan.

[0024] In some embodiments, generating a motion plan may include modifying the first candidate motion plan to generate a modified first motion plan. The modification may be based at least in part on determining that the first actor is in a blind spot of the autonomous vehicle. Generating a motion plan may further include selecting a motion plan from a plurality of candidate motion plans. The plurality of candidate motion plans may include the modified first motion plan.

[0025] In some implementations, modifying the first candidate motion plan may include at least one of: modifying an acceleration associated with the first candidate motion plan; or modifying a lateral velocity associated with the first candidate motion plan.

[0026] In some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium may include instructions thereon that, when executed by one or more processors, cause the one or more processors to perform operations. The operations may include accessing sensor position data describing the position of a first sensor on an autonomous vehicle. The autonomous vehicle may include a tractor and a trailer. The operations may also include accessing actor position data describing the position of a first actor in the environment of the autonomous vehicle, and determining that a line from the position of the first sensor to the position of the first actor intersects with the trailer. The operations may also include determining that the first actor is in a blind spot of the autonomous vehicle based at least in part on determining that the line from the position of the first sensor to the position of the first actor intersects with the trailer, and generating a motion plan for the autonomous vehicle based at least in part on determining that the first actor is in a blind spot of the autonomous vehicle. The operations may also include controlling the autonomous vehicle according to the motion plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a block diagram of an example operating scenario according to some embodiments of the present disclosure.

[0028] Figure 2 is a block diagram of an example autonomous system for an autonomous platform according to some implementations of the present disclosure.

[0029] Figure 3A An example environment including an example autonomous vehicle is shown.

[0030] Figure 3B yes Figure 3A An overhead view of the sample environment.

[0031] Figure 3C Another example environment including another example autonomous vehicle is shown.

[0032] Figure 3D yes Figure 3C An overhead view of the sample environment.

[0033] Figure 4 is a diagram illustrating one example of an environment including an autonomous vehicle traveling on a travel route.

[0034] Figure 5 It is shown that Figure 4 A diagram of another example environment for an autonomous vehicle.

[0035] Figure 6 is a flow chart of a process that may be performed by an autonomous system to control an autonomous towing vehicle taking into account blind spot data.

[0036] Figure 7 It is shown Figure 4 Diagram of the arrangement of autonomous vehicles and actors.

[0037] Figure 8 It is shown Figure 4 FIG. 1 is a diagram of another arrangement of an autonomous vehicle and an actor, where the actor is not in the blind spot.

[0038] Fig. 9 It is shown Figure 4 FIG. 1 is a diagram of yet another arrangement of an autonomous vehicle and an actor, where the autonomous vehicle is turning left and the actor is in a blind spot.

[0039] Fig.10 is a flow chart of a process for determining whether an actor exists in a blind spot.

[0040] Fig.11 is a flow chart of a process for determining whether an actor exists in a blind spot.

[0041] Fig.12 is a flow diagram of a process that may be performed to track multiple actors relative to a blind spot.

[0042] Fig.13 is a flow diagram of a process that may be performed to track multiple actors relative to a blind spot.

[0043] Fig.14 is a flow chart of a process that may be performed to generate a motion plan when the blind spot data indicates the presence of an actor in the blind spot.

[0044] Fig.15 is a block diagram of an example computing ecosystem according to an example implementation of the present disclosure. DETAILED DESCRIPTION

[0045] The following describes the technology of the present disclosure in the context of an autonomous vehicle for exemplary purposes only. As described herein, the technology described herein is not limited to autonomous vehicles and can be implemented for or within other autonomous platforms and other computing systems.

[0046] refer to Figures 1 to 15 , example embodiments of the present disclosure are discussed in further detail. Figure 1 1 is a block diagram of an example operating scenario according to some embodiments of the present disclosure. In the example operating scenario, an environment 100 includes an autonomous platform 110 and a plurality of objects, including a first actor 120, a second actor 130, and a third actor 140. In the example operating scenario, the autonomous platform 110 can move through the environment 100 and interact with objects (e.g., the first actor 120, the second actor 130, the third actor 140, etc.) located within the environment 100. The autonomous platform 110 can optionally be configured to communicate with a remote system 160 via a network 170.

[0047] Environment 100 may be or include an indoor environment (e.g., within one or more facilities, etc.) or an outdoor environment. For example, an indoor environment may be an environment surrounded by a structure such as a building (e.g., a service warehouse, a maintenance location, a manufacturing facility, etc.). An outdoor environment may be, for example, one or more regions in the outside world, such as, for example, one or more rural regions (e.g., having one or more rural travel routes, etc.), one or more urban regions (e.g., having one or more urban travel routes, highways, etc.), one or more suburban regions (e.g., having one or more suburban travel routes, etc.), or other outdoor environments.

[0048] Autonomous platform 110 may be any type of platform configured to operate within environment 100. For example, autonomous platform 110 may be a vehicle configured to autonomously sense and operate within environment 100. The vehicle may be a ground-based autonomous vehicle, such as, for example, an autonomous car, truck, van, etc. Autonomous platform 110 may be an autonomous vehicle that may control, connect to, or otherwise be associated with an appliance, attachments, and / or accessories for transporting people or cargo. This may include, for example, an autonomous tractor that is optionally coupled to a cargo trailer. Additionally or alternatively, autonomous platform 110 may be any other type of vehicle, such as one or more aircraft, water-based vehicles, space-based vehicles, other land-based vehicles, etc.

[0049] The autonomous platform 110 may be configured to communicate with the remote system 160. For example, the remote system 160 may communicate with the autonomous platform 110 for assistance (e.g., navigation assistance, situation response assistance, etc.), control (e.g., fleet management, teleoperation, etc.), maintenance (e.g., updating, monitoring, etc.), or other local or remote tasks. In some embodiments, the remote system 160 may provide data indicating tasks that the autonomous platform 110 should perform. For example, as further described herein, the remote system 160 may provide data indicating that the autonomous platform 110 is to perform a trip / service, such as a user transportation trip / service, a delivery trip / service (e.g., for goods, freight, items), etc.

[0050] The autonomous platform 110 may communicate with the remote system 160 using a network 170. The network 170 may facilitate the transmission of signals (e.g., electronic signals, etc.) or data (e.g., data from a computing device, etc.), and may include various wired (e.g., twisted pair cables, etc.) or wireless communication mechanisms (e.g., cellular, wireless, satellite, microwave, radio frequency, etc.) or any combination of any desired network topology (or topologies). For example, the network 170 may include a local area network (e.g., an intranet, etc.), a wide area network (e.g., the Internet, etc.), a wireless LAN network (e.g., via Wi-Fi, etc.), a cellular network, a SATCOM network, a VHF network, a HF network, a WiMAX-based network, or any other suitable communication network (or combination thereof) for transmitting data to or from the autonomous platform 110.

[0051] For example, Figure 1 As shown, environment 100 may include one or more objects. An object may be an object that is not in motion or is not predicted to move (a "static object") or an object that is in motion or is predicted to be in motion (a "dynamic object" or "actor"). In some embodiments, environment 100 may include any number of actors, such as, for example, one or more pedestrians, animals, vehicles, etc. Actors may move within the environment according to one or more actor trajectories. For example, first actor 120 may move along any one of first actor trajectories 122A-C, second actor 130 may move along any one of second actor trajectories 132, third actor 140 may move along any one of third actor trajectories 142, and so on.

[0052] As further described herein, the autonomous platform 110 can utilize its autonomous system to detect these actors (and their movements) and plan its motion according to one or more platform trajectories 112A-C to navigate through the environment 100. The autonomous platform 110 can include an onboard computing system 180. The onboard computing system 180 can include one or more processors and one or more memory devices. The one or more memory devices can store instructions executable by the one or more processors to cause the one or more processors to perform operations or functions associated with the autonomous platform 110, including implementing its autonomous system.

[0053] Figure 22 is a block diagram of an example autonomous system 200 for an autonomous platform according to some embodiments of the present disclosure. In some embodiments, the autonomous system 200 may be implemented by a computing system of the autonomous platform (e.g., the onboard computing system 180 of the autonomous platform 110). The autonomous system 200 may be operable to obtain input from a sensor 202 or other input device. In some embodiments, the autonomous system 200 may additionally obtain platform data 208 (e.g., map data 210) from local or remote storage. The autonomous system 200 may generate control outputs for controlling the autonomous platform (e.g., via a platform control device 212, etc.) based on the sensor data 204, the map data 210, or other data. The autonomous system 200 may include different subsystems for performing various autonomous operations. The subsystems may include a positioning system 230, a perception system 240, a planning system 250, and a control system 260. Positioning system 230 may determine the location of the autonomous platform within its environment; perception system 240 may detect, classify, and track objects and actors in the environment; planning system 250 may determine the trajectory of the autonomous platform; and control system 260 may convert the trajectory into vehicle control for controlling the autonomous platform. Autonomous system 200 may be implemented by one or more onboard computing systems. The subsystems may include one or more processors and one or more memory devices. The one or more memory devices may store instructions executable by the one or more processors to cause the one or more processors to perform operations or functions associated with the subsystems. The computing resources of autonomous system 200 may be shared among its subsystems, or a subsystem may have a set of dedicated computing resources.

[0054] In some embodiments, autonomous system 200 may be implemented for or by an autonomous vehicle (e.g., a ground-based autonomous vehicle). Autonomous system 200 may perform various processing techniques on inputs (e.g., sensor data 204, map data 210) to perceive and understand the vehicle's surroundings and generate an appropriate set of control outputs to implement for traversing the vehicle's surroundings (e.g., Figure 1 In some embodiments, an autonomous vehicle implementing autonomous system 200 can drive, navigate, operate, etc. with minimal or no interaction from a human operator (e.g., driver, pilot, etc.).

[0055] In some embodiments, the autonomous platform can be configured to operate in multiple operating modes. For example, the autonomous platform can be configured to operate in a fully autonomous (e.g., automatic driving, etc.) operating mode, in which the autonomous platform is controllable without user input (e.g., can drive and navigate without input from a human operator present in or away from the autonomous vehicle, etc.). The autonomous platform can operate in a semi-autonomous operating mode, in which the autonomous platform can operate with some input from a human operator present in the autonomous platform (or a human operator away from the autonomous platform). In some embodiments, the autonomous platform can enter a manual operating mode, in which the autonomous platform is completely controllable by a human operator (e.g., a human driver, etc.), and can be prohibited or disabled (e.g., temporarily, permanently, etc.) to perform autonomous navigation (e.g., autonomous driving, etc.). The autonomous platform can be configured to operate in other modes, such as, for example, a parking or sleep mode (e.g., for use between tasks such as waiting to provide a trip / service, recharging, etc.). In some embodiments, the autonomous platform can implement vehicle operation assistance technology (e.g., collision mitigation system, power-assisted steering, etc.), for example, to help assist the human operator of the autonomous platform (e.g., when in manual mode, etc.).

[0056] Autonomous system 200 may be located onboard an autonomous platform (e.g., on or within an autonomous platform) and may be configured to operate the autonomous platform in a variety of environments. The environment may be a real-world environment or a simulated environment. In some embodiments, one or more simulation computing devices may simulate one or more of the following: sensor 202, sensor data 204, communication interface 206, platform data 208, or platform control device 212 for simulating the operation of autonomous system 200.

[0057] In some implementations, autonomous system 200 may communicate with one or more networks or other systems via communication interface 206. Communication interface 206 may include a processor for communicating with one or more networks (e.g., Figure 1 The communication interface 206 may include any suitable components for interfacing with the network 170, etc., including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components that can help facilitate communication. In some embodiments, the communication interface 206 may include multiple components (e.g., antennas, transmitters, or receivers, etc.) that allow it to implement and utilize various communication technologies (e.g., multiple-input multiple-output (MIMO) technology, etc.).

[0058] In some embodiments, autonomous system 200 can communicate with one or more computing devices (e.g., remote system 160) remote from the autonomous platform via one or more networks (e.g., network 170) using communication interface 206. For example, in some examples, one or more inputs, data, or functions of autonomous system 200 can be supplemented or replaced by a remote system communicating via communication interface 206. For example, in some embodiments, map data 210 can be downloaded to a remote system via a network using communication interface 206. In some examples, one or more of positioning system 230, perception system 240, planning system 250, or control system 260 can be updated, influenced, nudged, communicated, etc. by a remote system for assistance, maintenance, situational response coverage, management, etc.

[0059] The sensor 202 may be located onboard the autonomous platform. In some embodiments, the sensor 202 may include one or more types of sensors. For example, one or more sensors may include an image capture device (e.g., a visible spectrum camera, an infrared camera, etc.). Additionally or alternatively, the sensor 202 may include one or more depth capture devices. For example, the sensor 202 may include one or more light detection and ranging (LIDAR) sensors or radio detection and ranging (RADAR) sensors. The sensor 202 may be configured to generate point data describing at least a portion of a three hundred and sixty degree view of the surrounding environment. The point data may be point cloud data (e.g., three-dimensional LIDAR point cloud data, RADAR point cloud data). In some embodiments, one or more of the sensors 202 for capturing depth information may be fixed to a rotating device so that the sensor 202 rotates around an axis. The sensor 202 may rotate around an axis while capturing data in spaced sector groups describing different portions of a three hundred and sixty degree view of the surrounding environment of the autonomous platform. In some embodiments, one or more sensors 202 for capturing depth information may be solid-state.

[0060] Sensors 202 may be configured to capture sensor data 204 indicative of or otherwise associated with at least a portion of the environment of the autonomous platform. Sensor data 204 may include image data (e.g., 2D camera data, video data, etc.), RADAR data, LIDAR data (e.g., 3D point cloud data, etc.), audio data, or other types of data. In some embodiments, autonomous system 200 may obtain input from additional types of sensors, such as an inertial measurement unit (IMU), an altimeter, an inclinometer, an odometer device, a location or positioning device (e.g., GPS, compass), a wheel encoder, or other types of sensors. In some embodiments, autonomous system 200 may obtain sensor data 204 associated with a particular component or system of the autonomous platform. The sensor data 204 may indicate, for example, wheel speed, component temperature, steering angle, cargo or passenger status, etc. In some embodiments, autonomous system 200 may obtain sensor data 204 associated with ambient environmental conditions, such as environmental or weather conditions. In some embodiments, sensor data 204 may include multimodal sensor data. The multimodal sensor data may be obtained by at least two different types of sensors (e.g., of sensors 202) and may indicate static objects or actors within the environment of the autonomous platform. The multimodal sensor data may include at least two types of sensor data (e.g., camera and LIDAR data). In some embodiments, the autonomous platform may use sensor data 204 for sensors that are remote from the autonomous platform (e.g., outside the vehicle). This may include, for example, sensor data 204 captured by different autonomous platforms.

[0061] Some or all of sensors 202 may have a sensing cycle. For example, one or more LIDAR sensors may scan an area during a particular sensing cycle to detect objects or environments in the area. In some versions of those embodiments, a given instance of LIDAR data may include LIDAR data from a given sensing cycle of the one or more LIDAR sensors. For example, a given LIDAR data instance may correspond to a given scan of one or more LIDAR sensors generated during a sensing cycle of the one or more LIDAR sensors.

[0062] The LIDAR data generated during a sensing cycle of one or more LIDAR sensors may include, for example, a plurality of points reflected from the surface of an object in the environment of the autonomous platform and detected as data points by at least one receiver component of the one or more LIDAR sensors. During a given sensing cycle, the one or more LIDAR sensors may detect a plurality of data points in an area of ​​the environment of the autonomous platform. One or more data points may also be captured in a subsequent sensing cycle. Thus, the range and speed of a point indicated by a LIDAR data scan of one or more LIDAR sensors may be based on a plurality of sensing cycle events by reference to a previous (and optionally subsequent) sensing cycle event. In some versions of those embodiments, a plurality of (e.g., all) sensing cycles may have the same duration, the same field of view, and / or the same waveform distribution pattern (by directing the waveform during the sensing cycle). For example, multiple scans may have the same duration (e.g., 50ms, 100ms, 200ms, 300ms, or other duration) and the same field of view (e.g., 60°, 90°, 180°, 360°, or other field of view). Furthermore, in some implementations, sensors 202 other than LIDAR sensors may similarly have sensing periods similar to the example sensing periods of the LIDAR sensors described herein.

[0063] The autonomous system 200 may obtain map data 210 associated with an environment in which the autonomous platform was, is, or will be located. The map data 210 may provide information about an environment or geographic area. For example, the map data 210 may provide information about the following: identification and location of different routes of travel (e.g., roads, etc.), routes of travel segments (e.g., road segments, etc.), buildings or other items or objects (e.g., lamp posts, crosswalks, curbs, etc.); location and direction of boundaries or boundary markers (e.g., location and direction of traffic lanes, parking lanes, turn lanes, bicycle lanes, other lanes, etc.); traffic control data (e.g., location and instructions of signs, traffic lights, other traffic control devices, etc.); obstacle information (e.g., temporary or permanent blockages, etc.); event data (e.g., road closures / traffic rule changes due to parades, concerts, sporting events, etc.); nominal vehicle path data (e.g., indicating an ideal vehicle path, such as along the center of a particular lane, etc.); or any other map data that provides information to assist the autonomous platform in understanding its surroundings and its relationships. In some embodiments, the map data 210 may include high-definition map information. Additionally or alternatively, the map data 210 may include sparse map data (eg, a lane map, etc.) In some implementations, the sensor data 204 may be fused with the map data 210 or used to update the map data 210 in real time.

[0064] Autonomous system 200 may include positioning system 230, which may provide the autonomous platform with an understanding of its location and orientation in the environment. In some examples, positioning system 230 may support one or more other subsystems of autonomous system 200, such as by providing a unified local reference frame for performing, for example, perception operations, planning operations, or control operations.

[0065] In some embodiments, the positioning system 230 can determine the current location of the autonomous platform. The current location can include a global location (e.g., with respect to a geographic reference anchor, etc.) or a relative location (e.g., with respect to an object in the environment, etc.). The positioning system 230 can generally include or interface with any device or circuit for analyzing the location or location change of an autonomous platform (e.g., an autonomous ground-based vehicle, etc.). For example, the positioning system 230 can determine the location by using one or more of the following: an inertial sensor (e.g., an inertial measurement unit, etc.), a satellite positioning system, a radio receiver, a networked device (e.g., based on an IP address, etc.), triangulation or proximity to a network access point or other network component (e.g., a cellular tower, a Wi-Fi access point, etc.), or other suitable technology. The location of the autonomous platform can be used by various subsystems of the autonomous system 200 or provided to a remote computing system (e.g., using the communication interface 206).

[0066] In some embodiments, the positioning system 230 can register the relative positions of elements of the surrounding environment of the autonomous platform with recorded positions in the map data 210. For example, the positioning system 230 can process the sensor data 204 (e.g., LIDAR data, RADAR data, camera data, etc.) for alignment or other registration to a map of the surrounding environment (e.g., from the map data 210) to understand the position of the autonomous platform within the environment. Thus, in some embodiments, the autonomous platform can identify its position within the surrounding environment (e.g., across six axes, etc.) based on a search of the map data 210. In some embodiments, given an initial location, the positioning system 230 can update the location of the autonomous platform using incremental realignment based on recorded or estimated deviations from the initial location. In some embodiments, the location can be registered directly within the map data 210.

[0067] In some implementations, map data 210 may include a large amount of data that is subdivided into geographic tiles, such that a desired area of ​​a map stored in map data 210 may be reconstructed from one or more tiles. For example, autonomous system 200 may stitch together multiple tiles selected from map data 210 based on a location obtained by positioning system 230 (e.g., multiple tiles selected near the location).

[0068] In some embodiments, the positioning system 230 can determine the position (e.g., relative or absolute) of one or more attachments or accessories of the autonomous platform. For example, the autonomous platform can be associated with a cargo platform, and the positioning system 230 can provide the position of one or more points on the cargo platform. For example, the cargo platform can include a trailer or other equipment towed or otherwise attached or manipulated by the autonomous platform, and the positioning system 230 can provide data describing the position (e.g., absolute, relative, etc.) of the autonomous platform and the cargo platform. Such information can be obtained by other autonomous systems to help operate the autonomous platform.

[0069] The autonomous system 200 may include a perception system 240, which may allow the autonomous platform to detect, classify, and track objects and actors in its environment. Environmental features or objects sensed within the environment may be those within the field of view of the sensor 202 or those predicted to be occluded by the sensor 202. This may include objects that are not in motion or not predicted to move (static objects) or objects that are in motion or predicted to be in motion (dynamic objects / actors).

[0070] The perception system 240 may determine one or more states (e.g., current or past states, etc.) of one or more objects within the surrounding environment of the autonomous platform. For example, the state may describe (e.g., for a given time, time period, etc.) an estimate of the current or past location (also referred to as position) of the object; current or past speed / velocity; current or past acceleration; current or past heading; current or past orientation; size / footprint (e.g., represented by boundary shape, object highlighting, etc.); classification (e.g., pedestrian category vs. vehicle category vs. bicycle category, etc.); uncertainty associated therewith; or other state information. In some embodiments, the perception system 240 may determine the state using one or more algorithms or machine learning models configured to identify / classify objects based on input from the sensors 202. The perception system 240 may use different modalities of the sensor data 204 to generate a representation of the environment to be processed by one or more algorithms or machine learning models. In some embodiments, the state of one or more identified or unidentified objects may be maintained and updated over time as the autonomous platform continues to perceive or interact with the object (e.g., maneuver with or around the object, yield to the object, etc.). In this way, perception system 240 can provide an understanding of the current state of the environment (e.g., including objects therein, etc.) informed by a record of previous states of the environment (e.g., including the movement history of objects therein). Such information can be helpful when the autonomous platform plans its movement in the environment.

[0071] The autonomous system 200 may include a planning system 250, which may be configured to determine how the autonomous platform interacts with and moves within its environment. The planning system 250 may determine one or more motion plans for the autonomous platform. The motion plan may include one or more trajectories (e.g., motion trajectories) indicating a path for the autonomous platform to follow. The trajectory may have a certain length or time range. The length or time range may be defined by a calculated planning horizon of the planning system 250. The motion trajectory may be defined by one or more waypoints (with associated coordinates). A waypoint may be a future location of the autonomous platform. The motion plan may be continuously generated, updated, and considered by the planning system 250.

[0072] The motion planning system 250 can determine a strategy for the autonomous platform. A strategy can be a set of discrete decisions made by the autonomous platform (e.g., yield to an actor, yield to an opposite direction of an actor, merge, change lanes). A strategy can be selected from a plurality of potential strategies. The selected strategy can be a minimum cost strategy determined by one or more cost functions. The cost function can, for example, evaluate the probability of colliding with another actor or object.

[0073] The planning system 250 may determine the desired trajectory for executing the strategy. For example, the planning system 250 may obtain one or more trajectories for executing one or more strategies. The planning system 250 may evaluate the trajectory or strategy (e.g., using scores, costs, rewards, constraints, etc.) and rank them. For example, the planning system 250 may use the predicted output indicating the interaction between the trajectory of the autonomous platform and one or more objects (e.g., proximity, intersection, etc.) to inform the evaluation of the candidate trajectory or strategy of the autonomous platform. In some embodiments, the planning system 250 may use static costs to evaluate the trajectory of the autonomous platform (e.g., "avoid lane boundaries", "minimize bumps", etc.). Additionally or alternatively, the planning system 250 may use dynamic costs to evaluate the trajectory or strategy of the autonomous platform based on the predicted results of the current operating scenario (e.g., predicted trajectories or strategies that lead to interactions between actors, predicted trajectories or strategies that lead to interactions between actors and autonomous platforms, etc.). The planning system 250 may rank the trajectory based on one or more static costs, one or more dynamic costs, or a combination thereof. The planning system 250 may select a motion plan (and a corresponding trajectory) based on the ranking of multiple candidate trajectories. In some implementations, planning system 250 may select the highest ranked candidate or the highest ranked feasible candidate.

[0074] Planning system 250 may then validate the selected trajectory against one or more constraints before the trajectory is executed by the autonomous platform.

[0075] To assist in its motion planning decisions, the planning system 250 may be configured to perform a prediction function. The planning system 250 may predict future states of the environment. This may include predicting future states of other actors in the environment. In some embodiments, the planning system 250 may predict future states based on current or past states (e.g., developed or maintained by the perception system 240). In some embodiments, the future state may be or include a predicted trajectory (e.g., position over time) of an object (such as other actors) in the environment. In some embodiments, one or more future states may include one or more probabilities associated therewith (e.g., marginal probabilities, conditional probabilities). For example, one or more probabilities may include one or more probabilities conditioned on a strategy or trajectory option available to the autonomous platform. Additionally or alternatively, a probability may include a probability conditioned on a trajectory option available to one or more other actors.

[0076] In some implementations, the planning system 250 can perform interactive prediction. The planning system 250 can determine the motion plan of the autonomous platform with an understanding of how the predicted future state of the environment may be affected by the execution of one or more candidate motion plans. As an example, referring again to Figure 1 , the autonomous platform 110 may determine candidate motion plans corresponding to a set of platform trajectories 112A-C that correspond to first actor trajectories 122A-C of the first actor 120, trajectory 132 of the second actor 130, and trajectory 142 of the third actor 140, respectively (e.g., having respective trajectory correspondences indicated by matching line types). For example, the autonomous platform 110 (e.g., using its autonomous system 200) may predict that a platform trajectory 112A that moves the autonomous platform 110 more quickly into an area in front of the first actor 120 may be associated with the first actor 120 reducing forward speed according to the first actor trajectory 122A and quickly yielding to the autonomous platform 110. Additionally or alternatively, the autonomous platform 110 may predict that a platform trajectory 112B that moves the autonomous platform 110 slightly into an area in front of the first actor 120 may be associated with the first actor 120 reducing speed slightly and slowly yielding to the autonomous platform 110 according to the first actor trajectory 122B. Additionally or alternatively, the autonomous platform 110 may predict that maintaining a parallel alignment with the first actor 120 in the platform trajectory 112C may be associated with the first actor 120 not yielding any distance to the autonomous platform 110 according to the first actor trajectory 122C. Based on a comparison of the predicted scenarios to a set of desired outcomes (e.g., by scoring the scenarios based on costs or rewards), the planning system 250 may select a motion plan (and its associated trajectory) based on the autonomous platform's interaction with the environment 100. In this manner, for example, the autonomous platform 110 may interleave its prediction and motion planning functions.

[0077] In order to implement the selected motion plan, the autonomous system 200 may include a control system 260 (e.g., a vehicle control system). In general, the control system 260 may provide an interface between the autonomous system 200 and the platform control device 212 for implementing the strategy and motion plan generated by the planning system 250. For example, the control system 260 may implement the selected motion plan / trajectory to control the motion of the autonomous platform in its environment by following the selected trajectory (e.g., the waypoints included therein). For example, the control system 260 may, for example, convert the motion plan into instructions for the appropriate platform control device 212 (e.g., acceleration control, braking control, steering control, etc.). As an example, the control system 260 may convert the selected motion plan into instructions to adjust the steering component (e.g., steering angle) by a certain degree, apply a certain amount of braking force, increase / decrease speed, etc. In some embodiments, the control system 260 may communicate with the platform control device 212 via a communication channel, the communication channel including, for example, one or more data buses (e.g., controller area network (CAN), etc.), on-board diagnostic connectors (e.g., OBD-II, etc.), or a combination of wired or wireless communication links. The platform control device 212 may send data, messages, signals, etc. to or obtain data, messages, signals, etc. from the autonomous system 200 (or vice versa) through the communication channel.

[0078] The autonomous system 200 may receive an assistance signal from the remote assistance system 270 via the communication interface 206. The remote assistance system 270 may communicate with the autonomous system 200 via a network (e.g., as the remote system 160 via the network 170). In some embodiments, the autonomous system 200 may initiate a communication session with the remote assistance system 270. For example, the autonomous system 200 may initiate a session based on or in response to a trigger. In some embodiments, the trigger may be an alarm, an error signal, a map feature, a request, a location, a traffic condition, a road condition, etc.

[0079] After initiating the session, the autonomous system 200 can provide contextual data to the remote assistance system 270. The contextual data can include sensor data 204 and state data of the autonomous platform. For example, the contextual data can include a real-time camera feed from a camera of the autonomous platform and the current speed of the autonomous platform. An operator (e.g., a human operator) of the remote assistance system 270 can use the contextual data to select auxiliary signals. The auxiliary signals can provide values ​​or adjustments for various operating parameters or characteristics of the autonomous system 200. For example, the auxiliary signals can include waypoints (e.g., paths around obstacles, lane changes, etc.), speed or acceleration profiles (e.g., speed limits, etc.), relative motion instructions (e.g., convoy formation, etc.), operating characteristics (e.g., use of auxiliary systems, reduced energy handling modes, etc.), or other signals to assist the autonomous system 200.

[0080] The autonomous system 200 may use the auxiliary signal as input into one or more autonomous subsystems to perform autonomous functions. For example, the planning system 250 may receive the auxiliary signal as input for generating a motion plan. For example, the auxiliary signal may include constraints for generating a motion plan. Additionally or alternatively, the auxiliary signal may include a cost or reward adjustment for influencing the motion plan of the planning system 250. Additionally or alternatively, the autonomous system 200 may consider the auxiliary signal as advisory input that is considered in addition to other received data (e.g., sensor input, etc.).

[0081] Autonomous system 200 may be platform agnostic, and control system 260 may provide control instructions to platform control device 212 for a variety of different platforms for autonomous mobility (e.g., multiple different autonomous platforms equipped with an autonomous control system). This may include a variety of different types of autonomous vehicles (e.g., sedans, vans, SUVs, trucks, electric vehicles, combustion-powered vehicles, etc.) from a variety of different manufacturers / developers operating in a variety of different environments and, in some embodiments, performing one or more vehicle services.

[0082] For example, refer to Figure 3A , the operating environment may include a dense environment 300. The autonomous platform may include an autonomous vehicle 310 controlled by the autonomous system 200. In some embodiments, the autonomous vehicle 310 may be configured for maneuverability in a dense environment, such as having a configured wheelbase or other specifications. In some embodiments, the autonomous vehicle 310 may be configured to transport cargo or passengers. In some embodiments, the autonomous vehicle 310 may be configured to transport many passengers (e.g., a high-top truck, a shuttle, a bus, etc.). In some embodiments, the autonomous vehicle 310 may be configured to transport cargo, such as large quantities of cargo (e.g., a truck, a van, a high-top truck, etc.) or smaller cargo (e.g., food, personal packages, etc.).

[0083] refer to Figure 3B , the selected top view 302 of the dense environment 300 is shown as overlaid with an example trip / service between a first location 304 and a second location 306. The example trip / service can be assigned to an autonomous vehicle 320, for example, by a remote computing system. Autonomous vehicle 320 can be, for example, a vehicle of the same type as autonomous vehicle 310. The example trip / service can include transporting passengers or cargo between the first location 304 and the second location 306. In some embodiments, the example trip / service can include traveling to or through one or more intermediate locations, such as loading or unloading passengers or cargo. In some embodiments, the example trip / service can be pre-scheduled (e.g., for regular traversal, such as according to a transportation schedule). In some embodiments, the example trip / service can be on-demand (e.g., requested by a taxi, ride-sharing, ride-hailing, courier, delivery service, etc. or used to perform a taxi, ride-sharing, ride-hailing, courier, delivery service, etc.).

[0084] refer to Figure 3C , in another example, the operating environment may include an open travel path environment 330. The autonomous platform may include an autonomous vehicle 350 controlled by the autonomous system 200. This may include an autonomous tractor for an autonomous truck. In some embodiments, the autonomous vehicle 350 may be configured for high payload transport (e.g., transporting cargo or other goods or passengers in bulk), such as for long-distance, high payload transport. For example, the autonomous vehicle 350 may include one or more cargo platform attachment devices, such as a trailer 352. Although in Figure 3C 350 as a towing attachment device, but in some embodiments, one or more cargo platforms may be integrated into (e.g., attached to) the chassis of autonomous vehicle 350 (e.g., as in a van, high roof truck, etc.).

[0085] refer to Figure 3D, showing a selected top view of an open route of travel environment 330, including a route of travel 332, an interchange 334, transfer hubs 336 and 338, an entry route of travel 340, and location locations 342 and 344. In some embodiments, an autonomous vehicle (e.g., autonomous vehicle 310 or autonomous vehicle 350) may be assigned an example trip / service to traverse one or more routes of travel 332 (optionally connected by an interchange 334) to transport goods between transfer hub 336 and transfer hub 338. For example, in some embodiments, the example trip / service includes a goods delivery / transportation service, such as a freight delivery / transportation service. The example trip / service may be assigned by a remote computing system. In some embodiments, transfer hub 336 may be an origin point of the goods (e.g., a storage place, a warehouse, a facility, etc.), and transfer hub 338 may be a destination point of the goods (e.g., a retailer, etc.). However, in some embodiments, transfer hub 336 may be an intermediate point along the final trip of the goods item between its respective origin point and its respective destination. For example, the point of origin of the cargo items can be located along the incoming travel route 340 at location 342. Thus, the cargo items can be transported to a transfer hub 336 (e.g., by a human-driven vehicle, by an autonomous vehicle 310, etc.) for staging. At the transfer hub 336, the various cargo items can be grouped or staged for longer distance transportation on the travel route 332.

[0086] In some embodiments of the example trip / service, a group of staging segments of cargo items may be loaded onto an autonomous vehicle (e.g., autonomous vehicle 350) for transport to one or more other transfer hubs, such as transfer hub 338. For example, although not shown, it should be understood that the open travel route environment 330 may include more transfer hubs than transfer hubs 336 and 338, and may include more travel routes 332 interconnected by more interchanges 334. A simplified map is presented here only for the purpose of clarity. In some embodiments, one or more cargo items transported to transfer hub 338 may be assigned to one or more local destinations (e.g., by human-driven vehicles, by autonomous vehicles 310, etc.), such as along an incoming travel route 340 to location 344. In some embodiments, the example trip / service may be pre-scheduled (e.g., for regular traversals, such as in accordance with a transportation schedule). In some embodiments, the example trip / service may be on-demand (e.g., as requested to perform a toll passenger or freight service).

[0087] To improve the performance of an autonomous platform, such as an autonomous vehicle (e.g., autonomous vehicle 310 or 350) controlled at least in part using autonomous system 200, various examples described herein utilize blind spot monitoring. In some arrangements, the field of view of some or all of sensors 202 of the autonomous platform may be blocked during use. This may create blind spots.

[0088] Blind spots can present challenges to an autonomous platform. For example, during operation of an autonomous platform, an actor may enter and / or leave a blind spot. When an actor is present in a blind spot, the sensor data 204 may not provide sufficient information about the actor. This may make it difficult for the autonomous platform to determine the actor's actor trajectory and / or actor state, thereby complicating the process of determining a motion plan for the autonomous platform.

[0089] An example type of autonomous platform that may be subject to sensor blind spots is an autonomous vehicle that includes an autonomous tractor pulling a trailer. For example, an autonomous tractor may have sensors 202 located on the autonomous tractor. When a trailer is coupled to the autonomous tractor, the trailer may block the field of view of one or more sensors 202, thereby causing a blind spot. In addition, a trailer coupled to the autonomous tractor via a hitch may move as the autonomous tractor maneuvers. For example, if the autonomous tractor is turning, the position of the trailer relative to the sensors 202 may change. This may also change the location and size of the blind spot.

[0090] Various examples described herein relate to systems and methods for controlling an autonomous vehicle while taking into account at least one vehicle blind spot. For example, an autonomous vehicle (e.g., an autonomous system thereof) may access sensor position data describing a position of at least one sensor on the autonomous vehicle and actor position data describing a position of an actor in an environment of the autonomous vehicle. Using the position of the at least one sensor and the position of the actor, the autonomous vehicle may determine whether the actor is present in a blind spot of the autonomous vehicle. Upon determining that the actor is present in the blind spot of the autonomous vehicle, the autonomous vehicle may determine a motion plan that takes into account the presence of the actor in the blind spot. The autonomous vehicle may perform motion planning to maneuver in the environment.

[0091] Figure 4 4 is a diagram showing an example of an environment 400 including an autonomous vehicle 401 traveling on a travel path 403. Autonomous vehicle 401 includes autonomous tractor 402 and trailer 404. Autonomous tractor 402 includes sensor assembly 410 including one or more sensors 420, 422. Sensors 420, 422 can be or include any suitable sensors or sensor types. In some examples, sensors 420, 422 can be configured similarly to those described herein with respect to Figure 2 For example, sensors 420, 422 may generate signals similar to those also described with respect to Figure 2 The sensor data 204 described herein is the sensor data. Although Figure 4 Two sensors 420, 422 are shown in FIG. 4 , but it should be understood that the autonomous vehicle may include more or less than two sensors.

[0092] Autonomous tractor 402 includes autonomous system 200, such as Figure 2 as described. Figure 4 Blind zone 411 is shown. Figure 4 In the example of FIG. 4 , blind spot 411 is caused by trailer 404 blocking the field of view of sensors 420, 422. In some examples, blind spot 411 is a portion of environment 400 that cannot be observed by either sensor 420, 422. For example, a portion of environment 400 that is obscured relative to one sensor 420, 422 but not the other may not be part of blind spot 411. In other example embodiments, blind spot 411 may include a portion of environment 400 that is obscured relative to one sensor 420, 422 but not the other sensor.

[0093] Also shown on the travel path 403 are actors 406 and 408. Figure 4 In the example of FIG. 4 , actors 408 and 406 are cars. However, it should be understood that an autonomous vehicle (such as autonomous vehicle 401) may encounter other types of actors on travel route 403 and / or other travel routes. Other types of actors ( Figure 4 Other types of vehicles (not shown) may include buses, trucks, motorcycles, etc. Other types of actors ( Figure 4 403 ). The path 403 may also include animals, pedestrians, debris, and / or other obstacles present on the path 403 .

[0094] exist Figure 4 In the example of , actor 408 is shown outside of blind spot 411 of autonomous vehicle 401. Therefore, actor 408 can be observed by at least one of sensors 420, 422 positioned at sensor assembly 410. Therefore, sensor data 204 generated by sensors 420, 422 can directly or indirectly include information that perception system 240 can use to determine properties of actor 408, such as the location of actor 408, the trajectory of actor 408, the speed of actor 408, the acceleration of actor 408, etc.

[0095] In contrast, because actor 406 exists in blind spot 411, it may not be observed by sensors 420, 422. As a result, sensor data 204 generated by sensors 420, 422 may lack information that can be used by perception system 240 to derive some or all properties of actor 406. This, in turn, may limit how the presence of actor 406 is considered when determining a motion plan for autonomous vehicle 401.

[0096] In some examples, the perception system 240 can implement a blind spot subsystem 412 to monitor and / or track actors in the blind spot 411. The blind spot subsystem 412 can include hardware and / or software components to monitor the presence of actors in the blind spot 411. The blind spot subsystem 412 can generate blind spot data 414. The blind spot data 414 can describe one or more actors in the blind spot 411. In an example embodiment, the blind spot data 414 includes a flag or other binary indicator describing whether there are any actors in the blind spot 411. Optionally, the blind spot data 414 can indicate the number of actors believed to be in the blind spot 411. In some examples, the blind spot data 414 can also describe the location of the blind spot 411 or an estimated or worst-case location of an actor in the blind spot 411.

[0097] Blind spot data 414 may be provided to planning system 250. Planning system 250 may take into account blind spot data 414 to generate motion plan data 416. Motion plan data 416 may describe at least one motion plan for autonomous vehicle 401 taking into account the presence of at least one actor, such as actor 406, in blind spot 411. Motion plan data 416 may be provided to control system 260, which may control autonomous vehicle 401 according to one or more motion plans described by motion plan data 416.

[0098] In some implementations, blind spot subsystem 412 monitors blind spot 411 using pose instances of trailer 404. Pose instances of trailer 404 describe the location and, in some examples, the orientation of trailer 404 at a particular time or instance. For example, pose instances of trailer 404 may indicate the location of one or more corners of trailer 404. In some examples, pose instances of trailer 404 are determined by perception system 240 using sensor data 204.

[0099] In some examples, the pose instance of trailer 404 is determined based on a corresponding pose instance of autonomous tractor 402. The pose instance of autonomous tractor 402 may be generated, for example, by positioning system 230. In some examples, the pose instance of autonomous tractor 402 is determined from sensor data 204, which includes, for example, IMU data, inclinometer data, odometer data, location or positioning device data, wheel encoder data, etc. Additionally, in some examples, the pose instance of autonomous tractor 402 may be determined using LIDAR data or other data generated by an image capture device.

[0100] The perception system 240 can receive a tractor pose instance and generate a corresponding trailer pose instance based at least in part on the tractor pose instance. For example, the perception system 240 can receive sensor data 204 indicating the position of the tractor 402. The perception system 240 can use the sensor data to determine the position of the trailer 404 relative to the autonomous tractor 402. In some embodiments, the trailer pose instance can include position and orientation information of the trailer 404, which can be mechanically coupled to the autonomous tractor 402. In some examples, the trailer pose instance can be defined relative to the same reference frame as the tractor pose instance. In addition, in some examples, the trailer pose instance can be defined relative to the autonomous tractor 402.

[0101] Autonomous system 200 (e.g., its blind spot subsystem 412) can generate blind spot data 414 in any suitable manner. In some embodiments, blind spot subsystem 412 determines a line from the location of each of the one or more sensors at sensor assembly 410 to the location of the actor. If blind spot subsystem 412 determines that the line intersects all of trailer 404, as indicated by the trailer pose instance, it can generate blind spot data 414 indicating that at least one actor is present in blind spot 411. If blind spot subsystem 412 determines that the line does not intersect trailer 404, it can indicate that the actor is not in blind spot 411.

[0102] Autonomous system 200 (e.g., perception system 240 and / or its blind spot subsystem 412) can use pose data describing a pose instance of trailer 404 to determine whether the line intersects trailer 404. The pose data describing a pose instance of trailer 404 can be generated from sensor data 204 describing environment 400. As described herein, the pose instance of trailer 404 can be determined in a common reference frame as a corresponding pose instance of autonomous tractor 402. Thus, blind spot subsystem 412 can determine the position of one or more associated sensors 420, 422 of autonomous vehicle 401 and the position of trailer 404 in the same reference frame.

[0103] In some examples, blind spot data 414 or other data generated by blind spot subsystem 412 can be used to operate an indicator 430 that can be physically coupled to trailer 404. Indicator 430 can be visible to vehicles (such as actor 406) that are present in blind spot 411. Indicator 430 can be or include a light, a display, or any other suitable output device. When blind spot subsystem 412 does not detect any vehicles in blind spot 411, the indicator can have a first state, and when blind spot subsystem 412 does detect a vehicle in blind spot 411, the indicator can have a second state. In this way, vehicles (such as actor 406) in blind spot 411 can be made aware that autonomous vehicle 401 may not be able to detect them.

[0104] Figure 5 4 is a diagram illustrating another example environment 500 including an autonomous vehicle 401 including an autonomous tractor 402 and a trailer 404. Figure 5 In the example of FIG. 4 , autonomous vehicle 401 is traveling on road 503. Actors 506, 508 are also traveling on road 503. Actors 506, 508 are represented by bounding boxes. In the depicted example, the bounding box is a rectangular prism including eight corners. For example, perception system 240 can generate the shown bounding box around the location indicated by sensor data 204 to include actors 506, 508.

[0105] Figure 5 An arrangement is shown in which the autonomous tractor 402 does not travel in a straight line. Figure 5 In the example of FIG. 4 , autonomous tractor 402 turns to its left. Therefore, trailer 404 is not directly behind autonomous tractor 402, but is offset to the left. This may change the location and / or size of the blind spot experienced by autonomous tractor 402. Blind spot subsystem 412 may take this into account when generating blind spot data 414, for example, as described herein.

[0106] Figure 6 is a flow chart of a process 600 that may be performed by autonomous system 200 to control autonomous vehicle 401 to take into account blind spot data 414, according to some embodiments of the present disclosure. In some examples, process 600 may be performed by autonomous system 200 periodically, such as, for example, once per sensing cycle.

[0107] At operation 602, the autonomous system 200 (e.g., its perception system 240) receives sensor position data. The sensor position data describes the position of one or more sensors of the autonomous tractor 402. In some examples, the sensor position data is described relative to a pose instance of the autonomous tractor 402. For example, the pose instance of the autonomous tractor 402 can be determined relative to a center or other reference position of the autonomous tractor 402. The position of one or more sensors indicated by the sensor position data can be expressed as an offset between the center or other reference position of the autonomous tractor 402 and one or more positions of the corresponding sensors.

[0108] At operation 604, the autonomous system (e.g., its perception system 240) can access estimated location data describing the estimated location of one or more actors. In some examples, operation 604 can be performed with respect to a single actor. Furthermore, in some examples, operation 604 can be performed with respect to more than one actor. For example, operation 604 can be performed with respect to all actors currently being tracked by the perception system 240.

[0109] The sensor data 204 generated by the sensors 202 may be used by the perception system 240 to determine the estimated location of the actor. It should be understood that in some cases, one or more actors may be present in the blind spot 411 of the autonomous tractor 402. Therefore, the current sensor data 204 may not indicate the location of some or all actors. In examples where the current sensor data 204 does not indicate the location of the actor, the perception system 240 may infer the estimated location of the actor from the last known state data describing the last known state of the actor. The last known state data of the actor may include, for example, the last known speed of the actor, the last known acceleration of the actor, the last known trajectory or heading of the actor, and / or the last known orientation of the actor.

[0110] In various examples, the last known state data describing the actor may be determined from previous sensor data 204 (e.g., sensor data 204 generated during one or more previous sensing cycles). Consider an example in which the actor is outside the blind spot 411 for multiple sensing cycles and then moves into the blind spot 411 during the last sensing cycle. The sensor data 204 generated during the sensing cycle before the last sensing cycle may be used to generate state data for the actor, which includes, for example, the speed of the actor, the acceleration of the actor, the heading or trajectory of the actor, the orientation or posture of the actor, etc. Because the actor was in the blind spot 411 during the last sensing cycle, the sensor data 204 generated during the last sensing cycle may not indicate the actor. Therefore, during the last sensing cycle, the last known state of the actor may be based on the state of the actor for one or more recent sensing cycles when the actor was outside the blind spot 411.

[0111] At operation 606, the blind spot subsystem 412 may determine whether the actor is in the blind spot 411. In some examples, the blind spot subsystem 412 may determine a line from the location of the first sensor indicated by the sensor location data to the estimated location of the actor. If the line intersects the trailer 404, as indicated by the trailer posture example, the blind spot subsystem 412 may determine that the actor is in the blind spot 411. If the line does not intersect the trailer 404, the blind spot subsystem 412 may determine that the actor is not in the blind spot 411.

[0112] At operation 606, if the blind spot subsystem 412 determines that an actor is in the blind spot 411, the blind spot data 414 provided to the planning system 250 may indicate that at least one actor is present in the blind spot 411. For example, the blind spot data 414 may include a flag or other binary indicator. When an actor is present in the blind spot 411, the blind spot subsystem 412 may set or assert a blind spot flag to indicate that the actor is present in the blind spot 411.

[0113] At operation 608, the autonomous system 200 (e.g., its planning system 250) may generate motion planning data 416 that reflects the presence of the actor in the blind spot 411. In some implementations, the planning system 250 generates the motion planning data 416 that reflects the presence of the actor in the blind spot 411 by using the blind spot data 414 in at least one cost function used to generate the candidate motion plans and / or by modifying one or more candidate motion plans.

[0114] For example, planning system 250 can increase the cost of one or more candidate motion plans that are affected by the presence of an actor in blind spot 411. Consider an example in which a first candidate motion plan involves applying the brakes of autonomous tractor 402. If an actor is present in blind spot 411, applying the brakes of autonomous tractor 402 may be a less favorable motion plan. Therefore, planning system 250 can modify the cost associated with the first candidate motion plan that includes applying the brakes of autonomous tractor 402 so as to increase its cost, thereby reducing the likelihood of selecting the first candidate motion plan. Consider a second example in which a second candidate motion plan involves changing lanes quickly (e.g., with a high level of speed and / or lateral acceleration). If an actor is present in blind spot 411, changing lanes quickly may not be the most favorable motion plan. Therefore, planning system 250 can modify the cost associated with the second candidate motion plan to increase its cost, thereby reducing the likelihood of selecting the second candidate motion plan.

[0115] In addition to or in lieu of modifying one or more costs associated with the candidate motion plans, planning system 250 may modify one or more candidate motion plans to account for the presence of an agent in blind spot 411. Planning system 250 may modify the candidate motion plans to reduce the likelihood of a negative outcome associated with the agent in blind spot 411. Consider again the first example described above where a first candidate motion plan involves applying the brakes of autonomous tractor 402. Planning system 250 may modify the first candidate motion plan to reduce deceleration, for example, by more gently applying the brakes of autonomous tractor 402. Consider also again the second example described above where a second candidate motion plan involves a rapid lane change. Planning system 250 may modify the second candidate motion plan, for example, to reduce the lateral velocity or acceleration of autonomous vehicle 401 during the lane change.

[0116] After generating the motion plan at operation 608 , at operation 614 the autonomous system 200 (eg, the control system 260 thereof) may use the generated motion plan data 416 to control the autonomous tractor 402 .

[0117] At operation 606, if the blind spot subsystem 412 determines that the actor is not in the blind spot 411, the blind spot subsystem 412 may determine whether the blind spot 411 is clean at optional operation 610. For example, even if no actor tracked by the perception system 240 during the current sensing cycle is in the blind spot 411, it may be the case that the actor entered the blind spot 411 during a previous sensing cycle. If the actor entered the blind spot 411 during a previous sensing cycle, the blind spot may not be clean. If the blind spot is clean at optional operation 610, the autonomous system 200 (e.g., its planning system 250) may generate a motion plan at operation 612, as described herein. The autonomous system 200 (e.g., its control system 260) may use the motion plan data 416 generated at operation 612 to control the autonomous tractor 402. On the other hand, if the blind spot 411 is not clear at optional operation 610, the autonomous system 200 (e.g., its planning system 250) can generate a motion plan based on the actors in the blind spot at operation 608, as described herein. It should be understood that in some examples, operation 610 can be omitted. In these examples, if none of the currently tracked actors are in the blind spot 411 at operation 606, the autonomous system 200 can generate a motion plan at operation 612, as described herein.

[0118] Figure 7 700 is a diagram showing an arrangement of autonomous vehicle 401 and actor 702. Figure 7In the example of FIG. 4 , the sensor assembly 410 includes two sensors 422, 420. The sensor 420 is positioned on the right-hand side of the sensor assembly 410. The sensor 422 is positioned on the left-hand side of the sensor assembly 410.

[0119] Figure 7 4 shows a line 704 determined from the location of sensor 420 to the outer corners of actor 702. In some examples, the outer corners of actor 702 are the outer corners of a bounding box generated by perception system 240 for actor 702, such as Figure 5 Line 704 may be determined by blind zone subsystem 412. Figure 7 As shown in the example of , line 704 intersects trailer 404 . Figure 7 Also shown is another line 706 determined by the blind spot subsystem 412 from the location of the sensor 422 to the outer corner of the actor 702. In this example, both lines 704, 706 intersect the trailer 404. Therefore, the actor 702 may exist in the blind spot 411.

[0120] Figure 7 An alternative arrangement is also shown in which lines 708, 710 are determined from the locations of the respective sensors 420, 422 to the center of the actor 702. In this case, both lines 708, 710 intersect the trailer 404, indicating that the actor 702 is in the blind spot 411. In various examples, the blind spot subsystem 412 may determine lines to the outside corners of the actor 702, such as lines 704, 706, or to the center of the actor 702, such as lines 708, 710.

[0121] Figure 8 800 is a diagram showing another arrangement of autonomous vehicle 401 and actor 802, wherein actor 802 is not in blind spot 411. Figure 8 In the example of FIG. 8 , the blind spot subsystem 412 can determine a first line 804 from the location of the sensor 420 to the outside corner of the actor 802. Figure 8 As shown, line 804 intersects trailer 404. However, blind spot subsystem 412 can also determine line 806 from the location of sensor 422 to the outside corner of actor 802. Line 806 does not intersect trailer 404. In some examples, blind spot subsystem 412 can determine that actor 802 is not in blind spot 411 because at least one of sensors 420, 422 has a direct line of sight to at least a portion of actor 802. Figure 8 Also included are lines 808, 810 determined from the locations of the respective sensors 420, 422 to the center of the actor 802. In this example, and Figure 7In the example of , regardless of whether the blind spot subsystem 412 considers the line to the outer corner of the actor 702 , 802 or the line to the center of the actor 702 , 802 , the corresponding actor 702 , 802 is in the blind spot 411 .

[0122] Fig. 9 9 is a diagram 900 showing yet another arrangement of autonomous vehicle 401 and actor 902, where autonomous vehicle 401 is turning left and actor 902 is in blind spot 411. Fig. 9 As shown, blind spot 411 is larger than when autonomous vehicle 401 is traveling in a straight line, and is also in a different position, extending further to the left of autonomous vehicle 401. It should be understood that blind spot 411 may change accordingly as autonomous vehicle 401 maneuvers, and as trailer 404 articulates relative to autonomous tractor 402. Fig. 9 In the example of , actor 902 exists in blind spot 411 , as shown by lines 904 , 906 from the locations of respective sensors 420 , 422 to the outer corners of actor 902 and / or lines 908 , 910 from the locations of respective sensors 420 , 422 to the center of actor 902 .

[0123] Fig.10 1 is a flow chart of a process 1000 for determining whether an actor is present in a blind spot 411. Process 1000 illustrates one example of how the blind spot subsystem 412 may perform operation 606 of process 600 described herein. At operation 1002, the blind spot subsystem 412 may generate a box indicating the location of the trailer 404. The box may be based on a suggested example of the trailer 404 described herein. At operation 1004, the blind spot subsystem 412 may access estimated actor location data. The estimated actor location may be based on sensor data 204 from a current sensing cycle, and / or may be based on sensor data 204 from a previous sensing cycle.

[0124] At operation 1006, the blind spot subsystem 412 may determine a line from the sensor location of the first sensor to the estimated location of the actor. At operation 1008, the blind spot subsystem 412 may determine whether the line from operation 1006 intersects a box indicating the location of the trailer 404. If the line does not intersect the box, the blind spot subsystem 412 may determine at operation 1010 that the actor is not present in the blind spot.

[0125] If blind spot subsystem 412 determines at operation 1008 that the line determined at operation 1006 does intersect the box indicating the location of trailer 404, it may indicate that the field of view of the sensor considered at operation 1006 is obstructed such that the actor is not represented by sensor data 204 generated by the first sensor. Therefore, blind spot subsystem 412 may determine whether autonomous vehicle 401 includes any other sensors that may generate sensor data 204 describing the actor. At operation 1012, blind spot subsystem 412 determines whether there are any additional sensors at autonomous vehicle 401. If there is at least one more sensor at operation 1012, blind spot subsystem 412 may consider the next sensor at operation 1014 and return to operation 1006 to determine a line from the next sensor to the estimated location of the actor. If there are no more sensors, then at operation 1016, blind spot subsystem 412 may indicate that the actor is present in blind spot 411. This may include, for example, generating blind spot data 414 with an asserted flag to indicate that an actor is present in blind spot 411 .

[0126] Fig.11 is a flow diagram of a process 1100 for determining whether an actor is present in the blind spot 411. The process 1100 illustrates another example manner in which the blind spot subsystem 412 may perform the operation 606 of the process 600 described herein.

[0127] At operation 1102, the blind spot subsystem 412 determines the location of the blind spot 411. This can be performed in any suitable manner. For example, the blind spot subsystem 412 can access the most recent pose instance of the trailer 404. The blind spot subsystem 412 can determine the intersection between the field of view of one or more sensors 420, 422 and the location of the trailer 404, as indicated by the trailer pose instance.

[0128] At operation 1104, the blind spot subsystem 412 may access estimated actor location data describing an estimated location of an actor, e.g., as described herein. If at operation 1106, the estimated location of the actor intersects the blind spot determined by operation 1102, the blind spot subsystem 412 may generate blind spot data 414 indicating the presence of at least one actor in the blind spot 411 at operation 1110. If the estimated location of the actor does not intersect the blind spot determined at operation 1102, then at operation 1108, the blind spot subsystem 412 may determine that the actor is not present in the blind spot 411.

[0129] Fig.12 is a flow diagram of a process 1200 that may be performed by the blind spot subsystem 412 to track multiple actors relative to the blind spot 411. For example, the process 1200 illustrates one example of how the blind spot subsystem 412 may perform the operation 610 of the process 600 described herein.

[0130] At operation 1202, the blind spot subsystem 412 accesses data describing the presence of actors in the blind spot 411. For example, the data accessed at operation 1202 may include some or all of the blind spot data 414 previously generated by the blind spot subsystem 412. In some examples, the blind spot subsystem 412 may track the number of actors present in the blind spot 411. For example, when the blind spot subsystem 412 determines that an actor is present in the blind spot 411, the blind spot subsystem 412 may increment an indicator of the number of actors in the blind spot, as described herein. In some examples, accessing the data describing the number of actors in the blind spot may include accessing an indicator maintained by the blind spot subsystem 412.

[0131] At operation 1204, the blind spot subsystem 412 may determine whether any actor is determined to be leaving the blind spot 411. The blind spot subsystem 412 may determine that an actor is leaving the blind spot 411, for example, if the sensor data 204 generated for a given sensing cycle shows that the actor is approaching the blind spot 411, and, for example, the state data of the actor shows a track outside the blind spot 411, or shows that the object was not tracked in a previous sensing cycle.

[0132] If no actor leaves the blind spot at operation 1204, the blind spot subsystem 412 may return to operation 1202. In some examples, the blind spot subsystem 412 may perform operations 1202 and 1204 for each sensing cycle of the autonomous system 200. Furthermore, in some examples, the blind spot subsystem 412 may perform operations 1202 and 1204 at less than all sensing cycles of the autonomous system 200, such as, for example, every two cycles, every three cycles, every four cycles, etc.

[0133] If the actor has left the blind zone 411 at operation 1204, the blind zone subsystem 412 may decrement the number of actors in the blind zone 411. If more than one actor is determined to be leaving the blind zone 411, the number of actors in the blind zone 411 may be decremented a number of times corresponding to the number of actors determined to be leaving the blind zone 411. At operation 1208, the blind zone subsystem may determine whether any actors remain in the blind zone 411. For example, if the number of actors in the blind zone 411 is one or less than one before operation 1206, there may be no actors in the blind zone 411 at operation 1208. Conversely, if the number of actors in the blind zone 411 is greater than one at operation 1206, there may be remaining actors in the blind zone 411. If there are remaining actors in the blind zone 411 at operation 1208, the blind zone subsystem 412 may return to operation 1202, for example, at the next sensing cycle, as described herein. If there are no actors remaining in the blind spot at operation 1208 , the blind spot subsystem 412 may determine that the blind spot is clear at operation 1210 .

[0134] Fig.13 1 is a flow chart of a process 1300 that may be performed by the blind spot subsystem 412 to track multiple actors relative to the blind spot 411. For example, the process 1300 illustrates another example of how the blind spot subsystem 412 may perform the optional operation 610 of the process 600 described herein. At operation 1302, the blind spot subsystem 412 accesses data describing the presence of actors in the blind spot 411. For example, the data accessed at operation 1302 may include some or all of the blind spot data 414 previously generated by the blind spot subsystem 412, as described herein.

[0135] At operation 1304, the blind spot subsystem 412 may determine whether any actor in the blind spot 411 has been considered to be in the blind spot 411 for more than a threshold time period. For example, the autonomous system 200 may not always determine when an actor in the blind spot 411 leaves the blind spot 411. Furthermore, in some examples, it may be undesirable and / or impractical to determine an actor leaving the blind spot 411. Therefore, the blind spot subsystem 412 may consider the actor to have left the blind spot 411 when a threshold time period has passed since it was determined that the actor had entered the blind spot 411. The threshold time period may be any suitable time period, such as, for example, a predetermined number of sensing cycles and / or a predetermined time period (e.g., two minutes, five minutes, etc.).

[0136] If it is determined at operation 1304 that no actor has been in the blind zone 411 for more than the threshold time period, the blind zone subsystem 412 may return to operation 1302. If one or more actors have been in the blind zone 411 for more than the threshold time period, the blind zone subsystem 412 may decrement the number of actors in the blind zone 411 at operation 1306. The number of actors in the blind zone 411 may be decremented a number of times corresponding to the number of actors determined at operation 1304 to have been in the blind zone 411 for more than the threshold time period.

[0137] At operation 1308, the blind spot subsystem 412 may determine whether any actors remain in the blind spot 411. If there are remaining actors in the blind spot 411 at operation 1308, the blind spot subsystem 412 may return to operation 1302, for example, at the next sensing cycle, as described herein. If there are no remaining actors in the blind spot at operation 1308, the blind spot subsystem 412 may determine at operation 1310 that the blind spot 411 is clear.

[0138] Fig.14 is a flow chart of a process 1400 that may be performed by the autonomous system 200 (eg, the planning system 250 thereof) to generate a motion plan when the blind spot data 414 generated by the blind spot subsystem 412 indicates that an actor is present in the blind spot 411 .

[0139] At operation 1402, the planning system 250 may access a blind spot indicator, such as in the blind spot data 414. As described herein, if the blind spot subsystem 412 determines that there is at least one actor in the blind spot 411, the blind spot indicator or sign may have a first state. If the blind spot subsystem 412 determines that there is no actor in the blind spot 411, the blind spot indicator may have a second state different from the first state.

[0140] At operation 1404, the planning system 250 determines whether the state of the blind spot indicator indicates that at least one object is present in the blind spot 411. If the state of the blind spot indicator indicates that there is no object in the blind spot 411, the planning system 250 may determine a motion plan at operation 1408, and the control system 260 may use the motion plan to control the vehicle at operation 1410.

[0141] If it is determined at operation 1404 that the state of the blind spot indicator indicates that at least one actor is present in the blind spot 411, the planning system 250 may modify the motion plan at operation 1406. Modifying the motion plan may include modifying at least one cost function associated with the candidate motion plan, as described herein. When modifying the motion plan at operation 1406, the motion planning system may determine the motion plan based on the modified motion plan at operation 1408, and the control system 260 may control the vehicle using the generated motion plan at operation 1410.

[0142] Fig.15 is a block diagram of an example computing ecosystem 10 according to an example implementation of the present disclosure. The example computing ecosystem 10 may include a first computing system 20 and a second computing system 40 communicatively coupled via one or more networks 60. In some implementations, the first computing system 20 or the second computing system 40 may implement one or more of the systems, operations, or functionalities described herein for data annotation (e.g., remote system 160, onboard computing system 180, autonomous system 200, etc.).

[0143] In some embodiments, the first computing system 20 may be included in an autonomous platform and used to perform the functions of the autonomous platform as described herein. For example, the first computing system 20 may be located on an autonomous vehicle and implement an autonomous system for autonomously operating an autonomous vehicle. In some embodiments, the first computing system 20 may represent the entire on-board computing system or a portion thereof (e.g., a positioning system 230, a perception system 240, a planning system 250, a control system 260, or a combination thereof, etc.). In other embodiments, the first computing system 20 may not be located on an autonomous platform. The first computing system 20 may include one or more different physical computing devices 21.

[0144] The first computing system 20 (e.g., its computing device 21) may include one or more processors 22 and a memory 23. The one or more processors 22 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or a plurality of processors operatively connected. The memory 23 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory devices, a flash memory device, etc., and combinations thereof.

[0145] The memory 23 may store information that may be accessed by the one or more processors 22. For example, the memory 23 (e.g., one or more non-transitory computer-readable storage media, memory devices, etc.) may store data 24 that may be obtained (e.g., received, accessed, written, manipulated, created, generated, stored, pulled, downloaded, etc.). The data 24 may include, for example, sensor data, map data, data associated with autonomous functions (e.g., data associated with perception, planning, or control functions), simulation data, or any data or information described herein. In some embodiments, the first computing system 20 may obtain the data from one or more memory devices remote from the first computing system 20.

[0146] The memory 23 may store computer readable instructions 25 that may be executed by one or more processors 22. The instructions 25 may be software written in any suitable programming language, or may be implemented in hardware. Additionally or alternatively, the instructions 25 may be executed in logically or virtually separate threads on the processor 22.

[0147] For example, the memory 23 may store instructions 25 executable by one or more processors (e.g., by one or more processors 22, by one or more other processors, etc.) to perform (e.g., using the computing device 21, the first computing system 20, or other systems having processors that execute instructions) any operations, functions, or methods / processes (or portions thereof) described herein. For example, the operations may include generating boundary data for annotating sensor data, such as part of a training pipeline for implementing a machine learning machine vision system.

[0148] In some embodiments, the first computing system 20 may store or include one or more models 26. In some embodiments, the model 26 may be or may otherwise include one or more machine learning models. As an example, the model 26 may be or may otherwise include various machine learning models, such as, for example, a regression network, a generative adversarial network, a neural network (e.g., a deep neural network), a support vector machine, a decision tree, an ensemble model, a k-nearest neighbor model, a Bayesian network, or other types of models including linear models or nonlinear models. Example neural networks include feedforward neural networks, recursive neural networks (e.g., long short-term memory recursive neural networks), convolutional neural networks, or other forms of neural networks. For example, the first computing system 20 may include one or more models for implementing a subsystem of the autonomous system 200, including any of the following: a positioning system 230, a perception system 240, a planning system 250, or a control system 260.

[0149] In some implementations, the first computing system 20 may obtain one or more models 26 using the communication interface 27 to communicate with the second computing system 40 via the network 60. For example, the first computing system 20 may store the models 26 (e.g., one or more machine learning models) in the memory 23. The first computing system 20 may then use or otherwise implement the models 26 (e.g., via the processor 22). As an example, the first computing system 20 may implement the models 26 to locate the autonomous platform in the environment, perceive the environment of the autonomous platform or objects therein, plan one or more future states of the autonomous platform to move through the environment, control the autonomous platform to interact with the environment, and the like.

[0150] The second computing system 40 may include one or more computing devices 41. The second computing system 40 may include one or more processors 42 and a memory 43. The one or more processors 42 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or a plurality of processors operatively connected. The memory 43 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory devices, a flash memory device, etc., and combinations thereof.

[0151] The memory 43 may store information that may be accessed by the one or more processors 42. For example, the memory 43 (e.g., one or more non-transitory computer-readable storage media, memory devices, etc.) may store data 44 that may be obtained. The data 44 may include, for example, sensor data, model parameters, map data, simulation data, simulated environment scenarios, simulated sensor data, data associated with vehicle trips / services, or any data or information described herein. In some embodiments, the second computing system 40 may obtain the data from one or more memory devices remote from the second computing system 40.

[0152] The memory 43 may also store computer readable instructions 45 that may be executed by one or more processors 42. The instructions 45 may be software written in any suitable programming language, or may be implemented in hardware. Additionally or alternatively, the instructions 45 may be executed in logically or virtually separate threads on the processor 42.

[0153] For example, memory 43 may store instructions 45 that may be executed (e.g., by one or more processors 42, by one or more processors 22, by one or more other processors, etc.) to perform (e.g., using computing device 41, second computing system 40, or other system having a processor for executing instructions, such as computing device 21 or first computing system 20) any operation, function, or method / process described herein. This may include, for example, functionality of autonomous system 200 (e.g., positioning, perception, planning, control, etc.) or other functionality associated with an autonomous platform (e.g., remote assistance, mapping, fleet management, trip / service allocation and matching, etc.).

[0154] In some embodiments, the second computing system 40 may include one or more server computing devices. Where the second computing system 40 includes multiple server computing devices, such server computing devices may operate according to various computing architectures, including, for example, sequential computing architectures, parallel computing architectures, or some combination thereof.

[0155] In addition to or in lieu of the model 26 at the first computing system 20, the second computing system 40 may include one or more models 46. As an example, the model 46 may be or may otherwise include various machine learning models, such as, for example, a regression network, a generative adversarial network, a neural network (e.g., a deep neural network), a support vector machine, a decision tree, an ensemble model, a k-nearest neighbor model, a Bayesian network, or other types of models including linear models or nonlinear models. Example neural networks include feedforward neural networks, recursive neural networks (e.g., long short-term memory recursive neural networks), convolutional neural networks, or other forms of neural networks. For example, the second computing system 40 may include one or more models of the autonomous system 200.

[0156] In some embodiments, the second computing system 40 or the first computing system 20 may train one or more machine learning models of the model 26 or the model 46 by using one or more model trainers 47 and training data 48. The model trainer 47 may train any of the models 26 or the model 46 using one or more training or learning algorithms. An example training technique is back propagation of errors. In some embodiments, the model trainer 47 may perform supervised training techniques using labeled training data. In other embodiments, the model trainer 47 may perform unsupervised training techniques using unlabeled training data. In some embodiments, the training data 48 may include simulated training data (e.g., training data obtained from simulated scenarios, inputs, configurations, environments, etc.). In some embodiments, the second computing system 40 may implement a simulation for obtaining training data 48 or for implementing a model trainer 47 for training or testing the model 26 or the model 46. For example, the model trainer 47 may train one or more components of a machine learning model for the autonomous system 200 using an unsupervised training technique using an objective function (e.g., cost, reward, heuristic, constraint, etc.). In some embodiments, the model trainer 47 may perform a variety of generalization techniques to improve the generalization ability of the trained model. Generalization techniques include weight decay, dropout, or other techniques.

[0157] For example, in some embodiments, the second computing system 40 can generate training data 48 according to example aspects of the present disclosure. For example, the second computing system 40 can generate training data 48. The second computing system 40 can use the training data 48 to train the model 26. For example, in some embodiments, the first computing system 20 may include a computing system that is onboard or otherwise associated with a real or simulated autonomous vehicle. In some embodiments, the model 26 may include a perception or machine vision model configured for onboard deployment on a real or simulated autonomous vehicle or in service of a real or simulated autonomous vehicle. In this way, for example, the second computing system 40 can provide a training pipeline for training the model 26.

[0158] The first computing system 20 and the second computing system 40 may each include a communication interface 27 and 49, respectively. The communication interfaces 27, 49 may be used to communicate with each other or with one or more other systems or devices, including systems or devices located remotely from the first computing system 20 or the second computing system 40. The communication interfaces 27, 49 may include any circuits, components, software, etc. for communicating with one or more networks (e.g., network 60). In some implementations, the communication interfaces 27, 49 may include, for example, one or more of a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software, or hardware for communicating data.

[0159] The network 60 may be any type of network or combination of networks that allows communication between devices. In some embodiments, the network may include one or more of a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a peer-to-peer communication link, or some combination thereof, and may include any number of wired or wireless links. Communication on the network 60 may be achieved, for example, by using a network interface that uses any type of protocol, protection scheme, encoding, format, encapsulation, etc.

[0160] Fig.15 An example computing ecosystem 10 that can be used to implement the present disclosure is shown. Other systems may also be used. For example, in some embodiments, the first computing system 20 may include a model trainer 47 and training data 48. In such embodiments, the models 26, 46 may be trained and used locally at the first computing system 20. As another example, in some embodiments, the computing system 20 may not be connected to other computing systems. Additionally, components shown or discussed as included in one of the computing systems 20 or 40 may be included in another of the computing systems 20 or 40 instead.

[0161] Computing tasks discussed herein that are performed at a computing device remote from an autonomous platform (e.g., an autonomous vehicle) may alternatively be performed at the autonomous platform (e.g., via a vehicle computing system of an autonomous vehicle), and vice versa. Such configurations may be implemented without departing from the scope of the present disclosure. The use of computer-based systems allows for a variety of possible configurations, combinations, and divisions of tasks and functions between components. Computer-implemented operations may be performed on a single component or across multiple components. Computer-implemented tasks or operations may be performed sequentially or in parallel. Data and instructions may be stored in a single memory device or across multiple memory devices.

[0162] Various aspects of the present disclosure have been described according to the illustrative embodiments of the present disclosure. By reading the present disclosure, a person of ordinary skill in the art can think of many other embodiments, modifications or changes within the scope and spirit of the attached claims. Any and all features in the following claims can be combined or rearranged in any possible way. Therefore, the scope of the present disclosure is to be taken as an example rather than as a limitation, and the present disclosure does not exclude the inclusion of such modifications, changes or additions to the subject matter, which is obvious to a person of ordinary skill in the art. In addition, a list of example elements connected by conjunctions such as "and", "or", "but" is used herein to describe terms. It should be understood that such connections are provided only for illustrative purposes. For example, a list connected by a specific conjunction such as "or" can refer to "at least one" or "any combination thereof" of the example elements listed therein, and unless otherwise specified, "or" is understood as "and / or". In addition, terms such as "based on" should be understood as "based at least in part on".

[0163] Using the disclosure provided herein, one of ordinary skill in the art will understand that the elements of any claim, operation, or process discussed herein may be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of the present disclosure. For illustrative purposes, some claims are described with letters that reference claim elements and are not meant to be restrictive. Letter references do not imply a particular order of operations. For example, letter identifiers such as (a), (b), (c), ..., (i), (ii), (iii), ..., etc. may be used to illustrate operations. Such identifiers are provided for the convenience of the reader and do not indicate a particular order of steps or operations. An operation indicated by a list identifier (a), (i), etc. may be performed before, after, or in parallel with another operation indicated by a list identifier (b), (ii), etc.

Claims

1. A method for controlling an autonomous vehicle comprising a tractor and a trailer, the method comprising: accessing sensor location data describing a location of a first sensor on the autonomous vehicle; accessing actor position data describing a position of a first actor in an environment of the autonomous vehicle; determining that a line from the location of the first sensor to the location of the first actor intersects the trailer; determining that the first actor is in a blind spot of the autonomous vehicle based at least in part on determining that the line from the location of the first sensor to the location of the first actor intersects the trailer; generating a motion plan for the autonomous vehicle based at least in part on determining that the first actor is in the blind spot of the autonomous vehicle; as well as The autonomous vehicle is controlled according to the motion plan.

2. The method of claim 1, wherein the sensor location data further describes a location of a second sensor on the autonomous vehicle, the method further comprising: Determining that a second line from the position of the second sensor to the position of the first actor intersects the trailer, and determining that the first actor is in the blind spot of the autonomous vehicle is also at least partially based on determining that the second line from the position of the second sensor to the position of the first actor intersects the trailer.

3. The method of claim 1 , wherein the sensor location data describes locations of a plurality of sensors, the plurality of sensors including the first sensor, the method further comprising: Determining that a line from a corresponding position of each of the plurality of sensors intersects the trailer, and determining that the first actor is in the blind spot of the autonomous vehicle is also based at least in part on determining that the line from a corresponding position of each of the plurality of sensors intersects the trailer.

4. The method of claim 1, wherein the sensor location data further describes a location of a second actor in the environment of the autonomous vehicle, and wherein the sensor location data further describes a location of a second sensor on the autonomous vehicle, the method further comprising: determining that a line from the location of the first sensor to the location of the second actor intersects the trailer; determining that a line from the location of the second sensor to the location of the second actor does not intersect the trailer; as well as It is determined that the second actor is outside of the blind spot of the autonomous vehicle.

5. The method according to claim 1, further comprising: determining a pose of the trailer, the pose of the trailer describing a position of the trailer; as well as The pose of the trailer is used to determine that a line from the location of the first sensor to the location of the first actor intersects the trailer. 6 . The method of claim 1 , further comprising determining a location of the first actor based at least in part on last known state data describing a last known state of the first actor.

7. According to the method of claim 6, the last known state data includes at least one of the last known location of the first actor, the last known speed of the first actor, the last known acceleration of the first actor, the last known heading of the first actor, or the last known orientation of the first actor.

8. The method of claim 1, further comprising setting a blind spot flag to indicate that at least one actor is in the blind spot of the autonomous vehicle, generating the motion plan based at least in part on the blind spot flag.

9. The method of claim 1 further comprising generating an estimated trajectory of the first actor in the blind spot of the autonomous vehicle using last known state data describing the last known state of the first actor, generating the motion plan based at least in part on the estimated trajectory of the first actor.

10. The method according to claim 1, generating the motion plan comprises: modifying a cost associated with a first candidate motion plan of the plurality of candidate motion plans to generate a modified cost; determining a cost associated with the first candidate motion plan using the modified cost; as well as The motion plan is selected from the plurality of candidate motion plans, the selection being based at least in part on the cost associated with the first candidate motion plan.

11. The method according to claim 1, generating the motion plan comprises: modifying a first candidate motion plan to generate a modified first motion plan, the modification being based at least in part on determining that the first actor is in the blind spot of the autonomous vehicle; as well as The motion plan is selected from a plurality of candidate motion plans, the plurality of candidate motion plans including the modified first motion plan.

12. The method of claim 11, wherein modifying the first candidate motion plan comprises at least one of: modifying the acceleration associated with the first candidate motion plan; or A lateral velocity associated with the first candidate motion plan is modified.

13. The method according to claim 1, further comprising: accessing blind spot actor data describing at least one actor in the blind spot of the autonomous vehicle; determining that sensor data generated by at least one sensor on the autonomous vehicle indicates that an actor has exited the blind spot of the autonomous vehicle; as well as Decrement the number of actors in the blind zone.

14. The method according to claim 1, further comprising: accessing blind spot actor data describing at least one actor in the blind spot of the autonomous vehicle, the at least one actor including the first actor; determining that more than a threshold period of time has elapsed since determining that the first actor is in the blind spot of the autonomous vehicle; as well as Decrement the number of actors in the blind zone.

15. The method according to claim 1, further comprising: determining that there are no remaining actors in the blind spot of the autonomous vehicle; generating a second motion plan for the autonomous vehicle based at least in part on determining that there are no remaining actors in the blind spot of the autonomous vehicle; as well as The autonomous vehicle is controlled according to the second motion plan.

Images