System for configuring activated lighting for directional guidance of an autonomous vehicle

Abstract

Disclosed are systems, apparatus, and methods for implementing active safety systems in autonomous vehicles. An autonomous vehicle may be traveling along a trajectory in an environment external to the autonomous vehicle. The environment may include one or more objects that could potentially collide with the autonomous vehicle, such as static and / or dynamic objects, or objects that pose some other hazard to occupants within the autonomous vehicle and / or to the autonomous vehicle. The objects are shown as having a trajectory that, if not altered, could result in a potential collision with the autonomous vehicle. The autonomous vehicle can detect the object using a sensor system to sense the environment and can take action to mitigate or prevent the object's potential collision with the autonomous vehicle.

Description

[Technical Field]

[0001] FIELD Embodiments of the present application generally relate to methods, systems, and apparatus for safety systems in robotic vehicles.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This PCT international application is a continuation of U.S. patent application Ser. No. 14 / 756,994, filed November 4, 2015, entitled "Autonomous Vehicle Fleet Service and System," U.S. patent application Ser. No. 14 / 932,948, filed November 4, 2015, entitled "Active Lighting Control for Communicating a State of an Autonomous Vehicle to Entities in a Surrounding Environment," and U.S. patent application Ser. No. 14 / 932,962, filed November 4, 2015, entitled "Robotic Vehicle Active Safety Systems and Methods," which is a continuation of U.S. patent application Ser. No. 14 / 932,959, filed November 4, 2015, entitled "Autonomous Vehicle Fleet Service And System," and U.S. patent application Ser. No. 14 / 932,962, filed November 4, 2015, entitled "Adaptive Mapping To Navigate Autonomous Vehicles Responsive To" No. 14 / 932,963, entitled "Physical Environment Changes," all of which are incorporated herein by reference in their entirety for all purposes. [Background technology]

[0003] Autonomous vehicles, such as those of the type configured to transport passengers in urban environments, may encounter many situations in which the autonomous vehicle should communicate its operational intent to people, vehicles, etc., such as the direction in which the autonomous vehicle is driving or will be driving. Moreover, passengers in autonomous vehicles may experience some uncertainty regarding determining which autonomous vehicle is assigned to service their transportation needs. For example, when there are many autonomous vehicles in front of them, a passenger scheduling a ride in one of the autonomous vehicles may want to easily distinguish which autonomous vehicle is intended for them.

[0004] Therefore, there is a need for a system, apparatus, and method for enforcing the operational status and intent of a robotic vehicle. [Brief explanation of the drawings]

[0005] Various embodiments or examples (“Examples”) are disclosed in the following detailed description and the accompanying drawings. [Figure 1] FIG. 1 illustrates an example of a system for implementing an active safety system in an autonomous vehicle. [Figure 2A] FIG. 1 illustrates an example of a flow diagram for implementing an active safety system in an autonomous vehicle. [Figure 2B] FIG. 10 illustrates another example of a flow diagram for implementing an active safety system in an autonomous vehicle. [Figure 2C] FIG. 10 illustrates yet another example of a flow diagram for implementing an active safety system in an autonomous vehicle. [Figure 3A] FIG. 1 illustrates an example of a system for implementing an active safety system in an autonomous vehicle. [Figure 3B] FIG. 1 illustrates another example of a system for implementing an active safety system in an autonomous vehicle. [Figure 4] FIG. 1 illustrates an example flow diagram for implementing a perception system in an autonomous vehicle. [Figure 5] FIG. 1 illustrates an example of object prioritization by a planner system in an autonomous vehicle. [Figure 6] FIG. 1 is a top plan view of an example of threshold locations and associated escalating alerts in an active safety system in an autonomous vehicle. [Figure 7] FIG. 1 illustrates an example of a flow diagram for implementing a planner system in an autonomous vehicle. [Figure 8] FIG. 1 illustrates an example of a block diagram of a system in an autonomous vehicle. [Figure 9] FIG. 1 is a top view of an example acoustic beam steering array in an exterior safety system for an autonomous vehicle. [Figure 10A] 1A-1C are top plan views of two examples of sensor coverage. [Figure 10B] 10A-10C are top plan views of two other examples of sensor coverage. [Figure 11A] FIG. 1 illustrates an example of an acoustic beam steering array in an exterior safety system for an autonomous vehicle. [Figure 11B] FIG. 1 illustrates an example flow diagram for implementing acoustic beam steering in an autonomous vehicle. [Figure 11C] FIG. 10 is a top plan view of an example of an autonomous vehicle directing acoustic energy associated with an acoustic alert to an object. [Figure 12A] FIG. 1 illustrates an example of a light emitter in an external safety system for an autonomous vehicle. [Figure 12B] 1 is a contour diagram of an example of a light emitter in an external safety system of an autonomous vehicle. FIG. [Figure 12C] FIG. 10 is a top plan view of an example of light emitter activation based on the orientation of an autonomous vehicle relative to an object. [Figure 12D] FIG. 10 is a contour diagram of an example of light emitter activation based on the orientation of an autonomous vehicle relative to an object. [Figure 12E] FIG. 1 illustrates an example flow diagram for implementing visual alerts from light emitters in an autonomous vehicle. [Figure 12F] FIG. 1 is a top plan view of an example of an autonomous vehicle light emitter that emits light to implement a visual alert. [Figure 13A] FIG. 1 illustrates an example of a bladder system in an external safety system for an autonomous vehicle. [Figure 13B] FIG. 1 illustrates an example of a bladder in an external safety system for an autonomous vehicle. [Figure 13C] FIG. 1 illustrates an example of bladder deployment in an autonomous vehicle. [Figure 14] FIG. 1 illustrates an example of a seat belt tensioning system in an interior safety system of an autonomous vehicle. [Figure 15] FIG. 1 illustrates an example of a seat actuator system in an interior safety system of an autonomous vehicle. [Figure 16A] FIG. 1 illustrates an example of a driving system in an autonomous vehicle. [Figure 16B] FIG. 1 illustrates an example of an obstacle avoidance maneuver in an autonomous vehicle. [Figure 16C] FIG. 1 illustrates another example of an obstacle avoidance maneuver in an autonomous vehicle. [Figure 17] FIG. 1 illustrates an example of visual communication with objects in an environment using visual alerts from light emitters on an autonomous vehicle. [Figure 18] FIG. 10 illustrates another example of a flow diagram for implementing visual alerts from light emitters in an autonomous vehicle. [Figure 19] FIG. 1 illustrates an example of visual communication with objects in an environment using visual alerts from light emitters on an autonomous vehicle. [Figure 20] FIG. 10 illustrates yet another example of a flow diagram for implementing visual alerts from light emitters in an autonomous vehicle. [Figure 21] FIG. 10 is a contour diagram of another example of a light emitter located external to an autonomous vehicle. [Figure 22] FIG. 10 is a contour diagram of yet another example of a light emitter located external to an autonomous vehicle. [Figure 23]FIG. 1 illustrates an example of an optical emitter for an autonomous vehicle. [Figure 24] FIG. 1 illustrates an example of data representing light patterns associated with light emitters of an autonomous vehicle. [Figure 25] FIG. 1 illustrates an example flow diagram for implementing a visual indication of directionality in an autonomous vehicle. [Figure 26] FIG. 1 illustrates an example of a flow diagram for implementing the visual display of information in an autonomous vehicle. [Figure 27] FIG. 1 illustrates an example of a visual indication of directional travel by light emitters of an autonomous vehicle. [Figure 28] 10A-10C illustrate other examples of visual indication of directionality of travel by light emitters of an autonomous vehicle. [Figure 29] FIG. 10 illustrates yet another example of a visual indication of directional travel by light emitters of an autonomous vehicle. [Figure 30] 10A-10C illustrate further examples of visual indication of directionality of travel by light emitters of an autonomous vehicle. [Figure 31] FIG. 1 illustrates an example of the visual display of information by light emitters of an autonomous vehicle. [Figure 32] FIG. 1 illustrates an example of a light emitter arrangement in an autonomous vehicle. [Figure 33] FIG. 1 illustrates an example of a light emitter arrangement in an autonomous vehicle.

[0006] While the above-described drawings illustrate various examples of the present invention, the present invention is not limited to the examples shown. It should be understood that like reference numerals refer to like structural elements in the drawings. It should also be understood that the drawings are not necessarily to scale. DETAILED DESCRIPTION OF THE INVENTION

[0007] Various embodiments or examples can be implemented in many ways, such as a system, a process, a method, an apparatus, a user interface, software, firmware, logic, a circuit, or a series of executable program instructions embodied in a non-transitory computer-readable medium. The program instructions can be transmitted over an optical, electronic, or wireless communications link, stored in, or otherwise fixed in, the non-transitory computer-readable medium. Examples of non-transitory computer-readable media include, but are not limited to, electronic memory, RAM, DRAM, SRAM, ROM, EEPROM, flash memory, solid-state memory, hard disk drives, and non-volatile memory. One or more non-transitory computer-readable media can be distributed across multiple devices. In general, the operations of a disclosed process can be performed in any order (unless otherwise provided in the claims).

[0008] A detailed description of one or more examples is provided below along with the accompanying figures. This detailed description is provided in connection with such examples, but is not limited to any particular example. The scope is limited only by the claims and encompasses many alternatives, modifications, and equivalents. In the following description, numerous specific details are set forth to provide a thorough understanding. These details are provided for the purpose of example, and the described technology can be practiced according to the claims without some or all of these specific details. For clarity, technical items known in the art related to the examples have not been described in detail to avoid unnecessarily obscuring the description.

[0009] Figure 1 illustrates an example of a system for implementing an active safety system in an autonomous vehicle. In Figure 1, autonomous vehicle 100 (shown in a top plan view) may be traveling along trajectory 105 in environment 190 external to autonomous vehicle 100. For purposes of illustration, environment 190 may include one or more objects that could potentially collide with autonomous vehicle 100, such as static and / or dynamic objects, or objects that pose some other hazard to occupants (not shown) riding within autonomous vehicle 100 and / or to autonomous vehicle 100. For example, in Figure 1, object 180 (e.g., a car) is shown as having trajectory 185 that, if not altered (e.g., by changing trajectory, slowing down, etc.), could result in potential collision 187 with autonomous vehicle 100 (e.g., by rear-ending autonomous vehicle 100).

[0010] Autonomous vehicle 100 may detect object 180 using a sensor system (not shown) to sense environment 190 (e.g., using passive and / or active sensors) and may take action to mitigate or prevent a potential collision of object 180 with autonomous vehicle 100. Autonomous vehicle system 101 may receive sensor data 132 from the sensor system and may receive autonomous vehicle location data 139 (e.g., implemented in a localizer system of autonomous vehicle 100). Sensor data 132 may include, but is not limited to, data representing sensor signals (e.g., signals generated by sensors of the sensor system). The data representing the sensor signals may be indicative of environment 190 external to autonomous vehicle 100. Autonomous vehicle location data 139 may include, but is not limited to, data representing the location of autonomous vehicle 100 in environment 190. As an example, data representing the location of autonomous vehicle 100 may include position and orientation data (e.g., local or regional position), map data (e.g., from one or more map tiles), data generated by a global positioning system (GPS), and data generated by an inertial measurement unit (IMU). In some examples, the sensor system of autonomous vehicle 100 may include a global positioning system, an inertial measurement unit, or both.

[0011] The autonomous vehicle system 101 may include, but is not limited to, hardware, software, firmware, logic, circuitry, computer-executable instructions embodied in a non-transitory computer-readable medium, or any combination of the foregoing, to implement the path calculator 112, the object data calculator 114 (e.g., implemented in the perception system of the autonomous vehicle 100), the collision predictor 116, the object classification determiner 118, and the kinematics calculator 115. The autonomous vehicle system 101 may have access to one or more data stores, including, but not limited to, an object type data store 119. The object type data store 119 may include data representing object types associated with object classifications for objects detected in the environment 190 (e.g., various pedestrian object types such as "sitting," "standing," or "running" can be associated with objects classified as pedestrians).

[0012] Path calculator 112 may be configured to generate data representing a trajectory (e.g., trajectory 105) of autonomous vehicle 100, for example, using data representing the location of autonomous vehicle 100 in environment 190 and other data (e.g., local position data included in vehicle location data 139). Path calculator 112 may be configured to generate a future trajectory to be executed by autonomous vehicle 100, for example. In some examples, path calculator 112 may be embedded within or as part of a planner system of autonomous vehicle 100. In other examples, path calculator 112 and / or the planner system may calculate data associated with the predicted movement of an object in the environment and determine a predicted object path associated with the predicted movement of the object. In some examples, the object path may constitute a predicted object path. In other examples, the object path may constitute a predicted object trajectory. In still other examples, the object path (e.g., in the environment) may constitute a predicted object trajectory, which may be the same as or similar to the predicted object trajectory.

[0013] The object data calculator 114 may be configured to calculate data representing the location of an object 180 disposed in the environment 190, data representing an object track associated with the object 180, data representing an object classification associated with the object 180, etc. The object data calculator 114 may calculate the data representing the object location, data representing the object track, and data representing the object classification using, for example, data representing a sensor signal included in the sensor data 132. In some examples, the object data calculator 114 is configured to receive data representing a sensor signal (e.g., a sensor signal from a sensor system) and may be implemented in or constitute a perception system or a portion thereof.

[0014] The object classification determinator 118 may be configured to access data representing an object type 119 (e.g., a type of object classification, a subclass of the object classification, or a subset of the object classification) and may be configured to compare the data representing the object truck and the data representing the object classification with the data representing the object type 119 to determine the data representing the object type (e.g., a type or subclass of the object classification). As an example, a detected object having an object classification of "car" may have an object type of "sedan," "coupe," "truck," or "school bus." Object types may include further subclasses or subsets, such as a parked "school bus" may have a further subclass of "static" (e.g., the school bus is not moving) or a further subclass of "dynamic" (e.g., the school bus is moving).

[0015] Collision predictor 116 can be configured to predict a collision (e.g., 187) between, for example, autonomous vehicle 100 and object 180 using data representing the object type, data representing the object's trajectory, and data representing the autonomous vehicle's trajectory.

[0016] Kinematics calculator 115 can be configured to calculate data representing one or more scalar and / or vector quantities associated with the movement of object 180 in environment 190, including, for example, but not limited to, velocity, speed, acceleration, deceleration, momentum, local position, and force. Data from kinematics calculator 115 can be used to calculate other data, including, for example, but not limited to, data representing an estimated time to impact between object 180 and autonomous vehicle 100, and data representing a distance between object 180 and autonomous vehicle 100. In some examples, kinematics calculator 115 can be configured to predict the likelihood that other objects in environment 190 (e.g., cars, pedestrians, bicycles, motorcycles, etc.) are alert or in control, or are unalert, out of control, or intoxicated. As an example, kinematics calculator 115 can be configured to estimate the probability that other agents (e.g., drivers or riders of other vehicles) are behaving rationally (e.g., based on the movement of objects they are driving or riding on), which may dictate the behavior of autonomous vehicle 100, or irrationally (e.g., based on erratic movement of objects they are riding on or driving). Rational or irrational behavior can be inferred based on sensor data received over time, which can be used to estimate or predict future locations of objects relative to the current or future trajectory of autonomous vehicle 100. Thus, the planner system of autonomous vehicle 100 can be configured, for example, to implement extra cautious vehicle maneuvers and / or activate safety systems of autonomous vehicle 100.

[0017] Safety system activator 120 may be configured to activate one or more safety systems of autonomous vehicle 100 when a collision is predicted by collision predictor 116 and / or upon the occurrence of other safety-related events (e.g., emergency maneuvers by vehicle 100, such as hard braking, sudden acceleration, etc.). Safety system activator 120 may be configured to activate internal safety system 122, external safety system 124, driving system 126 (e.g., causing driving system 126 to perform emergency maneuvers to avoid a collision), or any combination of the foregoing. For example, driving system 126 may receive data configured to cause a steering system (e.g., setting a steering angle or steering vector for the wheels) and a propulsion system (e.g., power supplied to an electric motor) to change the trajectory of vehicle 100 from trajectory 105 to collision-avoidance trajectory 105a.

[0018] 2A shows an example of a flow diagram 200 for implementing an active safety system in autonomous vehicle 100. In flow diagram 200, at stage 202, data may be received (e.g., implemented in a planner system of autonomous vehicle 100) representing a trajectory 203 of autonomous vehicle 100 in an environment external to autonomous vehicle 100 (e.g., environment 190).

[0019] At stage 204, object data associated with an object (e.g., automobile 180) located in an environment (e.g., environment 190) may be calculated. Sensor data 205 may be accessed at stage 204 to calculate the object data. The object data may include, but is not limited to, data representing an object location in the environment, an object track (e.g., static for a non-moving object and dynamic for a moving object) associated with the object, and an object classification (e.g., label) associated with the object (e.g., pedestrian, dog, cat, bicycle, motorcycle, car, truck, etc.). Stage 204 may output one or more types of data associated with the object, including, but not limited to, data representing object location 207 in the environment, data representing object track 209, and data representing object classification 211.

[0020] A predicted object path for an object in the environment can be calculated in stage 206. As an example, stage 206 can receive data representing object location 207 and process that data to generate data representing a predicted object path 213.

[0021] At stage 208, data representing object type 215 may be accessed, and at stage 210, data representing object type 217 may be determined based on the data representing object track 209, the data representing object classification 211, and the data representing object type 215. Examples of object types may include, but are not limited to, a pedestrian object type having a static object track (e.g., a pedestrian not moving), a vehicle object type having a dynamic object track (e.g., a vehicle moving), and an infrastructure object type having a static object track (e.g., a traffic sign, a lane marker, a fire hydrant), just to name a few. Stage 210 may output the data representing object type 217.

[0022] In stage 212, a collision between the autonomous vehicle and the object may be predicted based on the determined object type 217, the autonomous vehicle trajectory 203, and the predicted object path 213. As an example, a collision may be predicted based in part on the determined object type 217 due to objects having object tracks that are dynamic (e.g., the object is moving in the environment), object trajectories that are in potential conflict with the autonomous vehicle's trajectory (e.g., the trajectories may intersect or otherwise interfere with each other), and objects having object classifications 211 (e.g., used in calculating the object type 217) that indicate that the object is likely to be a collision threat (e.g., the object is classified as a car, skateboarder, bicyclist, motorcycle, etc.).

[0023] If a collision is predicted (e.g., in stage 212), safety systems of the autonomous vehicle may be activated in stage 214. Stage 214 may activate one or more safety systems of the autonomous vehicle, such as, for example, one or more interior safety systems, one or more exterior safety systems, one or more driving systems (e.g., steering, propulsion, braking, etc.), or a combination of the foregoing. Stage 214 may cause safety system activator 220 to activate one or more of the safety systems of autonomous vehicle 100 (e.g., by communicating data and / or signals).

[0024] 2B shows another example of a flowchart 250 for implementing an active safety system in autonomous vehicle 100. In flowchart 250, at stage 252, data may be received (e.g., from a planner system of autonomous vehicle 100) representing a trajectory 253 of autonomous vehicle 100 in an environment external to autonomous vehicle 100 (e.g., environment 190).

[0025] At stage 254, the location of the object in the environment may be determined. Sensor data 255 may be processed (e.g., by a perception system) to determine data representing the object location in environment 257. Data associated with the object in the environment (e.g., environment 190) (e.g., object data associated with object 180) may be determined at stage 254. The sensor data 255 accessed at stage 254 may be used to determine the object data. The object data may include, but is not limited to, data representing the object's location in the environment, an object track (e.g., static for a non-moving object and dynamic for a moving object) associated with the object, an object classification associated with the object (e.g., pedestrian, dog, cat, bicycle, motorcycle, car, truck, etc.), and an object type associated with the object. Stage 254 may output one or more types of data associated with the object, including, but not limited to, data representing the object location 257 in the environment, data representing an object track 261 associated with the object, data representing an object classification 263 associated with the object, and data representing an object type 259 associated with the object.

[0026] At stage 256, a predicted object path for an object in the environment may be calculated. As an example, stage 256 may receive data representing object location 257 and process the data to generate data representing predicted object path 265. In some examples, the data representing predicted object path 265 generated at stage 256 may be used as a data input at another stage of flowchart 250, such as stage 258. In other examples, stage 256 may be bypassed, and flowchart 250 may transition from stage 254 to stage 258.

[0027] In stage 258, a collision between the autonomous vehicle and the object may be predicted based on the autonomous vehicle trajectory 253 and the object location 265. The object location 257 may change from a first location to a next location due to the object's movement in the environment. For example, at different times, the object may be moving (e.g., having a dynamic object track of "D"), not moving (e.g., having a static object track of "S"), or both. However, the perception system may continuously track the object during those different times (e.g., using sensor data from the sensor system) to determine the object location 257 at those different times. Due to changes in the direction of the object's movement and / or the object's switching between moving and not moving states, the predicted object path 265 calculated in stage 256 may be difficult to determine, and therefore, the predicted object path 265 need not be used as a data input in stage 258.

[0028] 2B , such as object type 259 and predicted object path 265. As a first example, stage 258 can predict a collision between an autonomous vehicle and an object based on autonomous vehicle trajectory 253, object location 257, and object type 259. As a second example, stage 258 can predict a collision between an autonomous vehicle and an object based on autonomous vehicle trajectory 253 and predicted object path 265. As a third example, stage 258 can predict a collision between an autonomous vehicle and an object based on autonomous vehicle trajectory 253, predicted object path 265, and object type 259.

[0029] If a collision is predicted (e.g., in stage 258), safety systems of the autonomous vehicle may be activated in stage 260. Stage 260 may activate one or more safety systems of the autonomous vehicle, such as, for example, one or more interior safety systems, one or more exterior safety systems, one or more driving systems (e.g., steering, propulsion, braking, etc.), or a combination of the foregoing. Stage 260 may cause a safety system activator 269 to activate one or more of the safety systems of the autonomous vehicle (e.g., by communicating data and / or signals).

[0030] 2C shows yet another example of a flow diagram 270 for implementing an active safety system in an autonomous vehicle. At stage 272, data may be received (e.g., from a planner system of autonomous vehicle 100) representing a trajectory 273 of autonomous vehicle 100 in an environment external to autonomous vehicle 100 (e.g., environment 190).

[0031] At stage 274, the location of the object in the environment can be determined (e.g., by a perception system) using, for example, sensor data 275. Stage 274 can generate data representing object location 279. The data representing object location 279 can include data representing a predicted progress of the object's movement 281 relative to the object's location in the environment. For example, if the object has a static object track indicating no movement in the environment, the predicted progress of movement 281 may be zero. However, if the object's object track is dynamic and the object classification is a car, the predicted progress of movement 281 may be non-zero.

[0032] In stage 276, a predicted next location of the object in the environment can be calculated based on the predicted progress of the movement 281. Stage 276 can generate data representing the predicted next location 283.

[0033] In stage 278, a probability of impact between the object and the autonomous vehicle can be predicted based on predicted next location 283 and autonomous vehicle trajectory 273. Stage 278 can generate data representing probability of impact 285.

[0034] At stage 280, a subset of thresholds (e.g., locations or distances in the environment) for activating various escalating functions of a subset of the autonomous vehicle's safety systems may be calculated based on the probability of impact 285. At least one subset of thresholds is associated with activating various escalating functions of the autonomous vehicle's safety systems. Stage 280 may generate data representing one or more threshold subsets 287. In some examples, the subset of thresholds may constitute a location relative to the autonomous vehicle or may constitute a distance relative to the autonomous vehicle. For example, the threshold may be a function of a location or set of locations relative to a reference location (e.g., the autonomous vehicle). Additionally, the threshold may be a function of the distance relative to the object and the autonomous vehicle, or the distance between any objects or object locations, including the distance between predicted object locations.

[0035] At stage 282, one or more of various escalating features of the safety system may be activated based on the associated predicted probabilities (e.g., activating a bladder based on a predicted set of impact probabilities indicating an impending collision). Stage 282 may cause a safety system activator 289 (e.g., by communicating data and / or signals) to activate one or more of the autonomous vehicle's safety systems based on the corresponding set or sets of collision probabilities.

[0036] 3A illustrates an example system 300 for implementing an active safety system in an autonomous vehicle. In FIG. 3A, an autonomous vehicle system 301 can include a sensor system 320, which includes sensors 328 configured to sense an environment 390 (e.g., in real time or near real time) and generate (e.g., in real time) sensor data 332 and 334 (e.g., data representing sensor signals).

[0037] The autonomous vehicle system 301 may include a perception system 340 configured to detect objects in the environment 390, determine object tracks for the objects, classify the objects, track the location of objects in the environment 390, and detect particular types of objects in the environment 390, such as traffic signs / lights, road markings, lane markings, etc. The perception system 340 may receive sensor data 334 from the sensor system 320.

[0038] The autonomous vehicle system 301 may include a localizer system 330 configured to determine a location of the autonomous vehicle in the environment 390. The localizer system 330 may receive sensor data 332 from the sensor system 320. In some examples, the sensor data 332 received by the localizer system 330 may not be the same as the sensor data 334 received by the perception system 340. For example, the perception system 330 may receive data 334 from sensors including, but not limited to, LIDAR (e.g., 2D, 3D, color LIDAR), RADAR, and cameras (e.g., image capture devices), while the localizer system 330 may receive data 332 including, but not limited to, global positioning system (GPS) data, inertial measurement unit (IMU) data, map data, route data, route network definition file (RNDF) data, and map tile data. Localizer system 330 can receive data from sources other than sensor system 320, such as a data store, data repository, memory, etc. In other examples, sensor data 332 received by localizer system 330 may be the same as sensor data 334 received by perception system 340. In various examples, localizer system 330 and perception system 340 may or may not implement similar or equivalent sensors or similar or equivalent types of sensors. Furthermore, localizer system 330 and perception system 340 can each implement any type of sensor data 332 independently of each other.

[0039] Perception system 340 can process sensor data 334 to generate object data 349, which can be received by planner system 310. Object data 349 can include data associated with objects detected in environment 390, including, but not limited to, data representing, for example, object classification, object type, object track, object location, predicted object path, predicted object trajectory, and object velocity.

[0040] Localizer system 330 can process sensor data 334, and optionally other data, to generate position and orientation data, local position data 339, which can be received by planner system 310. Local position data 339 can include, for example, but is not limited to, data representing the location of the autonomous vehicle in environment 390, GPS data, IMU data, map data, route data, route network definition file (RNDF) data, odometry data, wheel encoder data, and map tile data.

[0041] Planner system 310 may process object data 349 and local position data 339 to calculate a path (e.g., a trajectory of the autonomous vehicle) for the autonomous vehicle through environment 390. The calculated path is determined in part by objects in environment 390 that may, for example, pose obstacles to the autonomous vehicle and / or pose a collision threat to the autonomous vehicle.

[0042] Planner system 310 can be configured to communicate control and data 317 with one or more vehicle controllers 350. Control and data 317 can include information configured to control driving operations (e.g., steering, braking, propulsion, signaling, etc.) of the autonomous vehicle via driving systems 326, activate one or more interior safety systems 322 of the autonomous vehicle, and activate one or more exterior safety systems 324 of the autonomous vehicle. Driving systems 326 can perform additional functions associated with the active safety of the autonomous vehicle, such as, for example, collision avoidance maneuvers.

[0043] Vehicle controller 350 can be configured to receive control and data 317 and, based on control and data 317, communicate internal data 323, external data 325, and driving data 327 to internal safety system 322, external safety system 324, and driving system 326, respectively, for example, as determined by control and data 317. As an example, if planner system 310 determines, based on some action of an object in environment 390, that internal safety system 322 should be activated, control and data 317 can include information configured to cause vehicle controller 350 to generate internal data 323 for activating one or more functions of internal safety system 322.

[0044] Autonomous vehicle system 301 and its associated systems 310, 320, 330, 340, 350, 322, 324, and 326 can be configured to access data 315 from data store 311 (e.g., a data repository) and / or data 312 from external resources 313 (e.g., the cloud, the internet, a wireless network). Autonomous vehicle system 301 and its associated systems 310, 320, 330, 340, 350, 322, 324, and 326 can be configured to access data in real time from various systems and / or data sources, including but not limited to those shown in FIG. 3A . As an example, localizer system 330 and perception system 340 can be configured to access sensor data 332 and sensor data 334 in real time. As another example, planner system 310 can be configured to access object data 349, local position data 339, and control and data 317 in real time. In other examples, the planner system 310 can be configured to access the data store 311 and / or external resources 313 in real time.

[0045] 3B shows another example system 399 for implementing an active safety system in an autonomous vehicle. In example 399, sensors 328 in sensor system 320 can include, for example, but are not limited to, one or more of light detection and ranging sensors 371 (lidar), image capture sensors 373 (e.g., cameras), radio detection and ranging sensors 375 (radar), sound capture sensors 377 (e.g., microphones), global positioning system sensors (GPS), and / or inertial measurement unit sensors (IMU) 379, and environmental sensors 372 (e.g., temperature, barometer pressure). Localizer system 330 and perception system 340 can receive sensor data 332 and / or sensor data 334, respectively, from one or more of sensors 328. For example, perception system 340 may receive sensor data 334 related to determining information associated with objects in environment 390, such as sensor data from lidar 371, camera 373, radar 375, environment 372, and microphone 377, while localizer system 330 may receive sensor data 332 associated with the location of the autonomous vehicle in environment 390, such as from GPS / IMU 379. Additionally, localizer system 330 may receive data from sources other than sensor system 320, such as, for example, map data, map tile data, route data, route network definition file (RNDF) data, data stores, data repositories, etc. In some examples, the sensor data (332, 334) received by localizer system 330 may be the same as the sensor data (332, 334) received by perception system 340. In other examples, the sensor data (332, 334) received by the localizer system 330 may not be the same as the sensor data (332, 334) received by the perception system 340. The sensor data 332 and 334 may each include data from one or more sensors or any combination of sensor types in the sensor system 320.The quantity and type of sensor data 332 and 334 may be independent of one another and may be similar or equivalent, or may not be similar or equivalent.

[0046] As an example, localizer system 330 may receive and / or access data from sources other than sensor data (332, 334), such as odometry data 336 from motion sensors to estimate changes in position of autonomous vehicle 100 over time, wheel encoders 337 to calculate movement, distance, and other metrics of autonomous vehicle 100 based on wheel rotations (e.g., by propulsion system 368), map data 335 from data representing map tiles, route data, route network definition file (RNDF) data, and / or other data, and data representing an autonomous vehicle (AV) model 338, which may be used to calculate vehicle location data based on a model of vehicle dynamics (e.g., from a simulation, captured data, etc.) of autonomous vehicle 100. Localizer system 330 may use one or more of the depicted data resources to generate data representing local position data 339.

[0047] As another example, the perception system 340 may parse or otherwise analyze, process, or manipulate the sensor data (332, 334) to perform object detection 341, object tracking 343 (e.g., determining which detected objects are static (not moving) and which are dynamic (moving)), object classification 345 (e.g., cars, motorcycles, bicycles, pedestrians, skateboarders, mailboxes, buildings, streetlights, etc.), object tracking 347 (e.g., tracking objects based on changes in the object's location in the environment 390), and traffic light / sign detection 342 (e.g., stop lights, stop signs, railroad crossings, lane markers, crosswalks, etc.).

[0048] As yet another example, planner system 310 can receive local position data 339 and object data 349 and can parse or otherwise analyze, process, or manipulate the data (local position data 339, object data 349) to perform functions including, but not limited to, trajectory calculation 381, threshold location estimation 386, audio signal selection 389, light pattern selection 382, ​​kinematics calculation 384, object type detection 387, collision prediction 385, and object data calculation 383. Planner system 310 can communicate trajectory and control data 317 to vehicle controller 350. Vehicle controller 350 can process vehicle control and data 317 to generate driving system data 327, interior safety system data 323, and exterior safety system data 325. Driving system data 327 can be communicated to driving system 326. Driving system 326 may communicate driving system data 327 to braking system 364, steering system 366, propulsion system 368, and signal system 362 (e.g., turn signals, brake signals, headlights, and running lights). For example, driving system data 327 may include steering angle data (e.g., steering angle relative to the wheels) for steering system 366, braking data (e.g., braking force to be applied to brake pads) for brake system 364, and propulsion data (e.g., voltage, current, or power to be applied to the motor) for propulsion system 368. Dashed line 377 may represent a boundary between the vehicle trajectory processing layer and the vehicle physics execution layer, where data processed in the vehicle trajectory processing layer is implemented by one or more of driving system 326, interior safety system 322, or exterior safety system 324. As an example, one or more portions of the interior safety system 322 may be configured to enhance the safety of occupants in the autonomous vehicle 100 in the event of a collision and / or other extreme event (e.g., a collision avoidance maneuver by the autonomous vehicle 100).As another example, one or more portions of the external safety system 324 may be configured to reduce the impact force or negative effects of the collisions and / or extreme events described above.

[0049] Interior safety systems 322 can include systems including, but not limited to, a seat actuator system 363 and a seat belt tensioning system 361. Exterior safety systems 324 can include systems including, but not limited to, an acoustic array system 365, an optical emitter system 367, and a bladder system 369. Driving systems 326 can include systems including, but not limited to, a braking system 364, a signal system 362, a steering system 366, and a propulsion system 368. The systems in external safety system 324 can be configured to interface with environment 390 by emitting light into environment 390 using one or more optical emitters (not shown) in optical emitter system 367, emitting a directed beam of acoustic energy (e.g., sound) into environment 390 using one or more acoustic beam steering arrays (not shown) in acoustic beam steering array 365, or by inflating one or more bladders (not shown) in bladder system 369 from an undeployed position to a deployed position, or any combination of the foregoing. Additionally, acoustic beam steering array 365 can radiate acoustic energy into the environment using, for example, a transducer, air horn, or resonator. The acoustic energy can be omnidirectional or can constitute a directed beam or otherwise focused sound (e.g., a directional acoustic source of an ultrasound source, a phased array, a parametric array, a large radiator). Thus, the systems in external safety system 324 may be located at one or more locations on autonomous vehicle 100 that are configured to allow those systems to interface with environment 390, such as a location associated with an exterior surface of autonomous vehicle 100 (e.g., 100e in FIG. 1 ).The systems in interior safety systems 322 may be located at one or more locations associated with the interior of autonomous vehicle 100 (e.g., 100i in FIG. 1 ) and may be connected to one or more structures of autonomous vehicle 100, such as a seat, a bench seat, a floor, a rail, a bracket, a support, or other structure. Seat belt tensioning system 361 and seat actuator system 363 may be coupled to one or more structures configured to support mechanical loads that may result from, for example, a collision, vehicle acceleration, vehicle deceleration, evasive maneuvers, sharp turns, hard braking, etc.

[0050] FIG. 4 illustrates an example flow diagram 400 for implementing a perception system in an autonomous vehicle. In FIG. 4, sensor data 434 received by perception system 440 (e.g., generated by one or more sensors in sensor system 420) is visually illustrated as sensor data 434a-434c (e.g., LIDAR data, color LIDAR data, 3D LIDAR data). At stage 402, a determination may be made as to whether the sensor data 434 includes data representative of a detected object. If the NO branch is taken, the flow diagram 400 may return to stage 402 to continue analyzing the sensor data 434 to detect objects in the environment. If the YES branch is taken, the flow diagram 400 may continue to stage 404, where a determination may be made as to whether the data representative of the detected object includes data representative of a traffic sign or traffic light. If the YES branch is taken, flowchart 400 may transition to stage 406, where data representing the detected object may be analyzed to classify the type of detected signal / signal object, such as, for example, a traffic signal (e.g., red, yellow, and green) or a stop sign (e.g., based on shape, character, and color). The analysis in stage 406 may include accessing a traffic object data store 424, where examples of data representing traffic classifications may be compared with the data representing the detected object to generate data representing traffic classifications 407. Stage 406 may then transition to another stage, such as stage 412. In some examples, stage 404 may be optional, and stage 402 may transition to stage 408.

[0051] If the NO branch is taken from stage 404, the flowchart 400 may transition to stage 408, where data representing the detected object may be analyzed to determine other object types to be classified. If the YES branch is taken, the flowchart 400 may transition to stage 410, where data representing the detected object may be analyzed to classify the object's type. An object data store 426 may be accessed to compare stored examples of data representing the object classification with the data representing the detected object to generate data 411 representing the object classification. Stage 410 may then transition to another stage, such as stage 412. If the NO branch is taken from stage 408, stage 408 may transition to another stage, such as back to stage 402.

[0052] At stage 412, the object data classified at stages 406 and / or 410 may be analyzed to determine whether the sensor data 434 indicates movement associated with the data representing the detected object. If movement is not indicated, the NO branch may be taken to stage 414, where the data representing the object track for the detected object may be set to static (S). At stage 416, data representing the location of the object (e.g., a static object) may be tracked. For example, a stationary object detected at time t may move at a later time t, becoming a dynamic object. Furthermore, the data representing the location of the object may be included in data received by a planner system (e.g., planner system 310 in FIG. 3B ). The planner system may use the data representing the object's location to determine the object's coordinates (e.g., coordinates relative to the autonomous vehicle 100).

[0053] On the other hand, if the detected object indicates movement, the YES branch can be taken to stage 418, where data representing the object track for the detected object can be set to dynamic (D). At stage 419, data representing the location of the object (e.g., a dynamic object) can be tracked. The planner system can analyze the data representing the object track and / or the data representing the object location to determine whether the detected object (static or dynamic) may potentially have a conflicting trajectory with respect to the autonomous vehicle and / or may be coming too close to the vicinity of the autonomous vehicle, whereby an alert (e.g., from an optical emitter and / or from an acoustic beam steering array) can be used to modify the behavior of the object and / or a person controlling the object.

[0054] At stage 422, one or more of data representing an object classification, data representing an object track, and data representing an object location may be included with the object data 449 (e.g., object data received by the planner system). As an example, the sensor data 434a may include data representing an object (e.g., a person riding a skateboard). Stage 402 may detect the object in the sensor data 434a. At stage 404, it may be determined that the detected object is not a traffic sign / traffic light. Stage 408 may determine that the detected object is of another class, and at stage 410, based on data accessed from the object data store 426, may analyze the data representing the object to determine that the classification matches a person riding a skateboard and output data representing the object classification 411. At stage 412, a determination may be made that the detected object is moving, at stage 418 the object track may be set to dynamic (D), and the location of the object may be tracked (e.g., by continuing to analyze the sensor data 434a for changes in the location of the detected object) at stage 419. At stage 422, the object data associated with the sensor data 434 may include, for example, a classification (e.g., a person riding a skateboard), an object track (e.g., the object is moving), a location of the object (e.g., a skateboarder) in an environment external to the autonomous vehicle, and object tracking data.

[0055] Similarly, with respect to sensor data 434b, flowchart 400 can determine, for example, that the object classification is a pedestrian, that the pedestrian is moving (e.g., walking), and has a dynamic object track, and can track the location of the object (e.g., the pedestrian) in the environment. Finally, with respect to sensor data 434c, flowchart 400 can determine, for example, that the object classification is a fire hydrant, that the hydrant is not moving, and has a static object track, and can track the location of the hydrant. Note that in some examples, object data 449 associated with sensor data 434a, 434b, and 434c can be further processed by the planner system based on factors including, for example, but not limited to, the object track, the object classification, and the object location. As an example, in the case of a skateboarder and a pedestrian, object data 449 can be used for one or more of trajectory calculation, threshold location estimation, motion prediction, location comparison, and object coordinates in the event the planner system determines to implement an alert (e.g., by an external safety system) for the skateboarder and / or pedestrian. However, the planner system may decide to ignore object data regarding the hydrant due to its static object track because, for example, the hydrant is unlikely to have motion that would bring it into conflict with the autonomous vehicle (e.g., it does not move) and / or the hydrant is not alive (e.g., it cannot respond to or notice alerts such as emitted lights and / or beam-guided sounds generated by the autonomous vehicle's external safety systems).

[0056] 5 illustrates an example 500 of object prioritization by a planner system in an autonomous vehicle. Example 500 shows a visualization of an environment 590 external to autonomous vehicle 100 as sensed by a sensor system (e.g., sensor system 320 of FIG. 3B ) of autonomous vehicle 100. Object data from the perception system of autonomous vehicle 100 can detect several objects in environment 590, including, but not limited to, a car 581 d, a bicycle rider 583 d, a walking pedestrian 585 d, and two parked cars 587 s and 589 s. In this example, the perception system can have assigned dynamic “D” object tracks to objects 581 d, 583 d, and 585 d (e.g., based on sensor data 334 of FIG. 3B ), and thus the label “d” is associated with a reference number for those objects. The perception system may also assign static "S" object tracks to objects 587s and 589s (e.g., based on sensor data 334 of FIG. 3B), and thus the label "s" is associated with a reference number for those objects.

[0057] A localizer system of the autonomous vehicle can determine local position data 539 (e.g., X, Y, Z coordinates or other metric or coordinate system relative to the location of the vehicle 100) regarding the location of the autonomous vehicle 100 in the environment 590. In some examples, the local position data 539 can be associated with the center of gravity (not shown) of the autonomous vehicle 100 or other reference point. Additionally, the autonomous vehicle 100 can have a trajectory Tav, as indicated by the arrows. Two parked automobiles 587s and 589s are static and do not have trajectories shown. Bicycle rider 583d has a trajectory Tb that is generally opposite the direction of trajectory Tav, and automobile 581d has a trajectory Tmv that is generally parallel to and in the same direction as trajectory Tav. Pedestrian 585d has a trajectory Tp that is predicted to intersect with the trajectory Tav of vehicle 100. The movement and / or position of the pedestrian 585d in the environment 590 or other objects in the environment 590 can be tracked or otherwise determined using metrics other than trajectory, including, for example, but not limited to, object location, predicted object movement, object coordinates, predicted progress of movement relative to the object's location, and predicted next location of the object. The movement and / or position of the pedestrian 585d in the environment 590 or other objects in the environment 590 can be determined at least in part due to probability. The probability can be based on data representing, for example, object classification, object track, object location, and object type. In some examples, the probability can be based on previously observed data regarding similar objects in similar locations. Furthermore, the probability can be affected by time of day, day of the week, or other time units. As an example, a planner system can know that there is an 85% chance that pedestrians at a given intersection will cross the street in a particular direction between approximately 3:00 PM and 4:00 PM on a weekday.

[0058] The planner system may place a lower priority on tracking the locations of static objects 587s and 589s and dynamic object 583d because static objects 587s and 589s are positioned off trajectory Tav (e.g., objects 587s and 589s are parked) and dynamic object 583d (e.g., an object identified as a bicyclist) is moving in a direction away from autonomous vehicle 100, thereby reducing or eliminating the possibility that trajectory Tb of object 583d may conflict with trajectory Tav of autonomous vehicle 100.

[0059] However, the planner system may place a higher priority on tracking the location of pedestrian 585d due to its potentially conflicting trajectory Tp, and a slightly lower priority on tracking the location of automobile 581d, because its trajectory Tmv does not currently conflict with trajectory Tav, but may conflict at a later time (e.g., due to a lane change or other vehicle maneuver). Thus, based on example 500, pedestrian object 585d may be a good candidate for an alert (e.g., using guided sounds and / or emitted lights) or other safety system of autonomous vehicle 100 because the path of pedestrian object 585d (e.g., based on its location and / or predicted movement) may result in a potential collision with autonomous vehicle 100 (e.g., at estimated location 560) or an unsafe distance between pedestrian object 585d and autonomous vehicle 100 (e.g., at some future time and / or location). The priority placed by the planner system on tracking the location of objects may be determined based at least in part on a cost function of trajectory generation in the planner system. Objects that are likely to be predicted to require a change in the trajectory of autonomous vehicle 100 (e.g., to avoid a collision or other extreme event) may be factored into the cost function with greater importance compared to objects that are predicted not to require a change in the trajectory of autonomous vehicle 100, for example.

[0060] The planner system may predict one or more regions 565 of likely locations of the object 585d in the environment 590 based on the predicted movement of the object 585d and / or the predicted location of the object. The planner system may estimate one or more threshold locations (e.g., threshold boundaries) within each region 565 of likely locations. The threshold locations may be associated with one or more safety systems of the autonomous vehicle 100. The number, distance, and location of the threshold locations may vary for different safety systems of the autonomous vehicle 100. The safety systems of the autonomous vehicle 100 may be activated in a parameter range in which a collision between the object and the autonomous vehicle 100 is predicted. The parameter range may have a location (e.g., within 565) within the region of likely locations. The parameter range may be based in part on parameters such as a range of time and / or a range of distance. For example, a range of time and / or a range of distance in which a predicted collision between the object 585d and the autonomous vehicle 100 may occur. In some examples, being a likely candidate for an alert by the safety system of autonomous vehicle 100 (e.g., as determined by the planner system) does not automatically result in an actual alert being issued. In some examples, an object can become a candidate if a threshold is met or exceeded. In other examples, the safety system of autonomous vehicle 100 can issue multiple alerts to one or more objects in the environment external to autonomous vehicle 100. In still other examples, autonomous vehicle 100 may not issue an alert even if the object is determined to be a likely candidate for an alert (e.g., the planner system could have calculated an alternative trajectory, safe stop trajectory, or safe stop maneuver, which would obviate the need to issue an alert).

[0061] FIG. 6 shows a top plan view of an example 600 of threshold locations and associated escalating alerts in an active safety system in an autonomous vehicle. In FIG. 6, the example 500 of FIG. 5 is further shown in a top plan view in which trajectories Tav and Tp are estimated to intersect (e.g., based on location data for vehicle 100 and pedestrian object 585d) at an estimated location shown as 560. Pedestrian object 585d is shown in FIG. 6 based on being a likely candidate for an alert (e.g., visual and / or acoustic) based on its predicted movement. Pedestrian object 585d is shown as having a location within region 565 of likely locations estimated by the planner system. Autonomous vehicle 100 may be configured to travel bidirectionally, as shown by arrow 680; that is, autonomous vehicle 100 may not have a front (e.g., hood) or rear (e.g., trunk) as in a traditional automobile. 6 and other figures may describe an embodiment as applied to a two-way vehicle, it should be noted that the functionality and / or structure need not be so limited and can be applied to any vehicle, including a one-way vehicle. Thus, the sensors and safety systems of the sensor system can be arranged on the vehicle 100 to provide sensor and alert coverage in multiple directions of travel of the vehicle 100 and / or to provide sensor and alert coverage for objects approaching the vehicle 100 from its side 1005 (e.g., one or more sensors 328 in the sensor system 330 in FIG. 3B can include overlapping areas of sensor coverage). The planner system can, for example, use the two-way travel capability 680 to implement a safety stop maneuver or a safety stop trajectory to avoid a collision with an object.As an example, autonomous vehicle 100 may be traveling in a first direction, stop, and begin traveling in a second direction opposite the first direction, maneuvering to a safe stopping location to avoid a collision or other extreme event. The planner system may estimate one or more threshold locations (e.g., threshold boundaries, which may be a function of distance, etc.) in environment 590, shown as 601, 603, and 605, and issue an alert if the location of an object (e.g., pedestrian object 585d) at the threshold location matches the threshold location, as shown by points of match 602, 604, and 606. Although three threshold locations are shown, there may be more or fewer threshold locations than shown. As a first example, when the trajectory Tp intersects the first threshold location 601 at the point shown as 602, the planner system can determine the coordinates of the location of the pedestrian object 585d at point 602 (e.g., having coordinates X1, Y1 in a relative or absolute frame of reference) and the location of the autonomous vehicle 100 (e.g., from local position data).The location data regarding the autonomous vehicle 100 and the object (e.g., object 585d) may be used to calculate a location (e.g., coordinates, angles, polar coordinates) regarding the direction of propagation (e.g., direction of the main lobe or focus of the beam) of directed acoustic energy (e.g., audio alert) emitted by an acoustic beam steering array (e.g., one or more of a directional acoustic source, a phased array, a parametric array, a large radiator, an ultrasonic source, etc.), may be used to determine which optical emitters to activate for a visual alert, may be used to determine which bladders to activate prior to a predicted collision with the object, and may be used to activate other safety and / or driving systems of the autonomous vehicle 100, or any combination of the foregoing. As autonomous vehicle 100 continues traveling along trajectory Tav in direction 625 from location L1 to location L2, the relative locations of pedestrian object 585d and autonomous vehicle 100 may change, such that at location L2, object 585d has coordinates (X2, Y2) at point 604 of second threshold location 603. Similarly, as it continues traveling along trajectory Tav in direction 625 from location L2 to location L3, the relative locations of pedestrian object 585d and autonomous vehicle 100 may change, such that at location L3, object 585d has coordinates (X3, Y3) at point 606 of third threshold location 605.

[0062] As the distance between the autonomous vehicle 100 and the pedestrian object 585d decreases, the data representing the alert selected for the safety system may be different to convey an increasing sense of urgency (e.g., an escalating threat level) to the pedestrian object 585d to change or stop its trajectory Tp, or otherwise modify its / her behavior to avoid a potential collision or close pass with the vehicle 100. As an example, if the vehicle 100 may be at a relatively safe distance from the pedestrian object 585d, the data representing the alert selected for the threshold location 601 may be a less startling, non-threatening alert a1 configured to attract the pedestrian object 585d's attention in a non-threatening manner. As a second example, if the vehicle 100 may be at a cautionary distance from the pedestrian object 585d, the data representing the alert selected for the threshold location 603 may be a more aggressive, urgent alert a2 configured to attract the pedestrian object 585d's attention in a more urgent manner. As a third example, if vehicle 100 may be at a potentially unsafe distance from pedestrian object 585d, the data representing the alert selected for threshold location 605 may be a very aggressive, highly urgent alert a3 configured to get the pedestrian object 585d's attention in a highly urgent manner. As the distance between autonomous vehicle 100 and pedestrian object 585d decreases, the data representing the alert may be configured to communicate escalating increases in urgency to pedestrian object 585d (e.g., escalating audio and / or visual alerts to get the pedestrian object 585d's attention). An estimate of the position of the threshold location in environment 590 may be determined by the planner system to provide an appropriate amount of time (e.g., approximately 5 seconds or more) based on the speed of the autonomous vehicle before vehicle 100 reaches a predicted impact point 560 with pedestrian object 585d (e.g., a point 560 in environment 590 where trajectories Tav and Tp are estimated to intersect one another).Point 560 may change as the speed and / or location of object 585d, vehicle 100, or both change. In some examples, the planner system of autonomous vehicle 100 can be configured to actively attempt to avoid potential collisions by (e.g., continuously) calculating safe trajectories (e.g., by calculating alternative collision-avoidance trajectories and executing one or more of those trajectories) (e.g., when possible based on the context) in conjunction with implementing active alerts (e.g., audible and / or visual alerts), while simultaneously issuing alerts as needed to objects in the environment when there is a significant probability that those objects may collide or otherwise pose a danger to the safety of occupants in autonomous vehicle 100, to the object, or both.

[0063] 6 , the trajectory Tp of pedestrian object 585d need not be a straight line as shown; its trajectory (e.g., actual trajectory) can be an arcuate trajectory as shown in example 650, or a non-linear trajectory as shown in example 670. Points 652, 654, and 656 in example 650 and points 672, 674, and 676 in example 670 indicate points where trajectory Tp intersects threshold locations 651, 653, and 655, and threshold locations 671, 673, and 675, respectively. The planner system can process object data from the perception system and local position data from the localizer system to calculate the threshold locations. The location, shape (e.g., linear, arcuate, non-linear), orientation (e.g., relative to the autonomous vehicle and / or object), and other characteristics of the threshold locations can depend on the application and are not limited to the examples shown herein. For example, in FIG. 6 , the threshold locations (601, 603, and 605) in example 600 are aligned approximately perpendicular to the trajectory Tp of pedestrian object 585d, although other configurations and orientations can be calculated and implemented by the planner system (see, for example, examples 650 and 670). As another example, the threshold locations can be aligned approximately perpendicular (or at other orientations) to the trajectory Tav of autonomous vehicle 100. Additionally, the trajectory of the object can be analyzed by the planner system to determine the configuration of the threshold locations. As an example, if the object is on a trajectory that is parallel to the trajectory of autonomous vehicle 100, the threshold locations can have an arcuate profile and can be aligned with the trajectory of the object.

[0064] FIG. 7 shows an example flow diagram 700 for implementing a planner system in an autonomous vehicle. In FIG. 7, planner system 710 may be in communication with perception system 740, from which it receives object data 749, and localizer system 730, from which it receives local position data 739. Object data 749 may include data associated with one or more detected objects. For example, in a typical scenario, object data 749 may include data associated with multiple detected objects in an environment external to autonomous vehicle 100. However, some detected objects may not need to be tracked or may be assigned a lower priority based on the type of object. For example, fire hydrant object 434c in FIG. 4 may be said to be a static object (e.g., fixed to the ground) and may not require processing for alerting or activating other safety systems, while skateboarder object 434a may require processing for an alert (e.g., using an emitted light, a guided sound, or both) due to it being a dynamic object and other factors, such as the expected human behavior of a skateboard rider, the location of skateboarder object 434a in the environment, which may indicate that the location (e.g., the predicted location) may conflict with the trajectory of autonomous vehicle 100, etc.

[0065] 7, three example detected objects are shown as 734a-734c. As shown by 734, there may be object data 749 for more or fewer detected objects. The object data for each detected object may include, but is not limited to, data representing an object location 721, an object classification 723, and an object track 725. The planner system 710 may be configured to perform an object type determination 731. The object type determination 731 may be configured to receive data representing the object classification 723, data representing the object track 725, and data representing the object type, which may be accessed from an object type data store 724. The object type determination 731 may be configured to compare the data representing the object classification 723 and the data representing the object track 725 with data representing the object type (e.g., accessed from the data store 724) to determine data representing the object type 733. Examples of data representing object type 733 may include, but are not limited to, static grounded object types (e.g., fire hydrant 434c in FIG. 4) and dynamic pedestrian objects (e.g., pedestrian object 585d in FIG. 5).

[0066] An object behavior determination 735 may be configured to receive data representing an object type 733 and data representing an object location 721. The object behavior determination 735 may be further configured to access an object behavior data store 726. The object behavior data store 726 may include data representing object behavior. The object behavior determination 735 may be configured to compare the data representing the object behavior with the data representing the object type 733 and the data representing the object location 721 to determine data representing a predicted movement 737 of the object.

[0067] An object location predictor 741 may be configured to receive data representing a predicted movement 737 of an object, data representing a location 743 of the autonomous vehicle (e.g., from local position data 739), data representing an object location 721, and data representing an object track 725. The object location predictor 741 may be configured to process the received data to generate data representing a predicted object location 745 in the environment. The object location predictor 741 may be configured to generate data representing multiple predicted object locations 745. The planner system 710 may generate data representing regions of likely locations of the object (e.g., 565 in FIGS. 5 and 6 ).

[0068] A threshold location estimator 747 may be configured to receive data representing the autonomous vehicle's location 743 (e.g., from local position data 739) and data representing predicted object locations 745, and to generate data representing one or more threshold locations 750 in the environment that are associated, for example, with an alert (e.g., a visual and / or an audible alert) to be triggered or with the activation of one or more safety systems of vehicle 100. One or more threshold locations 750 may be located with an area of ​​likely location (e.g., 601, 603, and 605 within 565 in FIG. 6 ). FIG. 8 shows an example block diagram 800 of a system in an autonomous vehicle. In FIG. 8, autonomous vehicle 100 may include a suite of sensors 820 located at one or more locations on autonomous vehicle 100. Each suite 820 may have sensors including, but not limited to, a lidar 821 (e.g., color lidar, 3D color lidar, 2D lidar, etc.), an image capture device 823 (e.g., a digital camera), a radar 825, a microphone 827 (e.g., for capturing ambient sounds), and a loudspeaker 829 (e.g., for greeting / communicating with occupants of AV100). A microphone 871 (e.g., which may be configured to capture sounds from driving system components such as the propulsion system and / or braking system) may be positioned in an appropriate location near the driving system components to, for example, detect sounds generated by those components. Each suite of sensors 820 may include multiple sensors of the same type, such as, for example, two image capture devices 823, a microphone 871 positioned near each wheel 852, etc. The microphone 871 may be configured to capture audio signals indicative of the driving maneuvers of the autonomous vehicle 100.Microphone 827 may be configured to capture audio signals indicative of ambient sounds in the environment external to autonomous vehicle 100. Multiple microphones 827 may be paired or otherwise grouped to generate signals that may be processed to estimate source locations of sounds occurring in environment 890, for example. Autonomous vehicle 100 may include sensors for generating data indicative of the location of autonomous vehicle 100, which may include, but are not limited to, a global positioning system (GPS) 839a and / or an inertial measurement unit (IMU) 839b. Autonomous vehicle 100 may include one or more sensors ENV 877 for sensing environmental conditions in the environment external to autonomous vehicle 100, such as temperature, air pressure, humidity, barometric pressure, etc. Data generated by sensor ENV 877 may be used to calculate the speed of sound, for example, wavefront transit time, in processing of data used by array 102 (see array 102 shown in FIG. 9 ). The autonomous vehicle 100 may include one or more sensors MOT888 configured to detect movement of the vehicle 100 (e.g., movement due to impact from a collision, movement due to an emergency / evasive maneuver, etc.). By way of example, the sensors MOT888 may include, but are not limited to, an accelerometer, a multi-axis accelerometer, and a gyroscope. The autonomous vehicle 100 may include one or more rotation sensors (not shown) (e.g., wheel encoders) associated with the wheels 852 and configured to detect rotation 853 of the wheels 852. For example, each wheel 852 may have an associated wheel encoder (e.g., an axle of the wheel 852 and / or a propulsion component of the wheel 852, such as an electric motor) configured to detect rotation of the wheel 852.Data representing the rotation signal generated by the rotation sensor may be received by one or more systems of the autonomous vehicle 100, such as, for example, one or more of a planner system, a localizer system, a perception system, a driving system, and a safety system.

[0069] Communications network 815 can route signals and / or data to / from sensors, one or more safety systems 875 (e.g., bladders, seat actuators, seat belt tensioners), and other components of autonomous vehicle 100, such as one or more processors 810 and one or more routers 830. Router 830 can route signals and / or data from sensors in sensor suite 820, one or more acoustic beam steering arrays 102 (e.g., one or more of an ultrasonic source, a directional acoustic source, a phased array, a parametric array, a large radiator), one or more optical emitters, other routers 830, processors 810, vehicle operation systems, such as propulsion (e.g., electric motor 851), steering, braking, one or more safety systems 875, and the like, and communications system 880 (e.g., for wireless communication with external systems and / or resources).

[0070] 8 , one or more microphones 827 may be configured to capture ambient sounds in an environment 890 external to the autonomous vehicle 100. Signals and / or data from the microphones 827 may be used to adjust gain values ​​for one or more speakers disposed in one or more of the acoustic beam steering arrays (not shown). As an example, loud ambient noises, such as those emanating from a construction site, may mask or otherwise impair the audibility of the beams of directed acoustic energy 104 being emitted by the acoustic beam steering array. Thus, the gain may be increased or decreased (e.g., in dB or other metric, such as frequency content) based on the ambient sounds.

[0071] Microphone 871 may be positioned near driving system components, such as electric motor 851, wheels 852, or brakes (not shown), to capture sounds generated by those systems, such as noise from rotation 853, regenerative braking noise, tire noise, and electric motor noise. The signal and / or data generated by microphone 871 may be used, for example, as data representing an audio signal associated with an audio alert. In other examples, the signal and / or data generated by microphone 871 may be used to modulate data representing the audio signal. As an example, the data representing the audio signal may constitute an audio recording (e.g., a digital audio file), and the signal and / or data generated by microphone 871 may be used to modulate data representing the audio signal. Furthering this example, the signal and / or data generated by microphone 871 may indicate the speed of vehicle 100, and as vehicle 100 slows down and stops at a crosswalk, the data representing the audio signal may be modulated by changes in the signal and / or data generated by microphone 871 as the speed of vehicle 100 changes. In another example, signals and / or data generated by microphone 871 indicative of the speed of vehicle 100 can be used as data representing an audio signal, and as the speed of vehicle 100 changes, the sound being emitted by acoustic beam steering array 102 can indicate the change in speed of vehicle 100. Thus, the sound (or magnitude of the acoustic energy) can be modified (e.g., in volume, frequency, etc.) based on the change in speed. The above example can be implemented to audibly notify pedestrians that autonomous vehicle 100 has detected their presence at a crosswalk and is slowing down and stopping.

[0072] The one or more processors 810 may be used, for example, to implement one or more of a planner system, a localizer system, a perception system, one or more safety systems, and other systems of the vehicle 100. The one or more processors 810 may be configured, for example, to execute algorithms embodied in a non-transitory computer-readable medium to implement one or more of the planner system, the localizer system, the perception system, one or more safety systems, or other systems of the vehicle 100. The one or more processors 810 may include, but are not limited to, circuitry, logic, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), programmable logic, a digital signal processor (DSP), a graphics processing unit (GPU), a microprocessor, a microcontroller, a big fat computer (BFC), or the like, or clusters thereof.

[0073] FIG. 9 shows a top view of an example acoustic beam steering array 900 in an external safety system for an autonomous vehicle. In FIG. 9, a sensor suite (e.g., 820 in FIG. 8) may be located at a corner (e.g., a support section) of the autonomous vehicle 100, and housings for the acoustic beam steering arrays 102 may be located on an upper surface 100u of the autonomous vehicle 100 (e.g., on the roof or other location on the vehicle) and positioned to direct their respective beams 104 of directed acoustic energy (see, e.g., FIG. 11 ) outward into the environment toward the location of an object of interest that is to receive an acoustic alert. The acoustic beam steering arrays 102 (e.g., ultrasonic, directional acoustic sources, phased arrays, parametric arrays, large radiators) may be configured to provide coverage of directed acoustic energy at one or more objects located in the environment external to the autonomous vehicle 100, which coverage may be, for example, within an approximately 360° arc centered on the autonomous vehicle 100. The acoustic beam steering arrays 102 do not have to be the same size, shape, or have the same number of speakers. The acoustic beam steering arrays 102 do not have to be linear as shown in FIG. 9. In addition to FIG. 9, the acoustic beam steering arrays 102 labeled C and D have different dimensions than the acoustic beam steering arrays 102 labeled A and B. The sensor suites 820 can be arranged to provide sensor coverage of the environment external to the autonomous vehicle 100 with one acoustic beam steering array 102 or multiple acoustic beam steering arrays 102. In the case of multiple acoustic beam steering arrays 102, the sensor suites 820 can provide overlapping areas of sensor coverage.The perception system of autonomous vehicle 100 may receive sensor data from multiple sensors or a suite of sensors and provide the data to a planning system to activate one or more of arrays 102, for example, to generate an acoustic alert and / or activate one or more other safety systems of vehicle 100. Autonomous vehicle 100 may not have a front (e.g., hood) or a rear (e.g., trunk) and thus may be configured for driving operations in at least two different directions as shown by arrow 980. Thus, acoustic beam steering array 102 may not have a front or rear designation, and array 102(A) may be the array facing the direction of travel, or array 102(B) may be the array facing the direction of travel, depending on which direction autonomous vehicle 100 is being driven.

[0074] Other safety systems of autonomous vehicle 100 may be located at interior 100i (shown in dashed lines) and exterior 100e locations on autonomous vehicle 100 and may be activated by the planner system to generate alerts or other safety functions using sensor data from the sensor suite. Overlapping areas of sensor coverage may be used by the planner system to activate one or more safety systems in response to multiple objects in the environment, which may be located at locations around autonomous vehicle 100.

[0075] 10A-10B show top plan views of examples of sensor coverage. In example 1010 of FIG. 10A , one of four sensor suites 820 (shown underlined) can provide sensor coverage 1011 using one or more of its respective sensors of an environment 1090 in a coverage area that can be configured to provide sensor data to one or more systems, such as a perception system, a localizer system, a planner system, and a safety system, of autonomous vehicle 100. In the top plan view of FIG. 10A , arrows A, B, C, and D demarcate four quadrants 1-4 that surround autonomous vehicle 100. In example 1010, sensor coverage 1011 by a single suite 820 can be partial sensor coverage because there can be partial sensor blind spots in quadrants 1, 3, and 4 that are not covered by the single suite 820, as well as complete sensor coverage in quadrant 2.

[0076] In example 1020, a second sensor suite 820 of the four sensor suites 820 can provide sensor coverage 1021 that overlaps with sensor coverage 1011, such that there can be partial sensor coverage in quadrants 1 and 4 and complete sensor coverage in quadrants 2 and 3. In Figure 10B, in example 1030, a third sensor suite 820 of the four sensor suites 820 can provide sensor coverage 1031 that overlaps with sensor coverages 1011 and 1021, such that quadrants 2, 3, and 4 have complete sensor coverage and quadrant 1 has partial coverage. Finally, in example 1040, a fourth sensor suite 820 of the four sensor suites 820 (e.g., all four sensor suites are online) can provide sensor coverage 1041 that overlaps with sensor coverages 1011, 1021, and 1031, such that quadrants 1-4 have complete coverage. The overlapping sensor areas of coverage can allow for redundancy in sensor coverage in the event that one or more of the sensor suites 820 and / or their respective sensors (e.g., lidar, camera, radar, etc.) are damaged, malfunction, or otherwise rendered inoperable. One or more of the sensor coverage patterns 1011, 1021, 1031, and 1041 can enable the planner system to activate one or more safety systems based on the location of those safety systems on the autonomous vehicle 100. As an example, a bladder of a bladder safety system located on the side of the autonomous vehicle 100 (e.g., the side of the vehicle 100 with arrow C) may be activated to counter a collision from an object approaching the vehicle 100 from that side.

[0077] 11A shows an example of an acoustic beam steering array in an external safety system of an autonomous vehicle. In FIG. 11A, control data 317 from a planner system 310 can be communicated to a vehicle controller 350, which can then communicate external data 325 configured to implement an acoustic alert by the acoustic beam steering array 102. While one array 102 is shown, the autonomous vehicle 100 can include multiple arrays 102 as indicated by 1103 (see, for example, the arrays 102 in FIG. 9). The external data 325 received by the array 102 can include, but is not limited to, object location data 1148 (e.g., coordinates of an object 1134 in an environment 1190), audio signal data 1162, trigger signal data 1171, and optionally modulation signal data 1169.

[0078] The acoustic beam steering array 102 may include a processor 1105 (e.g., a digital signal processor (DSP), field programmable gate array (FPGA), central processing unit (CPU), microprocessor, microcontroller, GPU and / or clusters thereof, or other embedded processing system) that receives external data 325 and processes the external data 325 to generate a beam 104 of directed acoustic energy into the environment 1190 (e.g., at an angle β relative to the trajectory T of the AV 100) (e.g., in response to receiving data representing a trigger signal 1171). The acoustic beam steering array 102 may include several speakers S, with each speaker S in the array 102 coupled to the output of an amplifier A. Each amplifier A may include a gain input and a signal input. The processor 1105 may calculate data representing a signal gain G for the gain input of each amplifier A and may calculate data representing a signal delay D for the signal input of each amplifier A. The processor 1105 can access and / or receive (e.g., from internal and / or external data sources) data representing information on the speakers S, which may include, but are not limited to, the array width (e.g., the distance between the first and last speakers in the array 102), the spacing of the speakers S in the array (e.g., the distance between adjacent speakers S in the array 102), the wavefront distance between adjacent speakers S in the array, the number of speakers S in the array, and speaker characteristics (e.g., frequency response, output level in watts of power, radiating area, etc.), to name just a few examples. Each speaker S and its associated amplifier A in the array 102 can constitute a mono channel, and the array 102 can include n mono channels, where the number n of mono channels can vary depending on the speaker size and enclosure size of the array 102. For example, n can be 30 or 320.For example, in an ultrasonic parametric array implementation of array 102, n may be on the order of about 80 to about 300. In some examples, the spacing between speakers in array 102 may not be linear for some or all of the speakers in array 102.

[0079] In the example 1100 of FIG. 11A , a directed beam 104 of acoustic energy is directed toward an object 1134 (e.g., a skateboarder) having a predicted location Lo in an environment 1190 that may conflict with a trajectory Tav of the autonomous vehicle 100. The location of the object 1134 may be included in the external data 325 as object location data 1148. The object location data 1148 may be data representing the coordinates (e.g., angles, Cartesian coordinates, polar coordinates, etc.) of the object 1134. The array 102 may process the object location data 1148 to calculate an angle β for directing the beam 104 toward the object 1134. The angle β may be measured relative to the trajectory Tav or relative to any point on the frame of the vehicle 100 (e.g., see 100r in FIG. 11C ). If the predicted location Lo changes due to movement of the object 1134, other systems of the autonomous vehicle system 101 can continue to track and update data associated with the autonomous vehicle 100 and the object 1134 to calculate updated object location data 1134. Thus, angle β can change (e.g., be recalculated by the processor 1105) as the location of the object 1134 and / or vehicle 100 changes. In some examples, angle β can be calculated (e.g., by a planner system) to take into account not only the current predicted direction (e.g., coordinates) for the object, but also the predicted direction for the object at a future time t, and the magnitude of the time delay in firing the array 102 can be a function of the data processing latency of a processing system of the vehicle 100 (e.g., processing latency in one or more of the planner system, perception system, or localizer system).Thus, if the object is predicted to be within a fraction of a second of the time the array 102 is fired, the planner system can cause the array 102 to emit sound in a manner that takes into account processing latency and / or the speed of the sound itself (e.g., based on environmental conditions), for example, depending on how far the object is from the vehicle 100.

[0080] 11B shows an example of a flow diagram 1150 for performing acoustic beam steering in an autonomous vehicle. At stage 1152, data representing a trajectory of autonomous vehicle 100 in an environment may be calculated based on data representing the location of autonomous vehicle 100 (e.g., local position data). At stage 1154, data representing a location (e.g., coordinates) of an object located in the environment may be determined (e.g., from object track data obtained from sensor data). Data representing an object type may be associated with the data representing the object's location. At stage 1156, data representing a predicted location of the object in the environment may be predicted based on the data representing the object type and the data representing the object's location in the environment. At stage 1158, data representing a threshold location in the environment associated with an audio alert (e.g., from array 102) may be estimated based on the data representing the predicted location of the object and the data representing the trajectory of the autonomous vehicle. At stage 1160, data representing an audio signal associated with the audio alert may be selected. At stage 1162, it may be detected that the location of the object matches the threshold location. As an example, one indication of a match may be the predicted location of an object crossing a threshold location. At stage 1164, data representing signal gains "G" associated with the speaker channels of the acoustic beam steering array 102 may be calculated. An array 102 having n speaker channels may have n different gains "G" calculated for each of the n speaker channels. The data representing the signal gain "G" may be applied to a gain input of an amplifier A coupled to a speaker S in the channel (e.g., as shown in the array 102 of FIG. 11A).At stage 1166, data representing a signal delay "D" may be calculated for each of the n speaker channels of the array 102. The data representing the signal delay "D" may be applied to a signal input of an amplifier A that is coupled to the speaker S in the channel (e.g., as shown in the array 102 of FIG. 11A). At stage 1168, the vehicle controller (e.g., 350 of FIG. 11A) may implement the acoustic alert by causing (e.g., triggering, activating, or commanding) the array 102 to emit a beam of directed acoustic energy (e.g., beam 104) in a direction of propagation (e.g., direction of propagation 106) determined by the object's location (e.g., coordinates of the object in the environment), the beam of directed acoustic energy being indicative of an audio signal.

[0081] The stages of flowchart 1150 may be performed for one or more of arrays 102, and one or more stages of flowchart 1150 may be repeated. For example, the predicted object path, object location (e.g., object coordinates), predicted object location, threshold location, vehicle trajectory, audio signal selection, match detection, and other stages may be repeated as needed to update and / or process data while autonomous vehicle 100 travels through the environment and / or as objects change location in the environment.

[0082] 11C shows a top plan view of one example 1170 of an autonomous vehicle directing acoustic energy associated with an acoustic alert to an object. In FIG. 11C, autonomous vehicle 100 may have a trajectory Tav along the roadway between lane markers indicated by dashed line 1177. A detected object 1171 in the environment external to autonomous vehicle 100 is classified as an automobile object type having a predicted location Lc in the environment that is estimated to conflict with trajectory Tav of autonomous vehicle 100. The planner system may generate three threshold locations t-1, t-2, and t-3 (e.g., having arcuate contours 1121, 1122, and 1123, respectively). The three threshold locations t-1, t-2, and t-3 may represent, for example, increasing threat levels ranked from a low threat level at threshold location t-1 (e.g., a relatively safe distance away from vehicle 100), a medium threat level at threshold location t-2 (e.g., an alarming proximity to vehicle 100), and a high threat level at threshold location t-3 (e.g., a dangerously close proximity to vehicle 100). Separate audio signals for the audio alert to be generated at each of the three threshold locations t-1, t-2, and t-3 may be selected to audibly communicate increasing levels of threat as the predicted location Lc of object 1171 brings object 1171 closer to the location of autonomous vehicle 100 (e.g., close enough for a potential collision with autonomous vehicle 100). The directed beam of acoustic energy 104 may indicate information contained in the selected audio signals (e.g., audio signals 104a, 104b, and 104c).

[0083] If the object type 1171 intersects or otherwise has a location that coincides with the threshold location t-1, the planner system can generate a trigger signal to activate the acoustic array 102, which is positioned to generate an acoustic alert using an audio signal 104a along the direction of propagation 106a (e.g., to communicate a non-threatening acoustic alert) based on the coordinates of the object 1171. For example, the coordinates can be the angle β measured between the trajectory Tav and the direction of propagation 106a. The reference point for the coordinates (e.g., angles βa, βb, and βc) can be, for example, point 102r on the array 102 or any other location on the autonomous vehicle 102, such as point 100r. If the object 1171 continues along its predicted location L and intersects threshold location t-2, another acoustic alert may be triggered by the planner system using coordinates (βb), audio signal 104b (e.g., to communicate an urgent acoustic alert), and direction of propagation 106b. Further travel by the object 1171 intersecting threshold location t-3 may trigger yet another acoustic alert by the planner system using coordinates (βc), audio signal 104c (e.g., to communicate a very urgent acoustic alert), and direction of propagation 106c. In this example, different audio signals (e.g., digital audio files having different sound patterns, different magnitudes of sound power and / or volume, whether pre-recorded or dynamically generated) may be selected for audio signals 104a, 104b, and 104c to communicate increasing levels of escalation to the object 1171 (e.g., to acoustically alert the vehicle driver).

[0084] For each of the acoustic alerts triggered by the planner system, the predicted location L of the object 1171 may change (e.g., relative to the location of the vehicle 100), and the planner system may receive updated object data (e.g., object tracking data from the perception system) to calculate (e.g., in real time) the change in the location of the object 1171 (e.g., calculate or recalculate β, β, and β). The audio signals selected by the planner system for each threshold location t-1, t-2, and t-3 may be distinct and may be configured, for example, to include audible information intended to convey increasing degrees of urgency for threshold locations t-1 to t-2 and from t-2 to t-3. The audio signals selected by the planner system may be configured to acoustically penetrate the vehicle's structure (1173, 1179), such as the vehicle glass, door panels, etc., to attract the vehicle driver's attention based on the object type data for the object 1171. In some examples, if an object (e.g., a vehicle driver) is detected (e.g., by the planner system) as changing its behavior (e.g., changing its predicted location, its predicted object path, or otherwise no longer posing a threat to the vehicle 100 and / or its occupants), the planner system can cause the array 102 to escalate the acoustic alert by reducing the level of urgency of the alert (e.g., by selecting an audio signal that indicates a reduced level of urgency). As one example, the selected audio signal can be configured to produce frequencies in the range of approximately 220 Hz to approximately 450 Hz to acoustically penetrate structures (1173, 1179) on the object 1171. As another example, the array 102 can be configured to produce sounds at frequencies in the range of approximately 220 Hz to approximately 4.5 kHz, or any other frequency range.Additionally, the sounds produced by the array 102 may be altered (e.g., volume, frequency, etc.) based on speed changes in the autonomous vehicle 100. The frequencies of the sounds emitted by the array 102 are not limited to the aforementioned examples, and the array 102 may be configured to produce sounds at frequencies that are within the human hearing range, above the human hearing range (e.g., ultrasonic frequencies), below the human hearing range (e.g., infrasonic frequencies), or some combination thereof.

[0085] Although the autonomous vehicle 100 is shown with two arrays 102 disposed on an upper surface 100u (e.g., the roof of the vehicle 100), the vehicle 100 may have more or fewer arrays 102 than shown, and the arrangement of the arrays 102 may differ from that shown in FIG. 11C . The acoustic beam steering array 102 may include several speakers and their associated amplifiers and driving electronics (e.g., processors, DSPs, etc.). For purposes of explanation, each amplifier / speaker pair will be denoted as a channel, whereby the array 102 includes n channels, denoted as C1 for channel 1 through Cn for the nth channel. In a close-up view of the acoustic beam steering array 102, each speaker S may be separated from its neighboring speaker by a distance d. The distance d (e.g., the spacing between neighboring speakers (S)) may be the same for all speakers S in the array 102, whereby all of the speakers S are separated from each other by the distance d. In some examples, the distance d may vary between speakers S in the array 102 (e.g., the distance d need not be the same between adjacent speakers in the array 102). The distance d may be measured from a reference point on each speaker S, such as the center point of each speaker S.

[0086] The width W of the array 102 may be measured as the distance between the first speaker in the array 102 (e.g., channel C1) and the last speaker in the array 102 (e.g., channel Cn), and the width W may be measured, for example, from the center of the speaker in C1 to the center of the speaker in Cn. In the direction 106 of propagation of the sound waves 104 generated by the array 102, each wavefront emitted by each adjacent speaker S in the array 102 may be delayed in time by a wavefront propagation time td. The wavefront propagation time td is the distance between adjacent wavefronts r multiplied by the speed of sound c (e.g., t d =r * In an example where the distance d between the speakers S is the same for all speakers S in the array 102, the delay D calculated for each speaker S can be calculated as t d Thus, for example, for channel C1, (t d1 =(r * c) * 1), and for channel C2 (t d2 =(r * c) * 2), for channel Cn (t dn =(r * c) * n). In some examples, the speed of sound c can be calculated using data from environmental sensors (e.g., sensor 877 in FIG. 8) to more accurately determine a value representing the speed of sound c based on environmental conditions such as altitude, air pressure, temperature, humidity, and barometric pressure.

[0087] Figure 12A shows an example of a light emitter 1200 located external to an autonomous vehicle. In Figure 12A, control data 317 from planner system 310 can be communicated to vehicle controller 350, which can then communicate external data 325 configured to implement visual alerts using light emitter 1202. Although one light emitter 1202 is shown, autonomous vehicle 100 can include multiple light emitters 1202, as indicated by 1203. The external data 325 received by the light emitters 1202 may include, for example, but is not limited to, data representing a light pattern 1212, data representing a trigger signal 1214 configured to activate one or more light emitters 1202, data representing an array selection 1216 configured to select which light emitters 1202 of the vehicle 100 to activate (e.g., based on the orientation of the vehicle 100 relative to the object for which the visual alert is directed), and, optionally, data representing a driving signaling 1218 configured to deactivate light emitter operation if the vehicle 100 is using its signaling lights (e.g., turn signal, brake signal, etc.). In other examples, the one or more light emitters 1202 may act as signal lights, headlights, or both. The orientation of the autonomous vehicle 100 relative to the object can be determined based, for example, on the location of the autonomous vehicle 100 (e.g., from a localizer system) and the location of the object (e.g., from a perception system), the trajectory of the autonomous vehicle 100 and the location of the object (e.g., from a planner system), or both.

[0088] The light emitter 1202 may include a processor 1205 configured to implement a visual alert based on external data 325. A selection function of the processor 1205 may receive data representing an array selection 1216 and enable activation of the selected light emitter 1202. The selected light emitter 1202 may be configured not to emit light L until data representing a trigger signal 1214 is received by the selected light emitter 1202. The data representing the light pattern 1212 may be decoded by a decode function, and sub-functions may operate on the decoded light pattern data to implement, for example, a color function configured to determine a color of light to be emitted by the light emitter 1202, an intensity function configured to determine an intensity of light to be emitted by the light emitter 1202, and a duration function configured to determine a duration of light emission from the light emitter 1202. The data store (Data) may include data representing the configuration of each light emitter 1202 (e.g., the number of light-emitting elements E, electrical characteristics of the light-emitting elements E, the location of the light emitter 1202 on the vehicle 100, etc.). Outputs from various functions (e.g., decoder, selection, color, intensity, and duration) may be coupled to a driver 1207 configured to apply signals to the light-emitting elements E1-En of the light emitter 1202. Each light-emitting element E may be individually addressable based on data representing the light pattern 1212. Each light emitter 1202 may include several light-emitting elements E, whereby n may represent the number of light-emitting elements E in the light emitter 1202. As an example, n may be greater than 50. The light emitter 1202 may vary in size, shape, number of light-emitting elements E, type of light-emitting element E, and location of the light emitter 1202 located outside the vehicle 100 (e.g., a light emitter 1202 coupled to a structure of the vehicle 100 that operates to enable the light emitter to emit light L into the environment, such as the roof 100u or other structure of the vehicle 100).

[0089] As an example, the light-emitting elements E1-En can be solid-state light-emitting devices, such as light-emitting diodes or organic light-emitting diodes. The light-emitting elements E1-En can emit light of a single wavelength or light of multiple wavelengths. The light-emitting elements E1-En can be configured to emit light of multiple colors, such as red, green, blue, and one or more combinations thereof, based on data representing the light pattern 1212. The light-emitting elements E1-En can be RGB light-emitting diodes (RGB LEDs) as shown in FIG. 12A. The driver 1207 can be configured to sink or source current to one or more inputs of the light-emitting elements E1-En, such as one or more of the red-R, green-G, or blue-B inputs of the light-emitting elements E1-En, to emit light L having a color, intensity, and duty cycle based on the data representing the light pattern 1212. The light-emitting elements E1-En are not limited to the example 1200 of FIG. 12A , and other types of light-emitter elements can be used to implement the light emitter 1202. 12A , an object 1234 having a predicted location Lo that is determined to be in conflict with the trajectory Tav of the autonomous vehicle 100 is the subject of a visual alert (e.g., based on one or more threshold locations in the environment 1290). One or more of the light emitters 1202 may be triggered to emit light L as the visual alert. If the planner system 310 determines that the object 1234 is not responding (e.g., the object 1234 has not changed its location to avoid a collision), the planner system 310 may select a different light pattern configured to escalate the urgency of the visual alert (e.g., based on different threshold locations with different associated light patterns).On the other hand, if the object 1234 is detected (e.g., by a planner system) as responding to the visual alert (e.g., the object 1234 is changing its location to avoid a collision), a light pattern configured to gradually de-escalate the urgency of the visual alert may be selected. For example, one or more of the light color, light intensity, light pattern, or some combination of the foregoing may be changed to indicate a gradual de-escalation of the visual alert.

[0090] The planner system 310 may select light emitters 1202 based on the orientation of the autonomous vehicle 100 relative to the location of the object 1234. For example, if the object 1234 is approaching the autonomous vehicle 100 head-on, one or more light emitters 1202 located on the exterior of the vehicle 100 that are generally facing the direction of the object's approach may be activated to emit light L for the visual alert. As the relative orientation between the vehicle 100 and the object 1234 changes, the planner system 310 may activate other emitters 1202 located at other exterior locations on the vehicle 100 to emit light L for the visual alert. Light emitters 1202 that may not be visible to the object 1234 may not be activated to prevent potential distraction or confusion among other drivers, pedestrians, etc., to whom the visual alert is not directed.

[0091] 12B shows a profile view of an example light emitter 1202 located on the exterior of autonomous vehicle 100. In example 1230, a partial profile view (e.g., a view along the direction of arrow 1236) of a first end of vehicle 100 shows several light emitters 1202 disposed at separate locations on the exterior of vehicle 100. The first end of vehicle 100 can include lights 1232 that can be configured for vehicle signaling functions, such as brake lights, turn signals, hazard lights, headlights, running lights, etc. In some examples, light emitters 1202 can be configured to emit light L that is different (e.g., in light color, light pattern, light intensity, etc.) from the light emitted by lights 1232, such that the function of light emitters 1202 (e.g., visual alert) is not confused with the vehicle signaling function performed by lights 1232. In some examples, lights 1232 can be configured to perform vehicle signaling functions and visual alert functions. The light emitters 1202 may be positioned in a variety of locations, including, but not limited to, for example, a support section, the roof 100u, a door, a bumper, and a fender. In some examples, one or more of the light emitters 1202 may be positioned behind an optically transparent or partially transparent surface or structure of the vehicle 100, such as, for example, behind a window, a lens, or a covering. The light L emitted by the light emitters 1202 may pass through the optically transparent surface or structure into the environment.

[0092] In example 1235, a second end of vehicle 100 (e.g., a view along the opposite direction of arrow 1236) can include light 1231 that can be configured for traditional automotive signaling and / or visual alert functions. The light emitter 1202 shown in example 1235 can also be positioned in a variety of locations, including, but not limited to, for example, a support section, roof 100u, doors, bumpers, and fenders.

[0093] It should be noted that, according to some examples, lights 1231 and 1232 may be optional, and functionality of vehicle signaling functions, such as brake lights, turn signals, hazard lights, headlights, running lights, etc., may be performed by any one of one or more light emitters 1202.

[0094] 12C shows a top plan view of one example 1240 of light emitter 1202 activation based on the orientation of the autonomous vehicle relative to an object. In FIG. 12C, object 1234 has a predicted location L relative to the location of autonomous vehicle 100. The relative orientation of autonomous vehicle 100 with object 1234 can provide full sensor coverage of object 1234 using sensor suite 820 positioned to sense quadrants 2 and 3, and partial sensor coverage in quadrant 1. Based on the relative orientation of vehicle 100 and object 1234, a subset of light emitters shown as 1202a (e.g., on the sides and ends of vehicle 100) may be visually perceptible by object 1234 and can be activated to emit light L into environment 1290.

[0095] FIG. 12D shows a contour diagram of one example 1245 of light emitter activation based on the orientation of an autonomous vehicle relative to an object. In FIG. 12D, object 1234 has a predicted location L relative to the location of autonomous vehicle 100. Based on the relative orientation of vehicle 100 and object 1234, a subset of light emitters shown as 1202a (e.g., on the side of vehicle 100) may be visually perceptible by object 1234 and may be activated to emit light L into environment 1290. In the example of FIG. 12D, the relative orientation of vehicle 100 with object 1234 is different from that shown in FIG. 12C because the approach of object 1234 is not within the sensor coverage of quadrant 1. Thus, light emitters 1202 located at the edge of vehicle 100 may not be visually perceptible to object 1234 and may not be activated for a visual alert.

[0096] FIG. 12E shows an example flow diagram 1250 for implementing a visual alert from a light emitter in an autonomous vehicle. At stage 1252, data representing a trajectory of the autonomous vehicle in an environment external to the autonomous vehicle may be calculated based on data representing a location of the autonomous vehicle in the environment. At stage 1254, data representing a location of an object in the environment may be determined. The object may include an object type. At stage 1256, a predicted location of the object in the environment may be predicted based on the object type and the object location. At stage 1258, data representing a threshold location in the environment associated with the visual alert may be estimated based on the trajectory of the autonomous vehicle and the predicted location of the object. At stage 1260, data representing a light pattern associated with the threshold location may be selected. At stage 1262, a location of the object may be detected that matches the threshold location. At stage 1264, data representing an orientation of the autonomous vehicle relative to the location of the object may be determined. At stage 1266, one or more light emitters of the autonomous vehicle may be selected based on the orientation of the autonomous vehicle relative to the location of the object. At stage 1268, the selected light emitter may be caused (e.g., activated, triggered) to emit light indicative of a light pattern into the environment to provide a visual alert.

[0097] FIG. 12F shows a top plan view of an example light emitter 1202 1270 of an autonomous vehicle 100 emitting light L to implement a visual alert in cooperation with an optional audible alert from one or more arrays 102. In FIG. 12F, the autonomous vehicle 100 has a trajectory Tav along the roadway between lane markers indicated by dashed line 1277, and a detected object 1272 has a predicted location L that is opposite and approximately parallel to the trajectory Tav. A planner system of the autonomous vehicle 100 can estimate three threshold locations T-1, T-2, and T-3 for implementing the visual alert. The light emitter 1202 located at the end of the vehicle 100 in the direction of travel 1279 (e.g., the light emitter 1202 faces the object 1272) is selected for the visual alert. In contrast, the arrays 1202 located at other ends of the vehicle 100 that do not face the direction of travel 1279 may not be selected for the visual alert. Because they may not be visually perceptible by the object 1272, they may be confusing to other drivers or pedestrians, for example. Light patterns 1204a, 1204b, and 1204c may be associated with threshold locations T-1, T-2, and T-3, respectively, and may be configured to provide escalating visual alerts (e.g., convey increasing urgency) as the predicted location Lo of the object 1272 approaches the location of the vehicle 100.

[0098] 12F, another detected object 1271 can be selected by the planner system of vehicle 100 for an acoustic alert, as described above with reference to FIGS. 11A-11C. The estimated threshold locations t-1 through t-3 for object 1271 may differ from the estimated threshold locations (e.g., T-1, T-2, and T-3) for object 1272. For example, the acoustic alert may be audibly perceptible at a greater distance from vehicle 100 compared to the visual perception of the visual alert. The velocity or speed of object 1271 may be greater than the velocity or speed of object 1272 (e.g., car versus bicycle), whereby threshold locations t-1 through t-3 are located farther away due to the greater velocity of object 1271, providing sufficient time for object 1271 to change its location to avoid a collision and / or for vehicle 100 to perform an evasive maneuver and / or activate one or more of its safety systems. 12F shows an example of a visual alert associated with object 1272 and an audible alert associated with object 1271, the number and type of safety systems (e.g., internal systems, external systems, or driving systems) activated to address the behavior of the objects in the environment are not limited to the shown example, and one or more safety systems may be activated. As an example, an audible alert, a visual alert, or both may be communicated to object 1271, object 1272, or both. As another example, if object 1271, object 1272, or both, is about to collide with vehicle 100 or comes within an unsafe distance of vehicle 100 (e.g., about 2 seconds or less until the predicted time of impact), one or more bladders may be deployed and one or more seat belts may be tightened.

[0099] The planner system can predict one or more regions of likely locations of objects in the environment (e.g., 565 in FIGS. 5 and 6) based on the predicted motion of each object and / or the predicted location of each object. The size, number, spacing, and shape of the regions of likely locations, as well as threshold locations within the regions of likely locations, can vary based on many factors, including, but not limited to, for example, characteristics of the objects (e.g., speed, type, location, etc.), the type of safety system selected, the speed of the vehicle 100, and the trajectory of the vehicle 100.

[0100] 13A shows an example bladder system 1300 in an external safety system for an autonomous vehicle. External data 325 received by the bladder system 369 can include data representing bladder selection 1312, data representing bladder deployment 1314, and data representing bladder deflation 1316 (e.g., from a deployed state to an undeployed state). The processor 1305 can process the external data 325 to control a bladder selector coupled to one or more bladder engines 1311. Each bladder engine 1311 can be coupled to a bladder 1310, denoted by BLADDER 1 through BLADDER n. Each bladder 1310 can be activated from an undeployed state to a deployed state by its respective bladder engine 1311. Each bladder 1310 can be activated from a deployed state to an undeployed state by its respective bladder engine 1311. In the deployed state, the bladders 1310 can extend outside the autonomous vehicle 100 (e.g., outside a body panel or other structure of the vehicle 100). The bladder engine 1311 can be configured to force a fluid (e.g., pressurized gas) into the bladder 1310 under pressure, causing the bladder 1310 to expand from an undeployed position to a deployed position, thereby causing a change in the volume of the respective bladder 1310. The bladder selector 1317 can be configured to select which bladders 1310 (e.g., among bladders 1 through n) to activate (e.g., deploy) and which bladders 1310 to deactivate (e.g., return to an undeployed state). The selected bladders 1310 can be deployed when the processor receives data representing bladder deployment 1314 for the selected bladders 1310. The selected bladder 1310 may be returned to an undeployed state when the processor receives data representing bladder deflation 1316 for the selected bladder 1310 .The processor 1305 can be configured to communicate data to the bladder selector 1317 to cause the bladder selector 1317 to deploy the selected bladder 1310 (e.g., via its bladder engine 1311) or return the deployed bladder 1310 (e.g., via its bladder engine 1311) to an undeployed state.

[0101] The bladder 1310 can be made, for example, from a flexible, resilient, or inflatable material, such as rubber or a synthetic material, or any other suitable material. In some examples, the material for the bladder 1310 can be selected based on a material that is reusable (e.g., if no anticipated impact to the bladder 1310 occurs, or if a collision does occur and does not damage the bladder 1310). As an example, the bladder 1310 can be made from a material used for air springs implemented in semi-tractor trailer trucks. The bladder engine 1311 can generate pressurized fluid that can be introduced into the bladder 1310 to inflate it to a deployed position, or can couple the bladder 1310 to a source of pressurized fluid, such as a tank of pressurized gas, or a gas generator. The bladder engines 1311 can be configured to release pressurized fluid from the bladders 1310 (e.g., via valves) to collapse the bladders 1310 from the deployed position back to their undeployed positions (e.g., collapse the bladders 1310 to their undeployed positions). As an example, the bladder engines 1311 can vent the pressurized fluid in their respective bladders 1310 to the atmosphere. In the deployed position, the bladders 1310 can be configured to absorb forces imparted by an impact of an object with the autonomous vehicle 100, thereby reducing or preventing damage to the autonomous vehicle and / or its occupants. For example, the bladders 1310 can be configured to absorb impact forces imparted by a pedestrian or bicyclist colliding with the autonomous vehicle 100.

[0102] Bladder data 1319 can be accessed by one or more of processor 1305, bladder selector 1317, or bladder engine 1311 to determine bladder characteristics, such as bladder size, bladder deployment time, bladder deflation time (e.g., the time to deflate the bladder 1310 from a deployed position to its original, undeployed position), the number of bladders, the location of bladders located outside of vehicle 100, etc. The bladders 1310 can vary in size and location on autonomous vehicle 100 and therefore can have different deployment times (e.g., inflation times) and different deflation times (e.g., deflation times). The deployment times can be used in determining whether there is sufficient time to deploy the bladder 1310 or multiple bladders 1310, for example, based on the predicted time of impact of an object being tracked by planner system 310. The bladder engine 1311 may include sensors, such as pressure sensors, to determine the pressure in the bladder 1310 when deployed and when undeployed, and to determine whether an impact ruptured or otherwise damaged the bladder 1310 (e.g., whether the rupture resulted in a leak in the bladder 1310). The bladder engine 1311 and / or sensor system 320 may include a motion sensor (e.g., an accelerometer, MOT888 in FIG. 8 ) to detect motion from an impact on the autonomous vehicle 100. A bladder 1310 and / or its respective bladder engine 1311 that is not damaged by an impact may be reused. If the predicted impact does not occur (e.g., there is no collision between the vehicle 100 and an object), the bladder 1310 may be returned to an undeployed state (e.g., via the bladder engine 1311), and the bladder 1310 may be reused later at a future time.

[0103] 13A , an object 1334 having a predicted location Lo may be said to be approaching autonomous vehicle 100 from one of its sides, such that the trajectory Tav of autonomous vehicle 100 is approximately perpendicular to the predicted object path of object 1334. Object 1334 may be predicted to impact the side of autonomous vehicle 100, and external data 325 may include data configured to cause bladder selector 1311 to select one or more bladders 1310 located on the side of autonomous vehicle 100 (e.g., the side where the impact is predicted to occur) for deployment prior to the predicted impact.

[0104] FIG. 13B illustrates examples 1330 and 1335 of bladders in an external safety system for an autonomous vehicle. In example 1330, vehicle 100 can include several bladders 1310 having positions associated with an exterior surface and / or structure of vehicle 100. The bladders 1310 can have different sizes and shapes. The bladders 1310 can be hidden behind other structures of the vehicle or can be disguised as decoration or the like. For example, a door panel or fender of vehicle 100 can include a membrane structure, and bladder 1310 can be positioned behind the membrane structure. The force generated by the deployment of bladder 1310 can rupture the membrane structure, allowing the bladder to expand outward toward the exterior of vehicle 100 (e.g., deploy outward into the environment). Similarly, in example 1335, vehicle 100 can include several bladders 1310. The bladder shown in example 1330 and the bladder shown in example 1335 may be symmetrical in location and / or size on vehicle 100.

[0105] Figure 13C shows examples 1340 and 1345 of bladder deployment in an autonomous vehicle. In example 1340, an object 1371 (e.g., a vehicle) is on a predicted collision course with vehicle 100 and has a predicted location Lo. The autonomous vehicle 100 has a trajectory Tav. Based on the relative orientation of vehicle 100 with respect to object 1371, the predicted impact location on vehicle 100 is estimated to be in quadrants 2 and 3 (e.g., an impact on the side of vehicle 100). Two passengers P located inside the vehicle 100, i.e., 100i, may be at risk due to the crumple zone distance z1 measured from the side of vehicle 100 to the seating position of passenger P in 100i. The planner system 310 can calculate the estimated time Timpact to impact with respect to object 1371 (e.g., using the kinematics calculator 384) to determine whether there is sufficient time to maneuver the autonomous vehicle 100 to avoid the collision or reduce potential injury to passenger P. If the planner system 310 determines that the time available to maneuver the vehicle 100 is less than the time to impact (e.g., Tmanuever < Timpact), the planner system can command the drive system 326 to execute an avoidance maneuver 1341 to rotate the vehicle 100 in the clockwise direction of the arrow (e.g., the arrow representing avoidance maneuver 1341) to position the end of vehicle 100 within the path of object 1371. The planner system can also command, for example, the activation of other internal safety systems and external safety systems such as the seatbelt tensioning system, the bladder system, the acoustic beam steering array 102, and the light emitter 1202 to occur simultaneously with the avoidance maneuver 1341.

[0106] In example 1345, the vehicle 100 has completed an evasive maneuver 1341, and the object 1371 is approaching from the end of the vehicle 100, rather than the side of the vehicle 100. Based on the new relative orientation between the vehicle 100 and the object 1371, the planner system 310 can command the selection and deployment of a bladder 1310′ located on the bumper at the end of the vehicle 100. In the event of an actual impact, the crumple zone distance increases from z1 to z2 (e.g., z2 > z1), providing the occupant P with a larger crumple zone in the interior 100i of the vehicle 100. Prior to the evasive maneuver 1341, the seat belt worn by the occupant P can be pre-tightened by a seat belt tensioning system to protect the occupant P during the maneuver 1341 and in preparation for a potential impact with the object 1371.

[0107] In example 1345, the deployment time, Tdeploy, of the bladder 1310' has been determined (e.g., by the planner system 310) to be less than the predicted impact time, Timpact, of the object 1371. Therefore, there is sufficient time to deploy the bladder 1310' before the potential impact of the object 1371. Comparisons of activation time to impact time can be performed for other safety systems, such as seat belt tensioning systems, seat actuator systems, acoustic arrays, and light emitters.

[0108] A driving system (e.g., 326 in FIG. 3B ) can be instructed (e.g., via planner system 310) to rotate the vehicle's wheels (e.g., via a steering system) and steer vehicle 100 (e.g., via a propulsion system) toward the configuration shown in example 1345. The driving system can be instructed (e.g., via planner system 310) to apply brakes (e.g., via a braking system) to prevent vehicle 100 from colliding with other objects in the environment (e.g., to prevent vehicle 100 from being pushed into other objects as a result of forces from the impact).

[0109] FIG. 14 illustrates an example seat belt tensioning system 1400 in an interior safety system 322 of an autonomous vehicle 100. In FIG. 14 , internal data 323 may be received by a seat belt tensioning system 361. The internal data 323 may include, but is not limited to, data representing a belt tensioner selection signal 1412, a seat belt tightening trigger signal 1414, and a seat belt tightening release signal 1416. The belt tensioner selection signal 1412 may select one or more belt tensioners 1411, with each belt tensioner 1411 including a seat belt 1413 (also shown as B-1, B-2, through B-n). Each belt 1413 may be mechanically coupled to a tensioning mechanism (not shown) in the belt tensioner 1411. The tensioning mechanism may include a reel configured to take up slack in the belt 1413 by winding a portion of the belt onto the reel on which the belt 1413 is wound. A belt tensioner 1411 selected by belt tensioner selection signal 1412 can apply tension to its respective belt 1413 upon receiving seat belt tightening trigger signal 1414. For example, belt tensioner 1411 carrying belt 1413(B-2) can actuate belt B-2 to move from a slack state (e.g., not tightly coupled to the occupant wearing belt B-2) to a tightened state (shown by the dashed line). In the tightened state, belt B-2 can apply pressure to the occupant, which pressure can tightly couple the occupant to the seat, for example, during an evasive maneuver and / or in anticipation of a predicted collision with an object.

[0110] Belt B-2 can be moved from a tightened state to a slack state and back to its original slack state by releasing the seat belt tightening trigger signal 1414 or by releasing the seat belt tightening trigger signal 1414 followed by receiving a seat belt release signal 1416. A sensor (e.g., a pressure sensor or force sensor) (not shown) can detect whether a seat in the autonomous vehicle is occupied by a passenger and can enable activation of the belt tensioner select signal 1412 and / or the seat belt tightening trigger signal 1414 if the sensor indicates that the seat is occupied. If the seat sensor does not detect seat occupancy, the belt tensioner select signal 1412 and / or the seat belt tightening trigger signal 1414 can be deactivated. Belt data 1419 can include data describing belt characteristics, including, but not limited to, belt tightening time, belt release time, and maintenance logs for seat belt 1413 in seat belt system 361, for example. The seat belt system 361 may operate as a separate system in the autonomous vehicle 100 or may operate in conjunction with other safety systems of the vehicle 100, such as optical emitters, acoustic arrays, bladder systems, seat actuators, and driving systems.

[0111] 15 illustrates an example 1500 of a seat actuator system 363 in an interior safety system 322 of an autonomous vehicle 100. In one example, the seat actuator system 363 can be configured to use energy from an impact force 1517 imparted to the vehicle 100 from an object (not shown) in the environment 1590 due to a collision between the vehicle 100 and the object (e.g., another vehicle) to actuate seats 1518 (e.g., seat-1 through seat-n) from a first position within the interior of the autonomous vehicle 100 to a second position within the interior of the autonomous vehicle 100. A force 1513 mechanically transmitted from the impact force 1517 to the seats (e.g., seat-n) can, for example, move the seats from the first position (e.g., near an end of the vehicle 100) to the second position (e.g., toward the center of the interior 100i of the vehicle 100). A reaction c-force 1515 may be applied to a seat (e.g., seat-n) to control the acceleration force resulting from force 1513. (Although not shown, note that the direction of c-force 1515 and the direction of force 1513 shown in FIG. 15 may be in a substantially common direction of impact force 1517.) The reaction c-force 1515 may be a spring configured to compress to counteract force 1513 or configured to expand to counteract force 1513. In other examples, the reaction c-force 1515 may be generated, for example, by a damper (e.g., a shock absorber) or by an air spring. A mechanism providing the reaction c-force 1515 may be coupled to the seat coupler 1511 and to the seat (e.g., seat-n). The seat coupler 1511 may be configured to secure the mechanism providing the reaction c-force 1515 to a chassis or other structure of the vehicle 100.If the impact force 1517 results in mechanical deformation of the vehicle 100 structure (e.g., crushing a crumple zone), a ram or other mechanical structure coupled to the seat may be urged forward (e.g., from a first position to a second position) by the deforming structure of the vehicle, exerting a force 1513 on the seat, while a reaction force c-force 1515 resists movement of the seat from the first position to the second position.

[0112] In other examples, the seat coupler 1511 may include an actuator for electrically, mechanically, or electromechanically actuating the seat 1518 (e.g., seat-n) from a first position to a second position in response to data representing the trigger signal 1516. The seat coupler 1511 may include a mechanism (e.g., a spring, a damper, an air spring, a deformable structure, etc.) that provides the reaction force c-force 1515. Data representing the seat selection 1512 and data representing the arming signal 1514 may be received by the processor 1505. A seat selector 1519 may select one or more of the seat couplers 1511 based on the data representing the seat selection 1512. The seat selector 1519 may not actuate the selected seat until data representing the arming signal 1514 is received. The data representing the arming signal 1514 may indicate a predicted collision with the vehicle 100 having a high probability of occurrence (e.g., that an object is predicted to soon collide with the vehicle based on its movement and location). Data representing the arming signal 1514 may be used as a signal to activate a seat belt tensioning system. The seats 1518 (e.g., seat-1 through seat-n) may be, for example, seats that can seat a single occupant (e.g., bucket seats) or seats that can seat multiple occupants (e.g., bench seats). The seat actuator system 363 may operate in conjunction with other interior and exterior safety systems of the vehicle 100.

[0113] 16A illustrates an example 1600 of a driving system 326 in an autonomous vehicle 100. In FIG. 16A , driving data 327 communicated to the driving system 326 can include, but is not limited to, data representing steering control 1612, braking control 1614, propulsion control 1618, and signal control 1620. The driving data 327 can be used for normal driving operations of the autonomous vehicle 100 (e.g., picking up passengers, transporting passengers, etc.), but can also be used to mitigate or avoid collisions and other potentially dangerous events associated with objects in the environment 1690.

[0114] Processor 1605 can communicate driving data 327 to particular driving systems, such as steering control data to steering system 361, braking control data to braking system 364, propulsion control data to propulsion system 368, and signaling control data to signaling system 362. Steering system 361 can be configured to process the steering data and actuate wheel actuators WA-1 to WA-n. Vehicle 100 can be configured for multi-wheel independent steering (e.g., four-wheel steering). Each wheel actuator WA-1 to WA-n can be configured to control the steering angle of the wheel coupled to that wheel actuator. Braking system 364 can be configured to process the braking data and actuate brake actuators BA-1 to BA-n. Braking system 364 can be configured to perform, for example, differential braking and anti-lock braking. Propulsion system 368 can be configured to process the propulsion data and actuate drive motors DM-1 to DM-n (e.g., electric motors). The signaling system 362 can process the signaling data to activate signal elements S-1 to Sn (e.g., brake lights, turn signals, headlights, running lights, etc.). In some examples, the signaling system 362 can be configured to perform signaling functions using one or more light emitters 1202. For example, the signaling system 362 can be configured to access all or a portion of the one or more light emitters 1202 to perform signaling functions (e.g., brake lights, turn signals, headlights, running lights, etc.).

[0115] FIG. 16B shows an example of obstacle avoidance maneuver 1640 for autonomous vehicle 100. In FIG. 16B, object 1641 (e.g., a car) has a predicted location Lo that is in conflict with trajectory Tav of autonomous vehicle 100. The planner system can calculate that there is not enough time to use the driving system to accelerate vehicle 100 forward into region 1642 located along trajectory Tav because object 1641 could collide with vehicle 100. Region 1642 therefore cannot be safely maneuvered into, and region 1642 can be designated (e.g., by the planner system) as blocked region 1643. However, the planner system can detect available open areas in the environment around vehicle 100 (e.g., via sensor data received by a perception system and map data from a localizer system). For example, region 1644 can be safely maneuvered into (e.g., region 1644 does not have any objects, whether moving or static, that interfere with the trajectory of vehicle 100). Region 1644 can be designated (e.g., by a planner system) as a vacant region 1645. The planner system can command the driving system (e.g., via steering data, propulsion data, and braking data) to change the trajectory of vehicle 100 from its original trajectory Tav to an evasive maneuver trajectory Tm to autonomously navigate autonomous vehicle 100 into vacant region 1645.

[0116] In conjunction with the obstacle avoidance maneuver, the planner system may activate one or more other internal and / or external safety systems of the vehicle 100 if the object 1641 changes speed or if the obstacle avoidance maneuver is unsuccessful, causing the bladder system to deploy bladders in portions of the vehicle that may be impacted by the object 1641.

[0117] In preparation for an avoidance maneuver and for potential collision with object 1641, a seatbelt tensioning system can be activated to tighten the seatbelt. Although not shown in FIG. 16B, other safety systems such as one or more acoustic arrays 102 and one or more light emitters 1202 can be activated to transmit acoustic alerts and visual alerts to object 1641. Belt data and bladder data can be used to calculate the tightening time for the seatbelt and the bladder deployment time for the bladder, and the calculated times are compared with the estimated impact time to determine if there is sufficient time to tighten the belt and / or deploy the bladder before the impact (e.g., Tdeploy < Timpact and / or Ttension < Timpact). Similarly, the time required for the driving system to perform a maneuver to area 1645 that is free is compared with the estimated impact time to determine if there is sufficient time to perform an avoidance maneuver (e.g., Tmanuever < Timpact).

[0118] FIG. 16C shows another example 1660 of an obstacle avoidance maneuver for autonomous vehicle 100. In FIG. 16C, object 1661 has a predicted location Lo that could cause a collision (e.g., a rear-end collision) of autonomous vehicle 100. The planner system can determine a predicted impact zone (e.g., an area or probability where a collision may occur based on object dynamics). Before the predicted collision occurs, the planner system can analyze the environment around vehicle 100 (e.g., using overlapping sensor fields of sensors in the sensor system) to determine whether there is an open area into which vehicle 100 can maneuver. The predicted impact zone is behind vehicle 100 and is effectively blocked (e.g., the vehicle cannot safely reverse direction to avoid the potential collision) by predicted location Lo, which is having a velocity toward the location of vehicle 100. The predicted impact zone can be designated (e.g., by the planner system) as a blocked area 1681. The left driving lane of vehicle 100 has no available clear area and is blocked due to the presence of three objects 1663, 1665, and 1673 (e.g., two cars located in adjacent driving lanes and a pedestrian on the sidewalk). Therefore, this area may be designated as blocked area 1687. The area ahead of vehicle 100 (e.g., in the direction of trajectory Tav) is blocked due to the approach of vehicle 1667 (e.g., a large truck). A location to the right of vehicle 100 is blocked due to pedestrian 1671 on the sidewalk at that location. Therefore, these areas may be designated as blocked areas 1683 and 1689, respectively. However, an object-free area may be detected in area 1691 (e.g., via the planner system and perception system). Area 1691 may be designated as clear area 1693, and the planner system may instruct the driving system to change the trajectory from trajectory Tav to evasive maneuver trajectory Tm.The resulting command may cause vehicle 100 to turn the corner into open area 1693 to avoid a potential rear-end collision with object 1661 .

[0119] In conjunction with an evasive maneuver into the open area 1693, other safety systems may be activated, such as, for example, bladders 1310 at the ends of the vehicle 100, seat belt tensioners 1411, acoustic arrays 102, seat actuators 1511, and optical emitters 1202. As an example, if the vehicle 1661 continues to approach the vehicle 100 at the predicted location Lo, one or more bladders 1301 may be deployed (e.g., at a time prior to the predicted time of impact sufficient to allow the bladders to expand to a deployed position), an acoustic alert may be communicated (e.g., by one or more acoustic beam steering arrays 102), and a visual alert may be communicated (e.g., by one or more optical emitters 1202).

[0120] The planner system may access data representing an object type (e.g., data on a vehicle, such as object 1661 predicted to rear-end vehicle 100) and compare the data representing the object with data representing the object type to determine data representing the object type (e.g., determine the object type for object 1661). The planner system may calculate the velocity or speed of the object based on the data representing the object's location (e.g., use kinematics calculator 384 to track changes in location over time to calculate velocity or speed). The planner system may access a data store, lookup table, data repository, or other data source to access data representing object braking capabilities (e.g., the braking capabilities of vehicle 1661). The data representing the object type may be compared with the data representing the object braking capabilities to determine data representing an estimated object mass and data representing an estimated object braking capability. The data representing the estimated object mass and data representing an estimated object braking capability may be based on estimated data for a particular class of object (e.g., various classes of cars, trucks, motorcycles, etc.). For example, if object 1661 has an object type associated with a mid-sized four-door sedan, then an estimated gross vehicle mass or weight, which may be average for that class of vehicle, may correspond to the estimated object mass for object 1661. The estimated braking capacity may also be an average value for the class of vehicle.

[0121] The planner system can calculate data representing the object's estimated momentum based on the data representing the estimated object braking capacity and the data representing the estimated object mass. The planner system can determine, based on the data representing the estimated momentum, that the braking capacity of an object (e.g., object 1661) is exceeded by its estimated momentum. Based on the momentum exceeding the braking capacity, the planner system can determine to calculate and execute the evasive maneuver of FIG. 16C to move vehicle 100 away from the predicted impact zone and into open area 1693.

[0122] 17 illustrates an example of visual communication with objects in an environment using visual alerts from light emitters of autonomous vehicle 100. In example 1720, autonomous vehicle 100 includes light emitters 1202 disposed at various locations external to autonomous vehicle 100. Autonomous vehicle 100 is shown traveling on a roadway 1711 having lane markers 1719 in an environment 1790 external to vehicle 100 and having a trajectory Tav. An object 1710 in environment 1790 (e.g., classified as a pedestrian object by vehicle 100's perception system) is shown standing on edge 1715 of sidewalk 1717 adjacent to roadway 1711. Object 1710 is standing near bicycle lane 1713 of roadway 1711. Initially, object 1710 may be unaware of the approach of autonomous vehicle 100 on roadway 1711 (e.g., due to low-noise emissions by vehicle 100's driving system, ambient noise, etc.). The perception system (e.g., 340 in FIG. 12A) can detect the presence of the object 1710 in the environment based on sensor signals from the sensor system (e.g., 320 in FIG. 12A) and can generate object data representing the object 1710, including, but not limited to, an object classification, an object track, an object type, the object's location in the environment 1790, the distance between the object and the vehicle 100, the orientation of the vehicle 100 relative to the object, a predicted progress of movement relative to the object's location, etc.

[0123] In furtherance of example 1720, the planner system (e.g., 310 in FIG. 12A ) of vehicle 100 can be configured to perform an estimation of a threshold event associated with one or more of light emitters 1202 that emit light L as a visual alert. For example, upon detecting object 1710, the planner system can prevent a visual alert from being immediately emitted by light emitter 1202; instead, the planner system can estimate a threshold event Te associated with causing the light emitter to emit light L for the visual alert based on data representing the location of object 1710 in environment 1790 and data representing the location of vehicle 100 in environment 1790 (e.g., posture data from localizer system 330 in FIG. 12A ).

[0124] As an example, if the vehicle 100 travels along the trajectory Tav and detects the object 1710, the initial distance between the vehicle 100 and the object 1710 at the time of detection may be a distance Di. The planner system may calculate another distance closer to the object as a threshold event for causing (e.g., triggering) the light emitter 1202 to emit light L for a visual alert. In example 1720, the distance Dt between the vehicle 100 and the object 1710 may be the distance associated with the threshold event Te. Further to this example, the threshold event Te may be associated with the distance Dt because that distance may be a more effective distance for providing a visual alert for various reasons, including, but not limited to, the vehicle 100 being too far away at the initial distance Di for the light L to be visually perceptible to the object 1710 and / or the object 1710 not perceiving that the light L is directed at it. As another example, the initial distance Di may be approximately 150 feet (45.72 meters) and the distance Dt for the threshold event Te may be approximately 100 feet (30.48 meters).

[0125] As a second example, if the vehicle 100 travels along the trajectory Tav and detects the object 1710, the initial time for the vehicle 100 to reduce the distance Di between the vehicle 100 and the object 1710 upon detection may be time Ti. The planner system may calculate a time after time Ti as the threshold event Te. For example, the threshold event Te may occur at time Tt after the initial time Ti, and the light emitter 1202 may emit light L.

[0126] In example 1740, vehicle 100 is shown traveling along a trajectory from an initial distance Di to a distance Dt, at which distance Dt light emitter 1202 is caused to emit light L. One or more of emitters 1202 located at various locations on vehicle 100 can emit light L according to data representing a light pattern. For example, initially, at threshold event Te (e.g., at time Tt or distance Dt), light L from light emitter 1202 at a first end of vehicle 100 facing the direction of travel (e.g., aligned with trajectory Tav) can emit light L because the first end faces object 1710. Meanwhile, light emitter 1210 on the side of the vehicle facing sidewalk 1717 may be invisible to object 1710 at distance Dt, for example.

[0127] Optionally, the planner system may activate one or more other safety systems of the vehicle 100 before, during, or after activation of the visual alert system. As an example, one or more acoustic beam steering arrays 102 may be activated to generate a directed beam 104 of acoustic energy toward the object 1710. The directed beam 104 of acoustic energy may be effective in making the object 1710 aware of the approaching vehicle 100 (e.g., by causing the object 1710 to turn 1741 its head toward the vehicle 100).

[0128] In example 1760, as the vehicle 100 approaches the object 1710, other light emitters 1202 may be visible to the object 1710, and the planner system may activate additional light emitters 1202, for example, located on the side of the vehicle facing the sidewalk 1717. Further to example 1760, a visual alert, and / or a visual alert combined with an audio alert (e.g., from the array 102) may be effective in moving 1761 the object 1710 onto the sidewalk 1717 and further away (e.g., to a safe distance) from the trajectory Tav of the approaching vehicle 100. After the vehicle 100 passes the object 1710, the light emitters 1202 and other safety systems that may have been activated may be deactivated.

[0129] FIG. 18 shows another example of a flow diagram 1800 for implementing a visual alert from a light emitter in autonomous vehicle 100. In flow 1800, at stage 1802, data representing a trajectory of autonomous vehicle 100 in an environment (e.g., environment 1790 of FIG. 17 ) may be calculated based on data representing the location of autonomous vehicle 100 in the environment. At stage 1804, data representing the location of an object (e.g., object 1710 of FIG. 17 ) in the environment may be determined (e.g., using sensor data received at a perception system). The object may have an object classification (e.g., object 1710 of FIG. 17 being classified as a pedestrian). At stage 1806, data representing a light pattern associated with the visual alert may be selected (e.g., from a data store, memory, or data file, etc.). The selected light pattern may be configured to visually notify an object (e.g., a pedestrian, or a driver of another vehicle) that autonomous vehicle 100 is present in the environment (e.g., environment 1790 of FIG. 17 ). The light pattern may be selected based on the object classification, for example, a first light pattern may be selected for an object having a classification of a pedestrian, a second light pattern may be selected for an object having a classification of a bicyclist, and a third light pattern may be selected for an object having a classification of a motor vehicle, and the first, second, and third light patterns may be different from one another.

[0130] At stage 1808, a light emitter of autonomous vehicle 100 may be caused to emit light L into the environment indicative of the light pattern. At stage 1808, the light emitter may be selected based on, for example, an orientation of vehicle 100 relative to an object (e.g., object 1710 of FIG. 17). Data representing the orientation of autonomous vehicle 100 relative to the location of the object may be calculated based on, for example, data representing the location of the object and data representing the trajectory of the autonomous vehicle.

[0131] The light emitter may include one or more subsections (not shown), and the data representing the light pattern may include one or more subpatterns, each subpattern associated with one of the subsections. Each subsection may be configured to emit light L into the environment that is indicative of its respective subpattern. The autonomous vehicle 100 may include many light emitters, some or all of which may include one or more subsections. In some examples, the subsections of a light emitter may be configured (e.g., via their respective subpatterns) to perform various functions. For example, one or more subsections of a light emitter may perform signaling functions of a driving system (e.g., turn signals, brake lights, hazard lights, running lights, fog lights, headlights, side marker lights, etc.), while one or more other subsections may perform visual alerts (e.g., via their respective subpatterns).

[0132] Data representing a threshold event (e.g., threshold event Te in FIG. 17) may be estimated based on data representing the location of the object and data representing the location of the autonomous vehicle 100. The occurrence of the threshold event may be detected (e.g., by object data, posture data, or both received by the planner system), and one or more light emitters may be caused to emit light L based on the occurrence of the threshold event.

[0133] In one example, estimating the data representing the threshold event may include calculating data representing a distance between the autonomous vehicle and the object based on the data representing the location of the autonomous vehicle and the data representing the location of the object. A threshold distance associated with the threshold event may be determined based on the data representing the distance between the autonomous vehicle and the object. The light pattern selected in stage 1806 may be determined, for example, based on the threshold distance. The threshold distance (e.g., Dt in FIG. 17) may be less than the distance (e.g., Di in FIG. 17).

[0134] In another example, estimating the data representing the threshold event may include calculating data representing a time (e.g., Ti in FIG. 17 ) associated with the location of autonomous vehicle 100 and the location of the object, which are consistent with one another, based on the data representing the location of the object and the data representing the trajectory of autonomous vehicle 100. A threshold time (e.g., Tt in FIG. 17 ) may be determined based on the data representing a time associated with the location of autonomous vehicle 100 and the location of the object, which are consistent with one another. The threshold time (e.g., Tt in FIG. 17 ) may be less than the time (e.g., Ti in FIG. 17 ).

[0135] 19 illustrates an example 1900 of visual communication with objects in an environment using visual alerts from light emitters of autonomous vehicle 100. In example 1900, autonomous vehicle 100 is autonomously traveling a trajectory Tav along roadway 1911 having lane markers 1915, a bike path 1913, a sidewalk 1917, a crosswalk 1920, traffic signs 1923, 1925, 1927, 1931, and 1933, and a traffic light 1921 (e.g., as detected and classified by a perception system). Two objects 1901 and 1902 have been detected by autonomous vehicle 100 and can be classified as pedestrian objects having a predicted location Lo in environment 1900.

[0136] Based on the data representing the traffic signs, traffic signals, or both, the autonomous vehicle can determine whether objects 1901 and 1902 are crossing roadway 1911 legally (e.g., as permitted by traffic signs 1923, 1925, 1927, 1931, and 1933 and / or traffic signal 1921) or illegally (e.g., as prohibited by traffic signs and / or traffic signals). In either case, autonomous vehicle 100 can be configured (e.g., via a planner system) to implement the safest interaction between vehicle 100 and the objects in the environment for the safety of the occupants of vehicle 100, the safety of the objects (e.g., 1901 and 1902), or both.

[0137] As pedestrian objects 1901 and 1902 cross crosswalk 1920 from first location L1 to second location L2, autonomous vehicle 100 can detect (e.g., via a sensor system and a perception system) a change in the predicted location La of objects 1901 and 1902 and cause a visual alert to be emitted by one or more light emitters 1202 (e.g., based on an orientation of vehicle 100 relative to objects 1901 and 1902). Data representing the location of vehicle 100 in environment 1990 can be used to calculate data representing a trajectory of vehicle 100 in environment 1990 (e.g., trajectory Tav along roadway 1911). The data representing the locations of objects 1901 and 1902 in environment 1990 can be determined based on, for example, data representing sensor signals from a sensor system (e.g., sensor data received by a perception system to generate object data).

[0138] Light patterns associated with visual alerts may be selected (e.g., by a planner system) to notify objects 1901 and 1902 of a change in the driving maneuver of vehicle 100. For example, as objects 1901 and 1902 cross crosswalk 1920 from first location L1 to second location L2, the pedestrians may be concerned that autonomous vehicle 100 is unaware of their presence and that the pedestrians (e.g., objects 1901 and 1902) may not stop or slow down before safely crossing crosswalk 1920. Thus, the pedestrians may be concerned about being hit by autonomous vehicle 100.

[0139] Autonomous vehicle 100 can be configured to notify pedestrians (e.g., objects 1901 and 1902) that vehicle 100 has detected their presence and is taking action to slow down, stop, or both at a safe distance from crosswalk 1920. For example, in region 1950 along roadway 1911, light pattern LP 1940 can be selected and configured to visually notify objects 1901 and 1902 (e.g., using light L emitted by light emitter 1202) that the vehicle is slowing down. The slowing of vehicle 100 can be indicated as a change in vehicle 100's maneuver implemented in light pattern 1940. As an example, as vehicle 100 slows down, the rate of flashing, strobe, or other pattern of light L emitted by light emitter 1202 can be changed (e.g., slowed down) to mimic the slowing of vehicle 100. Light pattern 1940 can be modulated with other data or signals indicative of a change in the driving operation of vehicle 100, such as, for example, a signal from a wheel encoder (e.g., percentage of wheel rotation for wheel 852 in FIG. 8), location data (e.g., from a GPS and / or IMU), a microphone configured to generate a signal indicative of the driving operation (e.g., 871 in FIG. 8), etc. In other examples, light pattern 1940 can include encoded data configured to mimic a change in the driving operation of vehicle 100.

[0140] In region 1950, as vehicle 100 decelerates, the pattern of light L emitted by light emitter 1202 can change, for example, as a function of velocity, speed, wheel rotation speed, or other metric. In another example, a driving maneuver of vehicle 100 can cause the vehicle to stop in region 1960, and light pattern 1970 can be selected to notify objects 1901 and 1902 that the driving maneuver is stopping the vehicle (e.g., at or before a safe distance Ds from crosswalk 1920). Dashed line 1961 can represent a predetermined safe distance Ds between vehicle 100 and crosswalk 1920, within which vehicle 100 is configured to stop. As an example, as the vehicle slows to a stop in region 1960, the light pattern emitted by light emitter 1202 can change from a dynamic pattern (e.g., indicating some movement of vehicle 100) to a static pattern (e.g., indicating no movement of vehicle 100). The light pattern 1970 may be modulated with other data or signals indicative of a change in the driving behavior of the vehicle 100, as described above, to visually indicate to the object a change in driving behavior.

[0141] FIG. 20 shows yet another example of a flow diagram 2000 for implementing a visual alert from a light emitter in autonomous vehicle 100. At stage 2002, data representing a trajectory (e.g., trajectory Tav) of autonomous vehicle 100 may be calculated based on data representing the location of vehicle 100 in the environment. At stage 2004, the location of an object (e.g., objects 1901 and 1902) in the environment may be determined. The detected object may include an object classification (e.g., a pedestrian object classification for objects 1901 and 1902). At stage 2006, data representing a light pattern associated with a visual alert and configured to notify the object of a change in the driving performance of vehicle 100 may be selected. At stage 2008, light emitter 1202 of autonomous vehicle 100 may be caused to emit light L into the environment indicative of the light pattern. As the orientation of autonomous vehicle 100 changes relative to the location of the object, the light emitter 1202 selected to emit light L may change.

[0142] Stopping and / or slowing movements of vehicle 100 can be implemented by the planner system commanding the driving system to alter the driving behavior of vehicle 100. The driving system can implement commands from the planner system by controlling the operation of the steering system, braking system, propulsion system, safety system, signaling system, or a combination of the foregoing.

[0143] FIG. 21 shows outline views of other examples 2110 and 2120 of light emitters disposed on the exterior of autonomous vehicle 100. Autonomous vehicle 100 can be configured for driving operations in multiple directions, as indicated by arrow 2180. In example 2110, a partial outline view of a first end of vehicle 100 (e.g., a view along the direction of arrow 2176) shows several light emitters 1202 disposed at various locations on the exterior of vehicle 100. The first end of vehicle 100 can include light emitters 1202 (shown as 2101) that can perform multiple functions, such as visual alerts and / or signaling functions, e.g., brake lights, turn signals, hazard lights, headlights, running lights, etc. The light emitters 1202 can be disposed in various locations, including, but not limited to, support sections, roof 100u, doors, bumpers, wheels, wheel covers, wheel wells, hubcaps, and fenders. In some examples, one or more of the light emitters 1202 may be positioned behind an optically transparent surface or structure of the vehicle 100, such as, for example, behind a window, lens, covering, etc. Light L emitted by the light emitter 1202 may pass through the optically transparent surface or structure into the environment.

[0144] In example 2120, a second end of vehicle 100 (e.g., a view along the opposite direction of arrow 2176) can include light emitter 1202 that can be configured for vehicle signaling functionality (shown as 2103) and / or visual alert functionality. The light emitter 1202 shown in example 2120 can also be positioned in a variety of locations, including, but not limited to, for example, a support section, roof 100u, doors, bumpers, wheels, wheel covers, wheel wells, hubcaps, and fenders.

[0145] 22 shows contour views of further example light emitters 2210 and 2220 disposed on the exterior of autonomous vehicle 100. Autonomous vehicle 100 can be configured for driving operations in multiple directions, as indicated by arrow 2280. In example 2210, at a first end of vehicle 100 (e.g., a view along the direction of arrow 2276), light emitter 1202 having a circular shape can be configured exclusively for signaling functions, or can be configured for visual alerts and signaling functions (e.g., as determined by the planner system via instructions to the driving system). Similarly, in example 2220, at a second end of vehicle 100 (e.g., a view along the opposite direction of arrow 2276), light emitter 1202 having a circular shape can also be configured exclusively for signaling functions, or can be configured for visual alerts and signaling functions. The light emitters 1202 shown in examples 2210 and 2220 can also be positioned in a variety of locations, including, but not limited to, a support section, roof 100u, doors, bumpers, wheels, wheel covers, wheel wells, hubcaps, and fenders. The signaling functions of the circular-shaped light emitters 1202 can include, but are not limited to, brake lights, turn signals, hazard lights, headlights, and running lights. The shapes, sizes, locations, and numbers of the light emitters 1202 shown in Figures 21-22 are not limited to the examples shown.

[0146] 23 shows examples 2300 and 2350 of light emitter 1202 of autonomous vehicle 100. In example 2300, light emitter 1202 may be partitioned into subsections shown as 1202a-1202c by data representing light pattern 2310. Light pattern 2310 may include data representing subsections 2312a-2312c associated with subsections 1202a-1202c, respectively, of light emitter 1202. Light pattern 2310 may include data representing subpatterns 2314a-2314c associated with subsections 1202a-1202c of light emitter 1202. For example, light L emitted by each subsection may have a pattern determined by the data representing the subpattern associated with that subsection. As another example, data representing subsection 2312b determines subsection 1202b of light emitter 1202, and the light pattern emitted by subsection 1202b is determined by data representing subpattern 2314b.

[0147] In example 2350, light emitter 1202 may have an oval shape, and each emitter in light emitter 1202 may be partitioned into subsections shown as 1202d-1202g. Subsection 1202d may, for example, implement a headlight or a backup light for autonomous vehicle 100 (e.g., depending on the direction of travel). Subsection 1202e or 1202f may, for example, implement a turn signal. Subsection 1202g may, for example, implement a brake light. The light-emitting elements (e.g., E1-En in FIG. 12A ) may be individually addressable by circuitry and / or software according to data representing light pattern 2320. Thus, in some examples, subsections 1202d-1202g can perform signaling functions for vehicle 100 as determined by data representing subsections 2322d-2322g in light pattern 2320 and can perform signaling functions according to subpatterns 2324d-2324g. In other examples, data representing light pattern 2320 can reassign light emitter 1202 to perform a visual alert function. Performing a visual alert can include partitioning one or more emitter elements of the light emitter into subsections, each of which can have an associated subpattern. In another example, light emitter 1202 can include a subsection that performs a visual alert function and another subsection that performs a signaling function.

[0148] 24 illustrates example data 2400 representing a light pattern associated with a light emitter of autonomous vehicle 100. In example 2400, data representing light pattern 2401 may include one or more data fields 2402-2414 having data that can be decoded by decoder 2420 to generate data 2421 received by the driver, which is configured to drive one or more light-emitting elements (e.g., elements E1-En) of light emitter 1202.

[0149] The data representing light pattern 2401 may include, for example, but is not limited to, data representing light emitter 2402 (e.g., data for selecting a particular light emitter 1202), one or more light patterns 2404 to be applied to light emitter 1202, one or more subsections 2406 of light emitter 1202, one or more subpatterns 2408 to be applied to the one or more subsections, one or more colors 2410 of light (e.g., wavelengths of light) to be emitted by one or more light-emitting elements of light emitter 1202, intensities 2412 (e.g., luminous intensity in candelas) of light emitted by one or more light-emitting elements of light emitter 1202, and duty cycles 2414 to be applied to one or more light-emitting elements of light emitter 1202. The data included in the data representing light pattern 2401 may be in the form of, for example, a data structure or a data packet.

[0150] A decoder 2420 may receive data representing a light pattern 2401 for one or more light emitters 1202, as indicated by 2407, and decode the data into a data format 2421 that is received by a driver 2430. The driver 2430 may optionally receive data representing a modulation signal 2433 (e.g., from a wheel encoder) and may be configured to modulate the data 2421 with the modulation signal 2433 using a modulation function 2435. The modulation function may, for example, implement a light pattern that indicates that the vehicle 100 is slowing down, stopping, or some other driving maneuver of the vehicle 100. The driver 2430 may generate data 2431 that is configured to drive one or more light-emitting elements in one or more light emitters 1202. The light-emitting elements (e.g., E1-En) may be implemented using a variety of light sources, including, for example, but not limited to, light-emitting diodes (LEDs), organic light-emitting diodes (OLEOs), multi-color LEDs (e.g., RGB LEDs), or other light-emitting devices.

[0151] 25 shows an example of a flow diagram 2500 for implementing a visual display of directionality in autonomous vehicle 100. At stage 2502, data representing the trajectory of autonomous vehicle 100 in an environment external to vehicle 100 may be determined (e.g., using attitude data from a localizer system of vehicle 100). For example, data representing the location of autonomous vehicle 100 in the environment may be used to determine data representing the trajectory of autonomous vehicle 100.

[0152] In stage 2504, autonomous vehicle 100 may be propelled (e.g., by a driving system under control of the planner system) in a direction of travel that may be coextensive with the trajectory, with a portion of autonomous vehicle 100 (e.g., a first portion or a second portion) oriented in the direction of travel (e.g., facing the direction of travel). As an example, the portion may be a first portion of vehicle 100 (e.g., a first end of vehicle 100) or a second portion of vehicle 100 (e.g., a second end of vehicle 100). The driving system may command the steering system and the propulsion system to propel the first portion or the second portion in the direction of travel that is coextensive with the trajectory.

[0153] At stage 2506, data representing a directional light pattern configured to indicate a direction of travel associated with a direction of travel of the autonomous vehicle 100 may be selected. The directional light pattern may be accessed from a data store (e.g., memory, data storage, etc.) of the autonomous vehicle 100. There may be different directional light patterns associated with different directions of travel of the autonomous vehicle 100. The directional light pattern may be configured differently for different light emitters 1202 of the autonomous vehicle 100 (e.g., different sizes of each emitter, different locations of each emitter, differences in light-emitting area of ​​each emitter, different shapes of each emitter, etc.). In some examples, a different directional light pattern may be selected for different light emitters 1202 of the autonomous vehicle 100. In other examples, a selected directional light pattern may be selected for multiple light emitters 1202 of the autonomous vehicle 100 (e.g., a single directional light pattern for multiple light emitters 1202). As an example, data representing a directional light pattern may be accessed at stage 2506 from a data store 2501 containing data for directional light patterns.

[0154] At stage 2508, a light emitter 1202 of the autonomous vehicle 100 may be selected to emit light exhibiting a directional light pattern into the environment. In some examples, multiple light emitters 1202 may be selected and the same directional light pattern may be applied to each light emitter 1202. In other examples, multiple light emitters 1202 may be selected and different directional light patterns may be applied to some or all of the multiple light emitters 1202.

[0155] At stage 2510, light emitter 1202 may be caused to emit light to visually communicate the direction of travel in which autonomous vehicle 100 is traveling. In some examples, multiple light emitters 1202 may be caused to emit light to communicate the direction of travel in which autonomous vehicle 100 is traveling.

[0156] Flow 2500 may end after stage 2510 or may return to any other stage, such as back to stage 2502. Optionally, flow 2500 may perform stage 2512. In stage 2512, the direction of travel of the autonomous vehicle indicated by one or more light emitters 1202 may be locked. Locking the direction of travel while the autonomous vehicle 100 is navigating a trajectory may be performed to prevent the light emitters 1202 from emitting light in a direction opposite to the actual direction of travel of the vehicle 100 (e.g., indicating a direction of travel that is opposite to the actual direction of travel). For example, locking the indicated direction of travel to a first direction while driving in a first direction may be useful in preventing potential confusion to pedestrians and drivers of other vehicles that may occur if the light emitters 1202 of the autonomous vehicle 100 visually indicated an opposite direction of travel, such as a second direction that is opposite to the first direction.

[0157] At stage 2514, a determination may be made as to whether the maneuver of autonomous vehicle 100 is progressing (e.g., vehicle 100 is proceeding on a trajectory). If vehicle 100 is engaged in a maneuver, the YES branch may be taken back to stage 2512, where the direction of travel indicated by the light emitter may remain locked. On the other hand, if the maneuver of vehicle 100 is stopped, the NO branch may be taken to stage 2516. At stage 2516, the direction of travel indicated by light emitter 1202 may be unlocked. After being unlocked, flow 2500 may return to another stage, such as stage 2502, or may end. For example, if driving operations have ceased (e.g., vehicle 100 has stopped or parked to pick up or drop off passengers) and the direction of travel has been unlocked, vehicle 100 may resume driving operations and the direction of travel indicated by light emitter 1202 may remain the same or may be changed to indicate a different direction of travel.

[0158] 26 shows an example of a flow diagram 2600 for implementing a visual display of information in autonomous vehicle 100. At stage 2602, data representing the location of autonomous vehicle 100 in the environment may be determined (e.g., using attitude data from a localizer system). At stage 2604, data representing a light pattern configured to indicate information associated with autonomous vehicle 100 may be selected based on the location of autonomous vehicle 100. As an example, the data representing the light pattern may be accessed at stage 2604 from data store 2601 having data of light patterns. At stage 2606, light emitter 1202 of autonomous vehicle 100 may be selected to emit light indicative of the light pattern into the environment. At stage 2608, a determination may be made as to whether a trigger event has been detected (e.g., by one or more systems of vehicle 100, such as a planner system, a sensor system, a perception system, and a localizer system). If a trigger event has not been detected, the NO branch can be taken from stage 2608 and flow 2600 can return to stage 2608 to await detection of a trigger event. On the other hand, if a trigger event has been detected, the YES branch can be taken from stage 2608 to stage 2610.

[0159] At stage 2610, light emitter 202 may be caused to emit light to visually communicate information associated with autonomous vehicle 100. At stage 2612, a determination may be made as to whether a maneuver of autonomous vehicle 100 is progressing. If the maneuver is progressing (e.g., vehicle 100 is moving along a trajectory), the YES branch may be taken from stage 2612 to stage 2614. At stage 2614, emission of light from light emitter 1202 may be stopped (e.g., to prevent the information from confusing other drivers and / or pedestrians). If the maneuver is not progressing, the NO branch may be taken from stage 2612 back to stage 2610, where light emitter 1202 may continue to emit light to visually communicate information associated with autonomous vehicle 100.

[0160] In flow 2600, the information represented by the data representing the light pattern associated with the autonomous vehicle 100 can include, but is not limited to, client / customer / user / occupant-created content, graphics, greetings, notices, warnings, images, videos, text, messages, or other forms of information that can be displayed by one or more of the light emitters 1202. As one example, the data representing the light pattern associated with the autonomous vehicle 100 can include information to identify the autonomous vehicle 100 as intended to serve a particular occupant using graphics or other visual information recognizable by the occupant. Further to this example, the light emitter 1202 can display a passenger-specific image that readily identifies the vehicle 100 as assigned to serve the occupant associated with the passenger-specific image (e.g., the occupant who may have created the passenger-specific image). As another example, the data representing the light pattern associated with the autonomous vehicle 100 can include a passenger-specific greeting or message displayed by the light emitter 1202. A passenger who visually perceives the greeting / message can understand that the autonomous vehicle 100 presenting the greeting / message is assigned to provide a service to the passenger (e.g., provide transportation for the passenger).

[0161] The trigger event that may be detected in stage 2608 may be, for example, location-based. As a first example, if vehicle 100 is assigned to pick up a passenger at a predetermined location, then upon arrival at the predetermined location, data representing the predetermined location may be compared (e.g., using a planner system, a localizer system, a perception system, and a sensor system) with data representing the location of vehicle 100 in the environment. If a match is determined between the predetermined location and the location of vehicle 100, then a trigger event has been detected (e.g., the vehicle has arrived at a location that matches the predetermined location).

[0162] As a second example, autonomous vehicle 100 can be configured to detect wireless communications (e.g., using COMS 880 in FIG. 8 ) from a wireless computing device (e.g., a client / customer / passenger smartphone or tablet). Vehicle 100 can be configured to intercept or otherwise listen for wireless communications from other devices, for example, upon arrival at a location. In another example, autonomous vehicle 100 can be configured to wirelessly transmit a signal configured to notify an external wireless device (e.g., a smartphone) that vehicle 100 has arrived at or is otherwise in its vicinity. Vehicle 100 and / or the external wireless device can wirelessly exchange access credentials (e.g., Service Set Identifier—SSID, MAC address, username and password, email address, biometric identifier, etc.). Vehicle 100 may require proximity between vehicle 100 and the external wireless device (e.g., determined by RSSI or other radio frequency metric), for example, via a Bluetooth protocol or an NFC protocol. Verification of the access credentials can be a triggering event.

[0163] FIG. 27 shows examples 2701 and 2702 of visual indications of directionality of travel by light emitters of autonomous vehicle 100. In examples 2701 and 2702, line 2721 demarcates a first portion of autonomous vehicle 100 (e.g., occupying quadrants 3 and 4 in FIGS. 10A-10B ) and a second portion of autonomous vehicle 100 (e.g., occupying quadrants 1 and 2 in FIGS. 10A-10B ). In example 2710, vehicle 100 is propelled along trajectory Tav (e.g., in the direction of arrow A). Light emitters 1202 may be positioned in quadrants 1, 3, and 2 (e.g., external to vehicle 100), and a subset of these light emitters 1202 may be selected to emit light to visually communicate the direction of travel of vehicle 100. The selected light emitter is shown as 12025. The data representing the light pattern applied to the light emitter 12025 may implement a light pattern of one or more arrows pointing and / or moving in the direction of travel to which the first portion is directed. In example 2701, the selected light emitter 12025 may be positioned on a first side of the vehicle 100 spanning the first and second portions (e.g., in quadrants 2 and 3).

[0164] In example 2702, vehicle 100 is shown rotated approximately 90 degrees clockwise to show another subset of selected light emitters located on a second side of vehicle 100 across first and second portions (e.g., in quadrants 1 and 4). Similar to example 2701, in example 2702, the data representing the light pattern applied to light emitters 12025 may implement a light pattern of one or more arrows pointing and / or moving in the direction of travel in which the first portion is directed.

[0165] 28 shows further examples 2801 and 2802 of visual indication of directionality of travel by light emitters of autonomous vehicle 100. In contrast to the example of FIG. 27, examples 2801 and 2802 show vehicle 100 having a trajectory Tav (e.g., along line B) with a second portion oriented in the direction of travel. Selected light emitters 12025 on a first side of vehicle 100 (e.g., in example 2801) and selected emitters 12025 on a second side of vehicle 100 (e.g., in example 2801) can be configured to implement one or more arrow light patterns pointing and / or moving in the direction of travel of vehicle 100.

[0166] FIG. 29 shows further example visual indications 2901 and 2902 of the direction of travel by light emitters of autonomous vehicle 100. In the example of FIG. 29, vehicle 100 can be configured to be propelled in a direction other than that shown in FIGS. 27 and 28, such as along trajectory Tav along the direction of arrow C. For example, the driving system of autonomous vehicle 100 can be instructed (e.g., by a planner system) via the steering system to orient each wheel (e.g., wheel 852 in FIG. 8) at an angle that implements a direction of travel along trajectory Tav (e.g., along the direction of arrow C). Light emitters 1202 that may be selected in FIGS. 27 and 28 can be deselected in examples 2901 and 2902 to select another subset of light emitters 1202s located at the end of the first portion (e.g., first end) and the end of the second portion (e.g., second end). The data representing the light pattern can be configured to implement (eg, at selected light emitters 1202s) arrows pointing and / or moving in the direction of travel described above.

[0167] FIG. 30 shows further example visual indications 3001 and 3002 of the direction of travel by light emitters of autonomous vehicle 100. In FIG. 30, examples 3001 and 3002 indicate that the vehicle's trajectory T is along the direction of arrow D and that selected light emitters are activated to emit light indicating the direction of travel. In the examples of FIGS. 27-30, each light emitter 1202 (e.g., including any selected light emitters 1202s) may be symmetrically positioned with respect to the first and second portions of vehicle 100. For example, each light emitter 1202 may be symmetrically positioned on a side of vehicle 100 with respect to the first and second portions of vehicle 100 and / or may be symmetrically positioned at an end of vehicle 100 with respect to the first and second portions of vehicle 100. The actual shape, size, configuration, location, and orientation of light emitters 1202 are not limited by the examples shown in FIGS. 27-30.

[0168] FIG. 31 shows examples 3100 and 3150 of visual displays of information by light emitters of autonomous vehicle 100. In example 3100, autonomous vehicle 100 is autonomously proceeding to hotel 3131 (e.g., having location (x, y)) to pick up passenger 3121 at hotel edge 3133. Autonomous vehicle 100 can determine its location in the environment (e.g., using a localizer system and / or a planner system) and, based on the location, can select data representing a light pattern (e.g., a pattern not related to directionality of travel) configured to indicate information associated with autonomous vehicle 100. Light emitters 1202 of autonomous vehicle 100 can be selected to emit light indicative of the light pattern. In some examples, a trigger event can be detected by autonomous vehicle 100 prior to emitting light indicative of the light pattern from one or more of light emitters 1202. The occurrence of the trigger event can cause the selected light emitters 1202s to emit light indicative of the light pattern.

[0169] As an example, a trigger event may be determined based on the location of the vehicle 100 relative to another location, such as the location of the hotel 3131 (e.g., (x,y)). The planner system may compare that location (e.g., GPS / IMU data, map tile data, attitude data, etc.) with data representing the location of the hotel 3131 to determine if there is a match. An exact match may not be necessary, as an indication that the vehicle is located near the location of the hotel 3131 may be sufficient to determine that the conditions for satisfying the trigger event are met. Other data, such as sensor data from a sensor system processed by a perception system, may be used to determine that objects in the environment surrounding the vehicle 100 (e.g., hotel signage, hotel building, road signs, road markings, etc.) are consistent with the vehicle 100 being at a hotel.

[0170] As another example, the trigger event may include the vehicle 100 and / or the passenger's computing device 3122 wirelessly 3123 communicating with and / or sensing each other to determine information indicative of the trigger event, such as an exchange of access credentials or a predetermined code or a handshake of a signal indicating that the wireless communication 3123 confirms that the computing device 3122 is associated with the passenger 3122 that the vehicle 100 is assigned to transport.

[0171] In example 3100, the light pattern emitted by the selected light emitter 12025 can be associated with an image (e.g., a winking image 3101) that the passenger 3121 expects to be presented to identify the vehicle 100 as the vehicle dispatched to service the transportation needs of the passenger 3121. Other information can also be presented on the selected light emitter 12025, with the winking image 3101 being a non-limiting example of information that can be presented.

[0172] The passenger 3121 may have a light pattern 3161 stored with or otherwise associated with a passenger profile or other data associated with the passenger 3121. In some examples, the passenger 3121 may create or access a preferred image or other information to be displayed by one or more of the light emitters 1202. As an example, an application APP 3124 running on a computing device 3122 (e.g., a smartphone or tablet) may be used to create, select, or access data representing the light pattern 3161. The computing device 3122 may communicate 3151 the light pattern 3161 to an external resource 3150 (e.g., the cloud, the internet, a data warehouse, etc.). The vehicle 100 may access 3153 the light pattern 3161 from the external resource 3150 (e.g., access the light pattern 3161 and store the light pattern 3161 locally in a memory of the vehicle 100).

[0173] In other examples, data (e.g., a digital audio file) representing sound pattern 3163 can be selected and audibly presented 3103 using the audio capabilities of vehicle 100, such as playing sound pattern 3163 via a loudspeaker (e.g., speaker 829 in FIG. 8 ). Data representing sound pattern 3163 can be created and accessed in a similar manner as described above with respect to data representing light pattern 3161.

[0174] In example 3150, the opposite side of vehicle 100 is shown. Light emitters at sides, ends, and other locations of vehicle 100 can be positioned symmetrically relative to first and second portions, or other reference points (e.g., 100r) on vehicle 100. Selected light emitters 1202s on the opposite side of vehicle 100 can emit light indicative of light pattern 3161, as described above with reference to example 3100. A sound indicative of sound pattern 3163 can also be audibly presented 3103 on the other side of vehicle 100. As an example, data representing sound pattern 3163 can include the message, "Your ride to the airport is here! Please board!" Vehicle 100 may include doors located on both sides of vehicle 100, and the presentation of light and / or sound patterns on both sides of the vehicle may be configured to notify (e.g., visually and / or audibly) occupants who may be on either side of vehicle 100 that their vehicle 100 has arrived and is identified as the vehicle assigned to them.

[0175] Light pattern 3161 and sound pattern 3163 are non-limiting examples of content that may be presented by vehicle 100. In some examples, the content may be created or selected, such as by a passenger. APP 3124 may be configured to perform content creation and / or content selection. Light emitter 1202 may be selected to perform one or more functions, including but not limited to, communicating direction of travel and / or communicating information associated with autonomous vehicle 100. In other examples, light emitter 1202 may be selected to perform a visual alert during a driving maneuver (e.g., while the vehicle is navigating a trajectory).

[0176] 32 illustrates an example light emitter arrangement 3200 on an autonomous vehicle 100. In example 3200, light emitters 1202 may be disposed, for example, on roof 100u of vehicle 100 (e.g., to communicate information and / or driving directions to an overhead viewer), on wheels 852 of vehicle 100 (e.g., on hubcaps or wheel covers), or behind optically transparent structures 3221, 3223 (e.g., behind windows of vehicle 100). The light emitters 1202 shown in example 3200 may be disposed symmetrically relative to other light emitters 1202 (not shown) on vehicle 100. One or more of the light emitters 1202 may emit light having a pattern of arrows 3251 that, for example, moves, strobes, or otherwise visually communicates that the autonomous vehicle 100 is traveling in a direction associated with trajectory Tav. Arrow 3251 can be static or dynamic (eg, strobed across light emitter 1202 in the direction of travel).

[0177] 33 illustrates an example light emitter arrangement 3300 in an autonomous vehicle. In example 3300, light emitter 1202 may be disposed on a wheel well cover 3350 coupled to vehicle 100. The light emitter 1202 shown in example 3300 may be disposed symmetrically relative to other light emitters 1202 (not shown) on vehicle 100. The wheel well cover 3350 may be configured to be removable from vehicle 100. During non-driving operation of vehicle 100 (e.g., when stopped to pick up or drop off passengers), light emitters 1202 on wheel well cover 3350 and / or other locations on vehicle 100 may be configured to provide courtesy lighting (e.g., illuminating the ground or area around vehicle 100 for ingress and egress at night or inclement weather). Each light emitter 1202 can include several light-emitting elements E (e.g., E1-En in FIG. 24 ), and each element E can be individually addressable (e.g., by driver 2430 in FIG. 24 ) to control, for example, the intensity of the emitted light, the color of the emitted light, and the duration of the emitted light. For example, each element E can have a specific address within light emitter 1202 defined by a row and column address. In example 3300, a darkened element E within light emitter 1202 can visually communicate the direction of travel using the image of an arrow 3351 pointed along the trajectory Tav of autonomous vehicle 100. Arrow 3351 can be static or dynamic (e.g., strobed across light emitter 1202 in the direction of travel).

[0178] Each light emitter 1202 can be partitioned into one or more partitions or portions, and the elements E in each partition are individually addressable. Data representing subsections and / or subpatterns (e.g., 1202a-1202c in FIG. 23) can be applied to the emitters E in one or more partitions. The data representing the light pattern, the data representing the directional light pattern, or both can include one or more data fields (e.g., 2402-2414 in FIG. 24) having data that can be decoded by a decoder to generate data received by a driver configured to drive one or more light-emitting elements E of the light emitter 1202. For example, image data related to the data representing the light pattern 3161 (e.g., related to the winking image 3101) can be included in one of the data fields. As another example, the data representing the directional light pattern (e.g., from data store 2501 in FIG. 25) can include a data field related to a light color associated with a direction of travel (e.g., orange for a first direction and purple for a second direction). The data representing the directional light pattern may include a data field regarding the strobe rate or rate of movement of the light emitted by the light emitter 1202 (e.g., the movement of the arrow 3351 or other image or icon indicating the direction of travel shown in Figures 27-30).

[0179] Although the foregoing examples have been described in some detail for purposes of clarity of understanding, the techniques of the above-described concepts are not limited to the details provided. There are many alternative ways of implementing the techniques of the above-described concepts. The disclosed examples are illustrative and not limiting. [Explanation of symbols]

[0180] 100 autonomous vehicles 100e external 100i internal 101 Autonomous Vehicle Systems 105 orbit 105a Collision avoidance trajectory 112 Route Calculator 114 Object Data Calculator 115 Kinematics Calculator 116 Collision Predictor 118 Object Classification Determinator 119 Object Type Datastore 120 Safety System Activator 122 Internal Safety System 124 External Safety Systems 126 Driving System 132 Sensor Data 139 Vehicle Location Data 180 objects 185 orbit 187 Collision 190 Environment

Claims

1. accessing sensor data generated by a sensor system of the autonomous vehicle; determining a location of the autonomous vehicle within an environment based at least in part on the sensor data; and calculating a trajectory of the autonomous vehicle based at least in part on the location of the autonomous vehicle and based at least in part on the sensor data; identifying objects in the environment based at least in part on the sensor data; determining a location of the object in the environment; determining an orientation of the autonomous vehicle relative to the object based on the calculated trajectory of the autonomous vehicle; determining to provide a visual alert based at least in part on the location of the object and the location of the autonomous vehicle; selecting a light pattern from a plurality of light patterns; selecting a light emitter from a plurality of light emitters of the autonomous vehicle to provide the visual alert based on the determined orientation of the autonomous vehicle; causing the selected light emitter to provide the visual alert, the selected light emitter emitting light indicative of the light pattern into the environment; and wherein, if the object is approaching the autonomous vehicle, a light emitter pointing generally in the direction of the object's approach is activated to emit the light to visually notify the object that the autonomous vehicle is slowing down.

2. the plurality of light emitters include a first light emitter disposed in a first area between a first wheel of the autonomous vehicle and a second wheel of the autonomous vehicle, a second light emitter disposed in a second area between a third wheel of the autonomous vehicle and a fourth wheel of the autonomous vehicle, a third light emitter disposed in a third area between the first wheel and the third wheel, and a fourth light emitter disposed in a fourth area between the second wheel and the fourth wheel, and the method further comprises: selecting the first light emitter from the plurality of light emitters to provide the visual alert based on the location of the object; The method of claim 1 further comprising:

3. estimating a threshold event associated with causing the light emitter to provide the visual alert based at least in part on the location of the object and the location of the autonomous vehicle; detecting an occurrence of said threshold event; Furthermore, 10. The method of claim 1, wherein causing the light emitter of the autonomous vehicle to provide the visual alert is based at least in part on the occurrence of the threshold event.

4. calculating a distance between the autonomous vehicle and the object based at least in part on the location of the object and the location of the autonomous vehicle. The method of claim 1 , wherein the light pattern is based at least in part on the distance.

5. 10. The method of claim 1, wherein selecting the light pattern is based at least in part on one or more of a threshold distance or a threshold time, wherein the threshold distance is determined based on data representing a distance between the autonomous vehicle and the object, the data representing the distance between the autonomous vehicle and the object being calculated based on data representing the location of the autonomous vehicle and data representing the location of the object, the threshold distance being less than the distance between the autonomous vehicle and the object, and the threshold time is determined based on data representing a time associated with the location of the autonomous vehicle and the location of the object, and is less than the time associated with the location of the autonomous vehicle and the location of the object coinciding with one another.

6. calculating, based at least in part on the location of the object and the trajectory of the autonomous vehicle, a time associated with the location of the autonomous vehicle and the location of the object coinciding with one another; 10. The method of claim 1, wherein causing the light emitter of the autonomous vehicle to provide the visual alert is based at least in part on the time.

7. determining an object classification for the object determined from a plurality of object classifications, the object classifications including a static pedestrian object classification, a dynamic pedestrian object classification, a static vehicle object classification, and a dynamic vehicle object classification; The method of claim 1 , wherein selecting the light pattern is based at least in part on the object classification.

8. accessing map data associated with the environment; determining position data and orientation data associated with the autonomous vehicle; and Furthermore, 2. The method of claim 1, wherein determining the location of the autonomous vehicle within the environment is based on the map data, the position data, and the orientation data.

9. selecting a different light pattern from the plurality of light patterns based at least in part on a first location of the object before the visual alert is provided and a second location of the object after the visual alert is provided; causing the light emitter to provide a second visual alert, the light emitter emitting light into the environment indicating the different light pattern; The method of claim 1 further comprising:

10. 10. The method of claim 1, wherein the light emitter includes subsections, and the light pattern includes sub-patterns associated with the subsections, the subsections configured to emit light indicative of the sub-patterns, at least one of the sub-patterns indicative of one or more of a signaling function of the autonomous vehicle or a braking function of the autonomous vehicle, and at least one other sub-pattern indicative of the visual alert.

11. receiving data representing a sensor signal indicative of a degree of rotation of a wheel of the autonomous vehicle; modulating the light pattern based at least in part on the degree of rotation; and The method of claim 1 further comprising:

12. 10. The method of claim 1, wherein selecting the light emitter from the plurality of light emitters of the autonomous vehicle to provide the visual alert is further based in part on the location of the autonomous vehicle and the location of the object.

13. a sensor operative to generate sensor data for an autonomous vehicle positioned within an environment, the sensor including a LIDAR sensor; a light emitter operative to emit light from the autonomous vehicle into the environment; determining a location of the autonomous vehicle within the environment based at least in part on some of the sensor data; calculating a trajectory of the autonomous vehicle based at least in part on the location of the autonomous vehicle and based at least in part on the sensor data; identifying an object in the environment based at least in part on a portion of the sensor data; determining a location of the object; determining an orientation of the autonomous vehicle relative to the object based on the calculated trajectory of the autonomous vehicle; selecting a light pattern from a plurality of light patterns based at least in part on a portion of the sensor data; selecting, from the light emitters, a light emitter that provides a visual alert based on the determined orientation of the autonomous vehicle; and causing the selected light emitter to provide the visual alert by emitting light indicative of the light pattern into the environment. one or more processors configured to perform operations including and wherein, when the object is approaching the autonomous vehicle, a light emitter pointing generally in the direction of the object's approach is activated to emit the light to visually notify the object that the autonomous vehicle is slowing down.

14. The system of claim 13, wherein light emitters that may not be visible to the object are not activated.

15. 14. The system of claim 13, wherein causing the light emitter to provide the visual alert comprises causing the light emitter to emit light indicative of the light pattern into the environment for a first period of time, and wherein the operating further comprises causing the light emitter to emit light indicative of a signaling function into the environment for a second period of time.

16. 14. The system of claim 13, wherein the light emitters include a first light emitter positioned at a first end of the autonomous vehicle, a second light emitter positioned at a second end of the autonomous vehicle, a third light emitter positioned at a first side of the autonomous vehicle, and a fourth light emitter positioned at a second side of the autonomous vehicle.

17. 14. The system of claim 13, wherein a first one of the light emitters includes a first subsection and a second subsection, the first subsection operative to emit the light indicative of the light pattern into the environment, and the second subsection operative to emit a second light indicative of a signaling function.

Images