Method for robotic vehicle to communicate with external environment via acoustic beamforming

Abstract

Systems and methods implemented in algorithms, software, firmware, logic, or circuits may be configured to process data to determine whether an object external to an autonomous vehicle is a person (e.g., such as a pedestrian) or another classification (e.g., such as a vehicle), and may be further configured to determine the position of the person relative to the autonomous vehicle. The logic may be configured to direct acoustic energy (e.g., via vehicle acoustic beamforming) to an object external to the autonomous vehicle as an audible sound alert. The vehicle-related acoustic beam may be directed to a driver in another vehicle. The logic may be configured to track the movement of external objects, such as a pedestrian crossing from one side of a street to the other side, and may direct the direction of one or more vehicle-related acoustic beams accordingly to track the movement of the person.

Description

[0001] This application is a divisional application of an invention patent application with an application date of November 2, 2016, application number 201680064195.3, and invention name “Method for communicating between a robot vehicle and an external environment through acoustic beamforming”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is a continuation of U.S. patent application No. 14 / 756,993, filed on November 4, 2015, entitled “Method for robotic vehicle communication with an external environment via acoustic beam forming,” which is related to U.S. patent application No. 14 / 932,959, filed on November 4, 2015, entitled “Autonomous Vehicle Fleet Service And System,” U.S. patent application No. 14 / 932,963, filed on November 4, 2015, entitled “Adaptive Mapping To Navigate Autonomous Vehicles Responsive To Physical Environment Changes,” and U.S. patent application No. 14 / 932,962, filed on November 4, 2015, entitled “Robotic Vehicle Active Safety Systems And Methods,” the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0004] Embodiments of the present invention generally relate to methods, systems, and apparatus for safety systems in robotic vehicles. Background Art

[0005] An autonomous vehicle, such as the type configured to transport passengers in an urban environment, may encounter many situations in which the autonomous vehicle should alert persons, vehicles, etc., of its presence in order to avoid a possible collision or in which the vehicle comes within an unsafe distance of an external object (e.g., a pedestrian).

[0006] As an example, in a conventional vehicle driven by a human, a pedestrian crossing the road in front of the vehicle may not obey traffic rules or may not notice the approach of the vehicle. In some cases, the driver of the vehicle may decide to use the horn of the vehicle to alert the pedestrian. However, the horn will usually have a suboptimal sound radiation pattern and may not be enough to warn the pedestrian that the horn being sounded is directed at him / her. On the contrary, other pedestrians or other drivers near the vehicle may think that the horn is sounding at them. Moreover, the pattern of the sound waves emitted by the horn may be difficult to locate the source of the horn. From the perspective of the pedestrian, the horn may be perceived as coming from another vehicle. In addition, the horn may cause the pedestrian to look along a direction different from the direction of the vehicle generating the horn sound, thereby possibly distracting the pedestrian from the actual horn source.

[0007] Finally, autonomous vehicles may share the road with other vehicles and people. However, autonomous vehicles may be difficult to detect audibly due to the low level of emitted noise from electric and / or hybrid propulsion systems (e.g., lack of internal combustion engine noise and / or lower level tire noise).

[0008] Thus, in situations where an autonomous vehicle is intended to sound an audible alarm, a more targeted and / or socially acceptable sound source (e.g., less rude than a horn) may be desired. Thus, there is a need for systems, devices, and methods for implementing focused acoustic alarms from robotic vehicles. Summary of the invention

[0009] The present application discloses a system for implementing an acoustic beam steering array, comprising: an acoustic beam steering array including a plurality of speakers configured to output directional audio; and one or more processors configured to perform operations, the operations comprising: receiving a first signal indicating an object type and an object location, and optionally a first signal indicating an object trajectory or a predicted intersection of an object trajectory and a vehicle trajectory, the first signal being associated with a first object; determining a direction in which an acoustic energy beam is emitted via at least a portion of the plurality of speakers of the acoustic beam steering array based at least in part on the first signal; receiving a second signal indicating a second object, the second object being different from the first object; and at least in part on the first signal. Based on the first signal and the second signal, determine a first gain setting associated with an acoustic energy beam emitted by a first part of the at least a part of the plurality of speakers and a second gain setting associated with an acoustic energy beam emitted by a second part of the at least a part of the plurality of speakers, wherein the first gain setting is different from the second gain setting; and cause the at least a part of the plurality of speakers to emit an acoustic energy beam indicating an alarm in the direction and with the first gain setting and the second gain setting, wherein the first gain setting and the second gain setting are configured to cause a sound pressure of the acoustic energy beam at a first object to be different from a sound pressure of the acoustic energy beam at a second object. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Various embodiments or examples ("examples") are disclosed in the following detailed description and accompanying drawings:

[0011] Figure 1 An example of a system for implementing an acoustic beam steering array in an autonomous vehicle is described;

[0012] Figure 2A depicts an example of a flow chart for implementing acoustic beam steering in an autonomous vehicle;

[0013] Figure 2B Another example of a flow chart for implementing acoustic beam steering in an autonomous vehicle is depicted;

[0014] Figure 3A Another example of a system for implementing an acoustic beam steering array in an autonomous vehicle is depicted;

[0015] Figure 3B Yet another example of a system for implementing an acoustic beamsteering array in an autonomous vehicle is depicted;

[0016] Figure 4 An example of a flow chart for implementing a perception system is depicted;

[0017] Figure 5 depicts an example of object prioritization by a planner system in an autonomous vehicle;

[0018] Figure 6 A top plan view depicting one example of acoustic beam steering from an acoustic beam steering array in an autonomous vehicle;

[0019] Figure 7 depicts an example of a flow chart for implementing a planner system;

[0020] Figure 8 An example of a block diagram of a system in an autonomous vehicle with an acoustic beam steering array is depicted;

[0021] Fig. 9 depicts a top view of one example of an acoustic beam steering array positioned on an autonomous vehicle;

[0022] Fig. 10A depicts top plan views of two examples of sensor system coverage in an autonomous vehicle;

[0023] Fig. 10B depicts top plan views of two additional examples of sensor system coverage in an autonomous vehicle;

[0024] Fig.11A top plan view depicting one example of an acoustic beam steering array for directing acoustic energy at an autonomous vehicle in proximity to the vehicle;

[0025] Fig.12 An example of a flow chart for implementing acoustic beam steering in an acoustic beam steering array is depicted;

[0026] Fig.13 Another example of a flow chart for implementing acoustic beam steering in an acoustic beam steering array is depicted;

[0027] Fig.14 An example of a flow chart for implementing adaptive acoustic beam steering in an acoustic beam steering array of an autonomous vehicle is depicted;

[0028] Fig.15 An example of a block diagram of an acoustic beam steering array is depicted;

[0029] Fig.16 depicts an example of array shadowing in an acoustic beam steering array; and

[0030] Fig.17 An example of a loudspeaker and a loudspeaker housing in an acoustic beamsteering array is depicted;

[0031] Although the above drawings depict various examples of the present invention, the present invention is not limited to the depicted examples. It should be understood that in the drawings, the same reference numerals represent the same structural elements. And it should be understood that the drawings are not necessarily to scale. DETAILED DESCRIPTION

[0032] Various embodiments or examples may be implemented in a variety of ways, including as a system, process, method, device, user interface, software, firmware, logic, circuit, or a series of executable program instructions embedded in a non-transitory computer-readable medium. The program instructions are sent to a non-transitory computer-readable medium or a computer network via an optical, electronic, or wireless communication link and stored or otherwise fixed in a non-transitory computer-readable medium. For example, examples of non-transitory computer-readable media include, but are not limited to, electronic memory, RAM, DRAM, SRAM, ROM, EEPROM, flash memory, solid-state memory, hard drive, volatile and non-volatile memory. One or more non-transitory computer-readable media may be distributed on multiple devices. In general, unless otherwise specified in the claims, the operations of the disclosed process may be performed in any order.

[0033] The following together with the attached Figure 1A detailed description of one or more examples is provided. Detailed descriptions are provided about these examples, but these detailed descriptions are not limited to any particular examples. The scope is limited only by the claims and encompasses a variety of alternatives, modifications, and equivalents. Many specific details are presented in the following description in order to provide a thorough understanding. These details are provided for the purpose of example and the described technology can be practiced according to the claims without some or all of these specific details. For the sake of clarity, technical materials that are well known in the technical field related to the examples are not described in detail to avoid making the description unnecessarily vague.

[0034] Figure 1 An example of a system for implementing an acoustic beam steering array in an autonomous vehicle is depicted 199. Figure 1 , autonomous vehicle 100 may include planner system 110, sensor system 120, localizer system 130, and perception system 140. Planner system 110 may receive data from localizer system 130 and perception system 140. Sensor system 120 may sense (e.g., 121) an environment 190 external to autonomous vehicle 100. Sensor data from sensor system 120 may be received by localizer system 130 and perception system 140. Localizer system 130 may use sensor data from sensor system 120 and may use other data (e.g., map data) to implement a localization process that determines the location of the autonomous vehicle in environment 190. Perception system 140 may use sensor data from sensor system 120 to implement a perception process that determines where an object is located in environment 190 (e.g., the location and / or coordinates of the object). Object data from perception system 140 and position data (e.g., position and orientation (pose) data) from positioning system 130 may be received by planner system 110 to perform one or more functions, such as driving operations of autonomous vehicle 100. Driving operations may include, but are not limited to, calculating and controlling a trajectory of autonomous vehicle 100, generating steering inputs to a steering system of autonomous vehicle 100, controlling a braking system of autonomous vehicle 100, controlling a safety system of autonomous vehicle 100, controlling a propulsion system of autonomous vehicle 100, and broadcasting an acoustic alert into environment 190 external to autonomous vehicle 100 using an acoustic beam steering array 102 positioned on autonomous vehicle 100 (e.g., located on the exterior of vehicle 100).

[0035] Objects positioned in environment 190 may be detected by perception system 140 using sensor data generated by sensor system 120. As an example, one or more sensors (e.g., a set of different sensor types) in sensor system 120 may detect (using active and / or passive sensing) object 180 positioned in environment 190. Perception system 140 may process the sensor data to detect object 180, classify object 180 (e.g., as bicycle 183 and rider 182), determine object tracking associated with object 180 (e.g., determine whether object 180 is a static object (not in motion) or a dynamic object (in motion)), the location of object 180 in environment 190 (e.g., a location or coordinates relative to the location of autonomous vehicle 100), and may track object 180 (e.g., track changes in the location of object 180). Although Figure 1 A single object 180 is depicted, but environment 190 may include many other objects (not shown) including, for example, but not limited to, pedestrians, motor vehicles, roads, traffic lights, street signs, buildings, road markings, trees, fire hydrants, and other on-road and off-road infrastructure.

[0036] exist Figure 1 In the example of , object 180 is depicted as having a trajectory 185 (e.g., a changed position in environment 190) that may conflict with trajectory 105 (Tav) of autonomous vehicle 100. A conflicting trajectory between an object in external environment 190 and autonomous vehicle 100 may be, but need not always be, a trajectory that may result in a collision between autonomous vehicle 100 and the object. A conflicting trajectory may include an object trajectory that, if not altered, may result in autonomous vehicle 100 and the object coming into an unsafe proximity (e.g., a distance of about two feet or less) of each other. As another example, a conflicting trajectory may include an object trajectory that, if not altered, may result in the trajectories crossing each other at a location in the path of travel of autonomous vehicle 100, the object, or both in environment 190.

[0037] Planner system 110 may receive pose data from localizer system 130 and object data from perception system 140 and may process the received data (e.g., using one or more computational engines and / or algorithms) to calculate (e.g., at trajectory calculator 112) a trajectory of autonomous vehicle 100 in environment 190, predict (e.g., at object motion predictor 116) the motion of object 180 in environment 190, calculate (e.g., at object coordinate calculator 114) coordinates of object 180 in environment 190 (e.g., based on data representing the position of the object in the environment), calculate (e.g., at threshold position estimator 118) a threshold position in the environment to trigger an acoustic alarm, compare (e.g., at position comparator 113) the position of object 180 to the threshold position and trigger an acoustic alarm when the position of object 180 and the threshold position match (e.g., are consistent with each other), and select (e.g., using audio signal selector 111) an audio signal (e.g., taken from audio signal data 117) for the acoustic alarm. The planner system 110 may use data representing the location of objects in the environment, data representing the coordinates of objects in the environment, or both.

[0038] Object motion predictor 116 may be configured to implement motion prediction based on one or more data included in the object data. For example, object motion predictor 116 may process data representing object classification to determine data representing the type of object (e.g., skateboard rider, ambulance, truck, wheelchair, etc.) and may process data representing the location of the object in environment 190 to predict data representing the predicted motion of the object. As an example, a runner jogging along a bicycle path on a street may be predicted to be more likely (e.g., >50%) to change its trajectory from the bicycle path and enter the street, thereby causing a potential trajectory conflict with autonomous vehicle 100. As another example, a runner jogging along a sidewalk parallel to the street may be predicted to be less likely (e.g., <50%) to change its trajectory from the sidewalk and enter the street; thereby reducing the likelihood of a conflicting trajectory with autonomous vehicle 100.

[0039] Object coordinate calculator 114 may be configured to process data representing the location of an object in an environment to calculate the coordinates of the object. For example, the calculated coordinates may be based on two or more Cartesian coordinates (e.g., X and Y), polar coordinates, or angles. The coordinates calculated by object coordinate calculator 114 may be relative to the location of autonomous vehicle 100 and / or a reference location on autonomous vehicle 100, such as a reference point on acoustic beam steering array 102.

[0040] Autonomous vehicle trajectory calculator 112 may be configured to receive data representing the position of autonomous vehicle 100 in environment 190, such as the aforementioned position and orientation data (pose data). Threshold position estimator 118 may receive data representing the position of autonomous vehicle 100 and data representing predicted motion of an object to estimate data representing a threshold position in the environment to trigger an acoustic alarm using acoustic beam steering array 102.

[0041] The position comparator 113 may receive data representing a threshold position and data representing a position of an object to determine when the data representing the threshold position and the data representing the position of the object match (e.g., are consistent with each other) and may generate data representing a trigger signal when the two positions match.

[0042] The audio signal selector 111 can select data representing the audio signal to be used to generate an acoustic alarm using the array 102. In some examples, one or more audio signals can be selected by the audio signal selector 111. For example, the audio signal selector 111 can select multiple audio signals, wherein each selected audio signal represents a different threat level. The threat level can be based on the distance of the object from the autonomous vehicle 100. The closer the object is to the autonomous vehicle 100, the higher the probability that a collision may occur and the higher the threat level that can be audibly conveyed by the selected audio signal. The threshold position estimator 118 can estimate more than one threshold position in the environment 190, and the audio signal selector 111 can select data representing different audio signals for each threshold position. For example, if three threshold positions are estimated, the audio signal selector 111 can select three different audio signals for each of the three threshold positions. As described above, each audio signal can be configured to convey a different threat level.

[0043] Planner system 110 may output data representing an audio signal, data representing a trigger signal, and data representing a location (or coordinates) of an object (e.g., object 180) to acoustic beam steering array 102. Acoustic beam steering array 102 may be configured to transmit a guided acoustic energy beam 104 indicative of data representing an audio signal (e.g., acoustic energy reproducing the sound encoded in the audio signal). Guided acoustic energy beam 104 may have a propagation direction 106 determined by the data representing the coordinates of object 180 or by the data representing the location of the object. For example, the data representing the coordinates of object 180 may be angular, polar, Cartesian, or other coordinate systems.

[0044] exist Figure 11, data representing the coordinates of object 180 are depicted as angle β (Beta). Angle β may be calculated relative to trajectory 105 of autonomous vehicle 100 and may be referenced to a reference location on autonomous vehicle 100 (e.g., a point on array 102). Position comparator 113 may compare data representing the location of object 180 to threshold location 192 as determined by threshold location estimator 118. While traveling along trajectory 185, object 180 may cross threshold location 192 or otherwise change its location (e.g., X O , Y O ) and the threshold position 192 (eg, X T , Y T ). When the position matches (for example, X O =X T and / or Y O =Y T ), the position comparator 113 may generate data representing a trigger signal. When the positions are determined to be consistent with each other (eg, the positions match), data other than the data representing the trigger signal may be generated.

[0045] Upon receiving the data representing the audio signal, the data representing the trigger signal, and the data representing the position (or coordinates) of the object (e.g., object 180), array 102 may transmit guided acoustic energy beam 104 along propagation direction 106 toward object 180. There may be more than one threshold position 192, such as indicated at 193, and as the position of object 180 changes (e.g., as trajectory 185 brings object 180 closer to the location of autonomous vehicle 100), planner system 110 may continue to receive updated object data from perception system 140 and pose data from positioning system 130, and may use the updated data to generate additional acoustic alerts from array 102 (e.g., using different audio signal data) as the position of object 180 crosses additional threshold positions. For example, if an acoustic alert emitted by the array 102 at a threshold position 192 fails to cause a change in the behavior of the object 180 (e.g., to change the trajectory to a trajectory that does not conflict with the trajectory of the vehicle 100), then at the next threshold position, data representing an audio signal may be selected to convey a more urgent warning due to the object being closer to the vehicle 100 and / or its trajectory 105. Subsequently, an acoustic alert may be emitted from the array 102 to convey a higher level of urgency. An acoustic alert that is ineffective (e.g., ignored or not understood by the object 180) may cause the planner system 110 to take action to avoid a collision and / or a close pass between the vehicle 100 and the object 180. The planner system 110 may generate data representing drive commands (e.g., to the steering, braking, signaling, and propulsion systems of the vehicle 100) that are configured to cause the vehicle 100 to change its trajectory, position, or both to avoid a collision with the object.

[0046] Figure 2A An example of a flowchart 200 for implementing acoustic beam steering in an autonomous vehicle (e.g., autonomous vehicle 100 using acoustic beam steering array 102) is depicted. In process 200, one or more systems of the autonomous vehicle may implement one or more stages of process 200. At stage 202, a trajectory of the autonomous vehicle in an environment external to the autonomous vehicle may be calculated. The calculated trajectory may be data (e.g., pose data) representing the trajectory of the autonomous vehicle in the environment. At stage 204, coordinates of an object located in the environment external to the autonomous vehicle may be calculated. For example, the calculated coordinates may be data representing coordinates, such as Cartesian coordinates, polar coordinates, or angles. In some examples, stage 204 may calculate data representing the position of the object in the environment. At stage 206, the movement of the object in the environment external to the autonomous vehicle may be predicted. The predicted movement of the object in the environment may be determined using data representing the object type of the object and data representing the position of the object in the environment. At stage 208, a triggering acoustic alarm in the environment (e.g., via Figure 1At stage 210, an audio signal (e.g., a digital audio file and / or a microphone signal) may be selected for the one or more threshold positions. The selected audio signal may be different for each threshold position. At stage 212, when the coordinates of the object coincide with the threshold position, the acoustic beam steering array (e.g., Figure 1 At stage 214, a guided acoustic energy beam (e.g., from a speaker of array 102) may be generated (e.g., emitted by a speaker of array 102) into the environment along a propagation direction determined by the coordinates of the object (e.g., determined by the data representing the coordinates or position of the object) via the acoustic beamsteering array. Figure 1 beam 104 of array 102 in FIG.

[0047] exist Figure 2A In the process 200, the planner system and / or the perception system (e.g., Figure 1 The planner system 110 and / or the perception system 140 in the autonomous vehicle may be used to calculate data representing the location of an object in the environment external to the autonomous vehicle and / or data representing the coordinates of an object in the environment external to the autonomous vehicle.

[0048] Figure 2B Another example of a flowchart 250 for implementing acoustic beam steering in an autonomous vehicle is depicted. At stage 252, data representing the trajectory of the autonomous vehicle 100 in the environment can be calculated based on data representing the position of the autonomous vehicle 100 (e.g., pose data). At stage 254, data representing the position (e.g., coordinates) of an object disposed in the environment can be determined (e.g., based on object tracking data derived from sensor data). Data representing the type of object can be associated with data representing the position of the object. At stage 256, data representing the trajectory of the object in the environment can be predicted based on the data representing the type of object and the data representing the position of the object in the environment. At stage 258, data representing a threshold position associated with an acoustic alarm (e.g., from the array 102) in the environment can be estimated based on the data representing the trajectory of the object and the data representing the trajectory of the autonomous vehicle. At stage 260, data representing an audio signal associated with the acoustic alarm can be selected. At stage 262, it can be detected that the position of the object is consistent with the threshold position. As an example, the trajectory of the object passing through the threshold position can be a consistent indication. At stage 264, an acoustic beam steering array (e.g., array 102) can be caused (e.g., triggered, activated, or commanded) to emit a guided acoustic energy beam (e.g., beam 104) along a propagation direction (e.g., propagation direction 106) determined by the position of an object (e.g., the coordinates of the object in the environment).

[0049] Stages of process 200, process 250, or both may be performed for one or more acoustic beam steering arrays 102. One or more stages of process 200, process 250, or both may be repeated. For example, object tracks, object positions (e.g., object coordinates), threshold positions, vehicle tracks, vehicle positions, audio signal selection, consistency detection, and other stages may be repeated to update and / or process data as needed (e.g., due to movement of the autonomous vehicle, the object, or both).

[0050] Figure 3A Another example 300 of a system for implementing an acoustic beam steering array in an autonomous vehicle is depicted. Figure 3A , sensor system 320 includes sensors 328 configured to generate information indicating an autonomous vehicle (e.g., Figure 1 The planner system 310 may receive sensor data 332 and 334 (e.g., data representing sensor signals) of an environment 390 external to the vehicle 100). The localizer system 330 may receive the sensor data 332 and 334, and the perception system 340 may receive the sensor data 332 and 334. The sensor data 332 and 334 received by the localizer system 330 need not be the same as the sensor data 332 and 334 received by the perception system 340. The planner system 310 may receive vehicle position data (e.g., pose data) from the localizer system 330 and may receive object data (e.g., object classification, object tracking, and object location) from the perception system 340.

[0051] Planner system 310 may determine which detected objects (e.g., from a domain of many possible objects in environment 390) to acoustically alert and may generate data and control signals 317 (e.g., trigger signals, audio signals, and object locations) that are received by acoustic beam steering array 302 to generate guided acoustic energy beam 304 into environment 390. Planner system 310 may control other systems of the autonomous vehicle, including, but not limited to, steering, braking, propulsion, signaling (e.g., brake lights, turn signals, headlights, etc.), and safety. Those other systems may be activated by planner system 310 as needed to mitigate any possible collision or close pass with the object for which the acoustic alert was intended. In some examples, data representing the location of the object may be received from perception system 340 at acoustic beam steering array 102, as indicated by arrow 357 for the location of the object. Perception system 340 may use data included in sensor data 334 to track the location of the object and output data (e.g., angles, polar coordinates, Cartesian coordinates, etc.) representing object coordinates 357. In other examples, perception system 340 may output data representing object coordinates to planner system 310 (e.g., in object data 349), array 102 (e.g., in object locations 357), or both.

[0052] Figure 3B Yet another example 399 of a system for implementing an acoustic beam steering array in an autonomous vehicle is depicted. In example 399, sensors 328 in sensor system 320 may include, but are not limited to, one or more of: light detection and ranging sensors 321 (lidar), image capture sensors 323 (e.g., cameras), radio detection and ranging sensors 325 (radar), sound capture sensors 327 (e.g., microphones), and global positioning system sensors (GPS) and / or inertial measurement unit sensors (IMU) 329. Localizer system 330 and perception system 340 may receive sensor data 332 and 334, respectively, from one or more of sensors 328. For example, perception system 340 may receive sensor data 334 related to determining information associated with objects in environment 390, such as sensor data from lidar 321, camera 323, radar 325, and microphone 327; while localizer system 330 may receive sensor data 332 associated with the position of the autonomous vehicle in environment 390, such as from GPS / IMU 329.

[0053] For example, localizer system 330 may receive and / or access data from sources other than sensor data 332, such as odometry data 333 from motion sensors to estimate changes in position of autonomous vehicle 100 over time, wheel encoders 335 to calculate motion, distance, and other metrics of autonomous vehicle 100 based on wheel rotations (e.g., via a propulsion system), map data 331 from a representation of map tiles, route network definition files (RNDF), and / or other files, and data representing an autonomous vehicle (AV) model 337, which may be used to calculate vehicle position data based on a vehicle dynamics model (e.g., from simulations, captured data, etc.) of autonomous vehicle 100. Localizer system 330 may use one or more of the data sources depicted as generating data representing pose data 339.

[0054] As another example, the perception system 340 can parse or otherwise analyze, process, or manipulate sensor data 334 to enable object detection 341, object tracking 343 (e.g., determining which detected objects are static (not moving) and which are dynamic (in motion)), object classification 345 (e.g., motor vehicles, motorcycles, bicycles, pedestrians, skateboarders, mailboxes, buildings, street lights, etc.), and traffic light / sign detection 347 (e.g., stop lights, stop signs, track road intersections, lane markings, crosswalks, etc.).

[0055] As yet another example, planner system 310 may receive pose data 339 and object data 349 and may parse or otherwise analyze, process, or manipulate the data (339, 349) to implement trajectory calculation 312, threshold position estimation 318, motion prediction 316, position comparison 313, object coordinate determination 314, and audio signal selection 311. Planner system 310 may transmit trajectory and control data 351 to one or more controllers 350. One or more controllers 350 may convert trajectory and control data 351 into vehicle control and data 352. Vehicle control and data 352 may be transmitted to one or more vehicle controllers 360. One or more vehicle controllers 360 may process vehicle control and data 352 to generate array system data 364 and drive system data 361. Array system data 364 may include object position data 348 (e.g., coordinates of an object in environment 390), audio signal data 362, trigger signal data 371, and, optionally, modulation signal data 369 received by acoustic beam steering array 302. Despite Figure 3B, but the autonomous vehicle may include additional acoustic beam steering arrays, labeled 303. Drive system data 361 may be transmitted to drive system 362. Drive system 362 may transmit drive system data 361 to braking 363, steering 365, and propulsion 367 systems of autonomous vehicle 100, respectively. For example, drive system data 361 may include steering angle data for steering system 365 (e.g., steering angle for wheels), braking data for braking system 363 (e.g., braking force to be applied to brakes), and propulsion data for propulsion system 367 (e.g., voltage, current, or power to be applied to motors). Dashed line 366 may represent a boundary between a vehicle trajectory processing layer and a vehicle physical execution layer, where data processed in the vehicle trajectory processing layer is implemented, for example, by the drive system and array systems.

[0056] Acoustic beam steering array 302 may include processor 305 (e.g., microprocessor, digital signal processor (DSP)), which may be configured to receive data (348, 362, 371, 369) and process the data to generate a guided acoustic energy beam 304 (e.g., at an angle 13 relative to trajectory TAV) into environment 390 using array 302 (e.g., in response to receiving data representing trigger signal 371). Acoustic beam steering array 302 may include multiple speakers S, wherein each speaker S in array 302 is coupled to an output of amplifier A. Each amplifier A may include a gain input and a signal input. Processor 305 may calculate data representing a gain G for the gain input of each amplifier A and may calculate data representing a signal delay D for the signal input of each amplifier A. The processor 305 can access and / or receive data representing information about the speakers S (e.g., from internal and / or external data sources) and the information can include, but is not limited to, array width, speaker S spacing in the array, wavefront distance between adjacent speakers S in the array, the number of speakers S in the array, speaker characteristics (e.g., frequency response, output level per watt of power, etc.), to name a few.

[0057] Figure 4 An example of a flow chart 400 for implementing a perception system is depicted. Figure 44, for purposes of explanation, sensor data 434 received at perception system 440 is visually depicted as sensor data 434a-434c (e.g., lidar data). At stage 402, it may be determined whether sensor data 434 includes data representing a detected object. If the "no" branch is taken, process 400 may return to stage 402 to continue analyzing sensor data 434 for object detection. If the "yes" branch is taken, process 400 may continue to stage 404, where it may be determined whether the data representing the detected object includes data representing a traffic sign or signal light. If the "yes" branch is taken, process 400 may transition to stage 406, where the data representing the detected object may be analyzed to, for example, classify the type of signal light / sign object detected, such as a traffic light (e.g., red, yellow, and green) or a stop sign. Analysis at stage 406 may include accessing traffic object data store 424, where stored examples of data representing traffic classifications may be compared with data representing detected objects to generate data representing traffic classifications 407. Stage 406 may then transition to another stage, such as stage 412.

[0058] If the "no" branch is taken, process 400 may transition to stage 408, where the data representing the detected object may be analyzed to determine other object types to be classified. If the "yes" branch is taken from stage 408, process 400 may transition to stage 410, where the data representing the detected object may be analyzed to classify the object type of the detected object. Object data store 426 may be accessed to compare the stored examples of data representing object classifications with the data representing the detected object to generate data representing object classifications 411. Stage 410 may then transition to another stage, such as stage 412. If the "no" branch is taken from stage 408, stage 408 may transition to another stage, such as returning to stage 402.

[0059] At stage 412, the object data classified at stages 406 and / or 410 may be analyzed to determine whether the sensor data 434 indicates motion associated with the data representing the detected object. If motion is not indicated, the "no" branch may be taken to stage 414, where the object tracking data representing the detected object may be set to static (S). At stage 416, data representing the location of an object (e.g., a static object) may be tracked. For example, a stationary object detected at time t0 may move at a later time t1 and become a dynamic object. In addition, data representing the location of the object may be included in the data received by the planner system 110. The planner system 110 may use the data representing the location of the object to determine the coordinates of the object (e.g., relative to the autonomous vehicle 100 and / or the array 102).

[0060] On the other hand, if motion is indicated in the detected object, then the “yes” branch may be taken from stage 412 to stage 418, where object tracking data representing the detected object may be set to dynamic (D). At stage 420, data representing the position of an object (e.g., a dynamic object) may be tracked. Planner system 110 may analyze the data representing object tracking and / or the data representing the position of an object to determine whether the detected object (static or dynamic) may have a conflicting position relative to the autonomous vehicle and / or may be coming into close proximity with the autonomous vehicle such that an audible alarm may be used to alter the behavior of the object and / or a person controlling the object.

[0061] At stage 422, one or more of data representing object classification, data representing object tracking, and data representing the location of the object may be included as object data 449 (e.g., object data received by the planner system). As an example, sensor data 434a may include data representing an object (e.g., a person riding a skateboard). Stage 402 may detect an object in sensor data 434a. At stage 404, it may be determined that the detected object is not a traffic sign / signal. Stage 408 may determine that the detected object is another object category and may analyze the data representing the object at stage 410 using data retrieved from object data store 426 to determine that the classification matches a person riding a skateboard and output data representing object classification 411. At stage 412, it may be determined that the detected object is in motion, and at stage 418, object tracking may be set to dynamic (D) and the location of the object may be tracked at stage 420 (e.g., by continuing to analyze sensor data 434a of the detected object). At stage 422, object data associated with sensor data 434 may include, for example, classification (e.g., a person riding a skateboard), object tracking (e.g., the object is in motion), and the location of the object in the environment outside the autonomous vehicle (e.g., the location of the skateboard user).

[0062] Similarly, for sensor data 434b, process 400 may, for example, determine that the object classification is a pedestrian, the pedestrian is in motion (e.g., walking) and has dynamic object tracking, and the location of the object (e.g., pedestrian) in the environment outside the automated vehicle may be tracked. Finally, for sensor data 434c, process 400 may determine that the object classification is a fire hydrant, the fire hydrant is not moving and has static object tracking, and the location of the fire hydrant may be tracked. Note that in some examples, object data 449 associated with sensor data 434a, 434b, and 434c may be further processed by the planner system based on factors including, but not limited to, object tracking, object classification, and object location. For example, in the case of a skateboard user and a pedestrian, if the planner system decides to implement an acoustic alert for the skateboard user and / or the pedestrian, the object data 449 may be used for one or more of trajectory calculation, threshold position estimation, motion prediction, position comparison, and object coordinates. However, the planner system may decide to ignore the object data for the fire hydrant due to its static object tracking, for example because the fire hydrant is unlikely to be in motion that conflicts with the autonomous vehicle (e.g., the fire hydrant is stationary) and / or because the fire hydrant is inanimate (e.g., unable to respond to or be aware of acoustic alarms).

[0063] Figure 5An example 500 of object prioritization by a planner system in an autonomous vehicle is depicted. Example 500 depicts the prioritization of objects as performed by a sensor system (e.g., Figure 1 and Figure 3A-3B 1, 320) of the sensor system 120, 320 of the autonomous vehicle 100 (e.g., based on lidar data or other sensor data). Object data from the perception system of the autonomous vehicle 100 has detected multiple objects in the environment 590, including but not limited to a motor vehicle 581d, a bicycle rider 583d, a walking pedestrian 585d, and two parked motor vehicles 587s and 589s. In this example, the perception system may have assigned dynamic object tracking to objects 581d, 583d, and 585d, so the label "d" is associated with the reference numerals of these objects. The perception system also assigns static object tracking to objects 587s and 589s, so the label "s" is associated with the reference numerals of these objects. The locator system can determine local pose data 539 (e.g., position and orientation estimate data) of the position of the autonomous vehicle 100 in the environment 590. In addition, the autonomous vehicle 100 may have a trajectory Tav as shown by the arrow. Two parked motor vehicles 587s and 589s are static and have no indicated trajectories. Bicyclist 583d has a trajectory Tb in a direction generally opposite to the direction of trajectory Tav, and motor vehicle 581d has a trajectory Tav in a direction generally parallel to and in the same direction as trajectory Tav. Pedestrian 585d has a trajectory Tp that is predicted (e.g., by a planner system) to intersect trajectory Tav of vehicle 100.

[0064] Since static objects 587s and 589s are located outside the path of trajectory Tav (e.g., objects 587s and 589s are parked) and dynamic object 583d is moving in a direction away from autonomous vehicle 100; thereby reducing or eliminating the possibility that trajectory Tb may conflict with trajectory Tav, the planner system may place a lower priority on processing data related to static objects 587s and 589s and dynamic object 583d. The movement and / or position of pedestrian 585d or other objects in environment 590 may be tracked or otherwise determined using metrics other than trajectory, such as, but not limited to, object position, predicted object motion, object coordinates, predicted rate of motion relative to the position of the object, and the next predicted position of the object. The movement and / or position of pedestrian 585d or other objects in environment 590 may be determined based at least in part on probability. For example, the probability may be based on data representing object classification, object tracking, object position, and object type.

[0065] However, the planner system may place a higher priority on tracking the position of pedestrian 585d due to the pedestrian's potentially conflicting trajectory Tp, and may place a slightly lower priority on tracking the position of motor vehicle 581d due to the fact that the trajectory Tav of motor vehicle 581d does not currently conflict with trajectory Tav, but motor vehicle 581d may conflict at a later time (e.g., due to a lane change or other vehicle maneuver). Thus, based on example 500, pedestrian object 585d is the most likely candidate for an acoustic alert because the pedestrian's trajectory (e.g., based on its position and / or predicted motion) may result in a possible collision with autonomous vehicle 100 (e.g., at estimated position 560) or an unsafe distance between pedestrian object 585d and autonomous vehicle 100.

[0066] Figure 6 A top plan view of an example 600 of acoustic beam steering from an acoustic beam steering array in an autonomous vehicle is depicted. Figure 6 middle, Figure 5 Example 500 is further illustrated in a top plan view, where trajectories Tav and Tp are estimated to intersect at an estimated position designated as 560 (e.g., based on position data of vehicle 100 and position data for pedestrian object 585d). Figure 6 , pedestrian object 585d is depicted as an example of a candidate that is most likely to receive an acoustic alert based on the predicted motion of pedestrian object 585d. Autonomous vehicle 100 is depicted as including four acoustic beam steering arrays 102 positioned at four different sides of vehicle 100 (e.g., on four different portions of the roof). Autonomous vehicle 100 may be configured to travel bidirectionally as indicated by arrows 680, that is, autonomous vehicle 100 may not have a front (e.g., a hood) or a rear (e.g., a trunk) as conventional motor vehicles. Thus, more than one acoustic beam steering array 102 may be positioned on vehicle 100 to provide acoustic alert coverage in more than one direction of travel of vehicle 100 and / or to provide acoustic alert coverage for objects approaching vehicle 100 from the sides 100s of vehicle 100.

[0067] The planner system can estimate one or more threshold positions, designated 601, 603, and 605, in the environment 590 and transmit an acoustic alert at a position of an object (e.g., a pedestrian object 585d) when the position coincides with a threshold position designated by points 602, 604, and 606 along the trajectory Tp. Although three threshold positions are depicted, there may be more or fewer threshold positions than depicted. As a first example, as the trajectory Tp crosses a first threshold position 601 at a point designated 602, the planner system can determine the position of the pedestrian object 585d at point 602 (e.g., with coordinates X1, Y1) and the position of the autonomous vehicle 100 (e.g., based on pose data) to calculate coordinates (e.g., angles) for the propagation direction 106 of the guided acoustic energy beam 104. For example, the coordinates can be based on a predetermined reference point 100r on the vehicle 100 and / or another predetermined reference point 102r on the acoustic array 102a. As an example, if the predetermined reference point 102r has coordinates (Xa, Ya), the processor, circuit, algorithm, or some combination of the foregoing may calculate the coordinates of the beam 104, such as the angle θ1 relative to the vehicle trajectory Tav (e.g., based on triangulation analysis). As the autonomous vehicle 100 continues to travel 625 from the position L1 to the position L2 along the trajectory Tav, the relative position between the pedestrian object 585d and the autonomous vehicle 100 may change, such that at the position L2, the coordinates (X2, Y2) at the point 604 of the second threshold position 603 may result in a new coordinate θ2 for the propagation direction 106. Similarly, continuing to travel 625 from the position L2 to the position L3 along the trajectory Tav may change the relative position between the pedestrian object 585d and the autonomous vehicle 100, such that at the position L3, the coordinates (X3, Y3) at the point 606 of the third threshold position 605 may result in a new coordinate θ3 for the propagation direction 106.

[0068] As the distance between the autonomous vehicle 100 and the pedestrian object 585d decreases, the data representing the audio signal selected for the guided beam 104 may be different to convey an increased sense of urgency (e.g., an elevated threat level) to the pedestrian object 585d to change or stop the trajectory Tp of the pedestrian object 585d, or otherwise modify his / her behavior to avoid a potential collision or close pass with the vehicle 100. As an example, when the vehicle 100 may be at a relatively safe distance from the pedestrian object 585p, the data representing the audio signal selected for the threshold position 601 may be an odd audio signal a1 configured to gain the attention of the pedestrian object 585p in a non-threatening manner. As a second example, when the vehicle 100 may be at a cautious distance from the pedestrian object 585p, the data representing the audio signal selected for the threshold position 603 may be a more aggressive audio signal a2 configured to gain the attention of the pedestrian object 585p in a more urgent manner. As a third example, when the vehicle 100 may be at a potentially unsafe distance from the pedestrian object 585p, the data representing the audio signal selected for the threshold position 605 may be a very aggressive audio signal a3 configured to obtain the attention of the pedestrian object 585p in an extremely urgent manner. The location of the estimated threshold position in the environment 590 may be determined by the planner system to provide sufficient time (e.g., approximately 5 seconds or more) before the vehicle 100 reaches a predicted impact point with the pedestrian object 585p (e.g., the point 560 where the estimated trajectories Tav and Tp intersect each other in the environment 590) based on the speed of the autonomous vehicle.

[0069] exist Figure 6 , the trajectory Tp of the pedestrian object 585p need not be straight as depicted, and the trajectory (e.g., the actual trajectory) may be an arcuate trajectory as depicted in example 650 or may be a non-linear trajectory as depicted in example 670. The planner system 110 may process the object data from the perception system 140 and the pose data from the locator system 130 to calculate the threshold position. The position, shape (e.g., linear, arcuate, non-linear), orientation (e.g., relative to the autonomous vehicle and / or the object), and other characteristics of the threshold position may be application-dependent and are not limited to the examples depicted herein. For example, in Figure 6 , the threshold position may be aligned approximately perpendicular to the trajectory Tp of pedestrian object 585d; however, other configurations may be calculated and implemented by planner system 110. As another example, the threshold position may be aligned in an orientation (or other orientation) that is not perpendicular to the trajectory Tav of autonomous vehicle 100 or the trajectory of the object. Furthermore, the trajectory of the object may be analyzed by the planner system to determine the configuration of one or more threshold positions.

[0070] Figure 7An example of a flow chart 700 for implementing a planner system is depicted. Figure 7 , planner system 710 may communicate with perception system 740 from which it receives object data 749 and localizer system 730 from which it receives pose data 739. Object data 749 may include data associated with one or more detected objects. For example, object data 749 may include data associated with a large number of detected objects disposed in an environment external to autonomous vehicle 100. However, some detected objects do not need to be tracked or may be assigned a lower priority based on the type of object. For example, Figure 4 Fire hydrant object 434c in the figure may be a static object (e.g., fixed to the ground) and may not require acoustic alarm processing; whereas skateboard user object 434a may require acoustic alarm processing due to it being a dynamic object and other factors, such as predicted human behavior such as a skateboard rider, the location of skateboard user object 434a, which may indicate that the location and / or location change may conflict with the trajectory of autonomous vehicle 100.

[0071] Object data 749 may include data for one or more detected objects. Figure 7 , three examples of detected objects are labeled 734a-734c. There may be object data 749 for more or fewer detected objects labeled 734. The object data for each detected object may include, but is not limited to, data representing object location 721, object classification 723, and object tracking 725. The planner system 710 may be configured to implement object type determination 731. Object type determination 731 may be configured to receive data representing object classification 723, data representing object tracking 725, and access object type data store 724. Object type determination 731 may be configured to compare the data representing object classification 723 and the data representing object tracking 725 with data representing object type (e.g., retrieved from data store 724) to determine data representing object type 733. Examples of data representing object type 733 include, but are not limited to, static grounded object types (e.g., Figure 4 hydrant 434c) and dynamic pedestrian objects (e.g. Figure 5 Pedestrian object 585d).

[0072] Object dynamics determination 735 may be configured to receive data representing object type 733 and data representing object location 721. Object dynamics determination 735 may be further configured to access object dynamics data store 726 and compare the data representing object dynamics with the data representing object type 733 and the data representing object location 721 to determine data representing predicted object motion 737.

[0073] Object trajectory predictor 741 can be configured to receive data representing predicted object motion 737, data representing the position of autonomous vehicle 743 (e.g., from pose data 739), data representing object position 721, and data representing object tracking 725. Object trajectory predictor 741 can be configured to process the received data to generate data representing predicted object trajectory 745 in the environment. In some examples, object trajectory predictor 741 can be configured to process the received data to generate data representing the predicted position of object 745 in the environment.

[0074] Threshold position estimator 747 can be configured to receive data representing position 743 of autonomous vehicle (e.g., from pose data 739) and data representing predicted object trajectory 745 and generate data representing one or more threshold positions 750 in the environment at which an acoustic alarm can be triggered. In some examples, threshold position estimator 747 can be configured to receive data representing position 743 of autonomous vehicle and data representing predicted position 745 of an object and generate data representing one or more threshold positions 750.

[0075] Figure 8 An example 800 of a block diagram of a system in an autonomous vehicle with an acoustic beam steering array is depicted. Figure 8, autonomous vehicle 100 may include a set of sensors 820 positioned at one or more locations on autonomous vehicle 100. Each set 820 may have sensors including, but not limited to, a lidar 821 (e.g., a color lidar, a three-dimensional lidar, a three-dimensional color lidar, a two-dimensional LIDAR, etc.), an image capture device 823 (e.g., a digital camera), a radar 825, a microphone 827 (e.g., to capture ambient sounds), a microphone 871 (e.g., to capture sounds from drive system components such as a propulsion system and / or a braking system), and a loudspeaker 829 (e.g., to greet / communicate with passengers of AV 100). Loudspeaker 829 is not one of the speakers S in array 102. Each set of sensors 820 may include, for example, more than one sensor of the same type, such as two image capture devices 823 or a microphone 871 positioned near each wheel 852. Microphone 871 may be configured to capture real-time audio signals indicative of driving operations of autonomous vehicle 100. Microphone 827 may be configured to capture real-time audio signals indicative of ambient sounds in an environment outside autonomous vehicle 100. Autonomous vehicle 100 may include sensors for generating data representing the location of autonomous vehicle 100, and these sensors may include, but are not limited to, a global positioning system (GPS) 839a and an inertial measurement unit (IMU) 839b. Autonomous vehicle 100 may include one or more sensors ENV 877 for sensing environmental conditions in an environment external to autonomous vehicle 100, such as air temperature, air pressure, humidity, barometric pressure, etc. The data generated by one or more sensors ENV 877 may be used, for example, to calculate the speed of sound when processing data such as wavefront propagation times used by array 102.

[0076] Communication network 815 may, for example, route signals and / or data to / from sensors and other components and / or systems of autonomous vehicle 100, such as one or more processors 810 and one or more routers 830. Router 830 may route signals and / or data from sensors in sensor group 820, one or more acoustic beam steering arrays 102, between other routers 830, between processors 810, drive operating systems such as propulsion (e.g., motor 851), steering, braking, safety systems, etc., and communication system 880 (e.g., for wireless communication with external systems and / or external resources).

[0077] exist Figure 8, one or more microphones 827 may be configured to capture ambient sounds in environment 890 external to autonomous vehicle 100. Signals and / or data from microphones 827 may be used to adjust gain values ​​of one or more speakers (not shown) positioned in one or more of acoustic beam steering arrays 102. As an example, loud ambient noise, such as noise emitted from a construction site, may mask or otherwise impair the audibility of guided acoustic energy beams 104 emitted by array 102. Thus, gain may be increased or decreased based on the ambient sound (e.g., in dB or other metrics such as frequency content). Signals and / or data generated by microphones 827 may, for example, be converted from one signal format to another, such as from analog to digital (e.g., using an ADC or DSP) or from digital to analog (e.g., using a DAC or DSP).

[0078] The microphone 871 can be positioned near a drive system component such as a motor 851, a wheel 852, or a brake (not shown) to capture sounds generated by these systems, such as rotational noise 853, regenerative braking noise, tire noise, and motor noise. The signal and / or data generated by the microphone 871 can be used, for example, as data representing an audio signal. The signal and / or data generated by the microphone 871 can be converted from one signal format to another signal format, such as from analog to digital (e.g., using an ADC or DSP) or from digital to analog (e.g., using a DAC or DSP). In other examples, the signal and / or data generated by the microphone 871 can be used to modulate data representing an audio signal (e.g., using a DSP).

[0079] The one or more processors 810 may be used, for example, to implement one or more of a planner system, a locator system, a perception system, a drive system, a safety system, and an acoustic beam steering array. The one or more processors 810 may be configured, for example, to execute an algorithm embedded in a non-transitory computer-readable medium to implement one or more of a planner system, a locator system, a perception system, a drive system, a safety system, and an acoustic beam steering array. The one or more processors 810 may include, but are not limited to, circuits, logic, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), programmable logic, digital signal processors (DSPs), graphics processing units (GPUs), microprocessors, microcontrollers, mainframe computers (BFCs), or other devices or clusters thereof.

[0080] exist Figure 8, multiple acoustic beam steering arrays 102 may be positioned at multiple locations on autonomous vehicle 100 to generate one or more acoustic alerts at one or more objects positioned at one or more locations in environment 890. Acoustic beam steering arrays 102 need not be identical in configuration or size. For example, acoustic beam steering arrays 102 labeled C and D may be longer in length and may include more speakers than acoustic beam steering arrays 102 labeled A and B. As an example, the side of autonomous vehicle 100 where arrays C and D are located may be longer than the side where arrays C and D are located; thereby allowing arrays 102 to have a longer length dimension. A longer length dimension may allow for a greater number of speakers in array 102. A housing or other structure configured to house acoustic beam steering arrays 102 may be mounted on the exterior of autonomous vehicle 100.

[0081] Fig. 9 A top view of an example 900 of an acoustic beam steering array positioned on an autonomous vehicle is depicted. Fig. 9 , sensor group 820 may be positioned at a corner of autonomous vehicle 100 (e.g., at a pillar section), and a housing for acoustic beam steering array 102 may be positioned on an upper surface 100u of autonomous vehicle 100 (e.g., on the roof) and may be positioned to direct corresponding beams 104 of array 102 outwardly into the environment toward the location of an object targeted for receiving an acoustic alert. Acoustic beam steering array 102 may be configured to provide coverage of directed acoustic energy at one or more objects in the environment disposed external to autonomous vehicle 100 and the coverage may, for example, be in an arc of approximately 360 degrees around autonomous vehicle 100. As described above in Figure 8 As described in , acoustic beam steering arrays 102 need not be of the same size or have the same number of speakers. Fig. 9 , the acoustic beam steering arrays 102 labeled C and D have different dimensions than the acoustic beam steering arrays 102 labeled A and B. Sensor groups 820 may be positioned as one array 102 or multiple arrays 102 to provide sensor coverage of the environment external to autonomous vehicle 100. In the case of multiple arrays 102, the sensor groups may provide overlapping areas of sensor coverage. The perception system of autonomous vehicle 100 may receive sensor data from more than one sensor or more than one group of sensors to provide data to the planning system to trigger one or more of arrays 102 to generate an acoustic alert. Autonomous vehicle 100 may not have a front or rear and may therefore be configured for driving operations in at least two different directions, labeled as arrows 980. Therefore, arrays 102 may not have a front or rear indication, and depending on which direction autonomous vehicle 100 is driving, array 102(A) may be an array facing the direction of travel or array 102(B) may be an array facing the direction of travel.

[0082] Fig. 10A A top plan view depicting two examples of sensor system coverage in an autonomous vehicle. Fig. 10B Depicts top plan views of two additional examples of sensor system coverage in an autonomous vehicle. Fig. 10A In example 1010, one of four sensor groups 820 (underlined) may use one or more of its corresponding sensors to provide sensor coverage 1011 of environment 1090 in a coverage area that may be configured to provide sensor data for acoustic beam steering array 102 (B and D). In example 1010, because there may be blind spots that are not covered by a single sensor group 820, sensor coverage 1011 for acoustic beam steering array 102 (B and D) may be partial coverage, labeled B' and D'.

[0083] exist Fig. 10A In example 1020, the second sensor group of the four sensor groups 820 may provide sensor coverage 1021 that overlaps sensor coverage 1011, such that array D has full coverage and arrays A and B may have partial coverage, indicated as A' and B'. Fig. 10B In example 1030, the third sensor group of the four sensor groups 820 may provide sensor coverage 1031 that overlaps sensor coverage 1011 and 1021, such that arrays A and D have full coverage and arrays C and B may have partial coverage labeled B' and C'. Finally, in Fig. 10B In example 1040, a fourth sensor group of four sensor groups 820 (e.g., all four sensor groups are online) can provide sensor coverage 1041 that overlaps with sensor coverage 1011, 1021, and 1031, so that arrays A, B, C, and D have complete coverage. The overlapping sensor fields of coverage can achieve redundancy if one or more of the sensor groups 820 and / or their corresponding sensors (e.g., lidar, camera, radar, etc.) are damaged, fail, or otherwise fail. Figures 10A-10B , dashed lines 1061 and 1063 divide four quadrants 1-4, and sensor groups 820 can be symmetrical so that sensors in sensor group 820 in quadrant 1 can be replicated in sensor groups 820 in quadrants 2, 3, and 4, for example. Safety systems (e.g., external systems, internal systems, and drive systems) provided in a quadrant can also be replicated in other quadrants. For example, a safety system associated with quadrant 1 can be replicated in quadrants 2, 3, and 4. As described above in the above examples 1010, 1020, 1030, and 1040, sensor groups 820 in one or more of the quadrants can provide overlapping sensor coverage.

[0084] One or more sensors in sensor group 820 may be arranged in corner clusters (e.g., to provide sensor coverage that overlaps with other sensor groups 820), and one or more sensors in sensor group 820 may be arranged at different vertical heights of vehicle 100, such as one lidar sensor positioned vertically above another lidar sensor. In some examples, the sensors may include a combination of sensor types, such as a lidar including one or more cameras (e.g., each camera covering 160 degrees for a total of 270 degrees of sensor coverage) and another lidar without a camera. The sensors in sensor group 820 may include, but are not limited to, one or more of a lidar sensor, an image capture sensor (e.g., a camera), a stereo camera, a long-range stereo camera, a radar (e.g., a radar positioned at each end of vehicle 100), and a sound navigation and ranging (sonar) sensor (e.g., for detecting small objects low to the ground to assist in parking vehicle 100). One or more sensors in sensor group 820 may communicate with one or more processors of autonomous vehicle 100. For example, output signals and / or data from sensors in sensor group 820 may be electronically transmitted to multiple processing units to provide redundancy in signal / data processing. For example, each sensor in sensor group 820 can have its output sent to two processors to provide double redundancy, to three processors to provide triple redundancy, or to four processors to provide quadruple redundancy. For example, Gigabit Ethernet can be used to communicate signals / data from one or more LiDAR sensors to multiple processors. As another example, a high-speed Gigabit Multimedia Serial Link (GMSL) can be used to transmit signals / data from one or more image capture sensors (e.g., cameras, stereo cameras) to multiple processors.

[0085] Fig.11 A top plan view of an example 1100 of an acoustic beam steering array for directing acoustic energy at an autonomous vehicle approaching the vehicle is depicted. Fig.11, the autonomous vehicle 100 may have a trajectory Tav along the road between lane markings indicated by dashed line 1177. Detected objects in the environment outside the autonomous vehicle 100 have been classified as motor vehicle object type 1170 with a trajectory Tc, which is estimated to conflict with the trajectory Tav of the autonomous vehicle 100. The planner system may generate three threshold positions t-1, t-2, and t-3 having arc profiles 1121, 1122, and 1123, respectively. The three threshold positions t-1, t-2, and t-3 may represent different threat levels, which may be arranged, for example, from a low threat level at t-1 (e.g., a relatively safe distance from the vehicle 100), a medium threat level at t-2 (e.g., a cautiously approaching distance to the vehicle 100), and a high threat level at t-3 (e.g., a dangerously close distance to the vehicle 100). The audio signal for the acoustic alarm generated at each of the three threshold positions t-1, t-2, and t-3 can be selected to audibly convey an increasing threat level as the trajectory Tc of object 1170 brings object 1170 closer to the location of autonomous vehicle 100 (e.g., potentially colliding with autonomous vehicle 100).

[0086] When the motor vehicle object type 1170 passes through or otherwise makes its position coincident with the threshold position t-1, the planner system may generate a trigger signal to activate the acoustic array 102 positioned approximately in the direction of the motor vehicle object type 1170 to generate an acoustic alarm using the audio signal 104a along the propagation direction 106a based on the position (e.g., coordinates) of the motor vehicle object type 1170. For example, the position may be represented by the angle βa measured between the coordinates, the trajectory Tav, and the propagation direction 106a. As the motor vehicle object type 1170 continues on the trajectory Tc and passes through the threshold position t-2, another acoustic alarm may be triggered by the planner system using the coordinates βb, the audio signal 104b, and the propagation direction 106b. Further travel of the motor vehicle object type 1170 along the trajectory Tc through the threshold position t-3 may trigger yet another sound alarm by the planner system using the coordinates βc, the audio signal 104c, and the propagation direction 106c.

[0087] For each of the acoustic alerts triggered by the planner system, the position of the motor vehicle object type 1170 may change (e.g., relative to the position of the vehicle 100), and the planner system may receive updated object data (e.g., object tracking data from the perception system) to calculate (e.g., in real time) the change in position of the motor vehicle object type 1170 (e.g., calculate the coordinates of βa, βb, and βc). The audio file selected by the planner system for each threshold position t-1, t-2, and t-3 may be different and may be configured to include audio information intended to convey increasing urgency at each of the threshold positions t-1 to t-2 and t-2 to t-3. The one or more audio signals selected by the planner system may be configured to acoustically penetrate a structure (1171, 1173) of a motor vehicle, such as a motor vehicle glass, door, etc., based on the object type data of 1170 to attract the attention of the motor vehicle driver. As an example, the selected audio signal may be configured to generate frequencies within a range of approximately 220 Hz to approximately 450 Hz to acoustically penetrate structures ( 1171 , 1173 ) on the motor vehicle 1170 .

[0088] Acoustic beam steering array 102 may include a number of speakers and amplifiers and drive electronics (e.g., processors, DSPs, etc.) associated therewith. For purposes of explanation, each amplifier / speaker pair will be labeled as a channel, such that array 102 will include n channels labeled C1 for channel 1 through Cn for the nth channel. In a zoomed-in view of acoustic beam steering array 102, each speaker S may be spaced a distance d from an adjacent speaker. Distance d (e.g., spacing between adjacent speakers (S)) may be the same for all speakers S in array 102, such that all of the speakers S are spaced a distance d from each other. In some examples, distance d may vary between speakers S in array 102 (e.g., distance d between adjacent speakers in array 102 need not be the same). Distance d may be measured from a reference point on each speaker S, such as a center point of each speaker S.

[0089] The width W of the array 102 can be measured as the distance between the first speaker in the array 102 (e.g., channel C1) and the last speaker in the array 102 (e.g., channel Cn) and the width W can be measured, for example, from the center of the speaker in C1 to the center of the speaker in Cn. The length of the housing housing the speakers S can be determined in part by the width W between the first and last speakers in the array 102. In the propagation direction 106 of the sound waves 104 generated by the array 102, the wavefronts emitted by adjacent speakers S in the array can be delayed in time based on the time delay by the wavefront propagation time t d . Wavefront propagation time td It can be calculated as the distance between adjacent wavefronts r multiplied by the speed of sound c (e.g., t d = r*c). In an example where the distance d between the speakers S is the same for all speakers S in the array 102, the delay D calculated for each speaker S may be t d Thus, for example, for channel C1: (t d1 =(r*c)*1), for channel C2: (t d2 =(r*c)*2), for channel Cn:(t dn =(r*c)*n). In some examples, data from a sensor (e.g., Figure 8 The speed of sound c is calculated based on data from an environmental sensor 877) to more accurately determine the value of the speed of sound c based on environmental conditions such as altitude, air pressure, air temperature, humidity, and air pressure.

[0090] Fig.12 An example of a flow chart 1200 for implementing acoustic beam steering in an acoustic beam steering array is depicted. Fig.12 , planner system 1210 may transmit signals and / or data to an acoustic beam steering array 1202 configured to receive the signals and / or data and generate an acoustic energy guided beam 104. At stage 1201, a trigger signal detector may receive data representing a trigger signal 1204 and generate output data indicating that the data representing the trigger signal 1204 has been received. At stage 1203, it may be determined whether the output data indicates that data representing a trigger signal 1204 (hereinafter referred to as a trigger signal) has been received. If the output data does not indicate that a trigger signal has been received, the "no" branch may be taken and the process 1200 may return to stage 1201 to await the arrival of a trigger signal. If the output data indicates that a trigger signal has been received, the "yes" branch may be taken and one or more audio signals 1206 selected by the planner system 1210 may be received by the array 1202. The planner system 1210 may select data representing the audio signal from one or more resources including, but not limited to, an audio signal data store 1251 (e.g., one or more digital audio files), an external resource 1253 (e.g., the cloud, the Internet, a data repository, etc.), a microphone 827 configured to capture ambient sound (see Figure 8 ) and a microphone 871 configured to capture sounds generated by driving operations (see Figure 8 ).

[0091] At stage 1207, a determination may be made as to whether to apply bandpass filtering to the data representing audio signal 1206. If the "yes" branch is taken, then at stage 1209, one or more bandpass filters may be applied to the data representing audio signal 1206 (e.g., using circuitry in the analog domain and / or using a DSP algorithm in the digital domain). Stage 1209 may access data representing bandpass filters 1270. The data representing bandpass filters 1270 may include, but are not limited to, data representing a high passband filter 1271 and data representing a low passband filter 1273. Other types of bandpass filters, designated as 1275, may be included in 1270. If the "no" branch is taken from stage 1207, flow 1200 may continue at stage 1211, where a beam steering algorithm may receive data representing the position of object 1208 (e.g., an angle, at least two Cartesian coordinates, polar coordinates, etc.).

[0092] At stage 1213, speaker channel compensation may be applied to each speaker channel in the array. Speaker channel compensation at stage 1213 may include calculating a gain G for each speaker channel and calculating a delay D for each speaker channel. Speaker channel compensation at stage 1213 may, for example, include accessing speaker data 1260 to obtain information about each speaker S in the array, the array width W, the distance d between adjacent speakers in the array, stored data representing the distance r between adjacent speakers S in the propagation direction, the number of speakers S in the array, the size of the speakers S, and other speaker related information.

[0093] At stage 1215, the compensating speaker signal may be applied to an amplifier A associated with each speaker S in each channel of the array. At stage 1215, the signal gain G for each channel and the signal delay D for each channel may be applied to the gain input and signal input of each amplifier in each channel of the array, such that the amplifier A1 of the first channel C1 receives its corresponding compensating gain G and delay D signal, and so on up to the amplifier An of the nth channel Cn. As the amplifier in each channel drives its corresponding speaker, A1 drives S1 and An drives Sn. The speakers S1-Sn transmit n single-channel guided acoustic energy beams 104 into the environment outside the autonomous vehicle 100.

[0094] Fig.13 Another example of a flow chart 1300 for implementing acoustic beam steering in an acoustic beam steering array is depicted. At stage 1301, data representing speaker data may be accessed from data store 1360. One or more stages of flow chart 1300 may use data included in data store 1360. Examples of data representing speaker data include, for example, but are not limited to, the number of speakers in array 102 (e.g., Fig.11 n in ), the spacing between adjacent speakers in the array 102 (e.g., Fig.11 d in ), the width of the array 102 (e.g., Fig.11 In the example, the speaker type may be piezoelectric, the speaker drive type may be voltage driven, the number of speakers may be 32 (e.g., n=32), the spacing between adjacent speakers may be 50 mm (e.g., d=50 mm), the width of the array 102 may be 163 cm (e.g., W=163 cm), and the speaker radiation area may be 2600 mm 2 As another example, the speaker type may be piezoelectric ceramic, the speaker drive type may be voltage drive, the number of speakers may be 330 (e.g., n=330), the interval between adjacent speakers may be 2.5 mm (e.g., d=2.5 mm), the width of the array 102 may be 170 cm (e.g., W=170 cm), and the speaker radiation area may be 25 mm 2 .

[0095] At stage 1303, data representing the position of object 1308 (e.g., angle) and data representing speaker data 1360 (e.g., spacing d) may be used to determine a distance r projected along the beam propagation direction between adjacent speakers in object array 102 (e.g., see Fig.11 At stage 1303, data representing the position of object 1308 may be used to determine the direction of propagation of beam 104 (e.g., Fig.12 , where β is the angle relative to the trajectory Tav).

[0096] At stage 1305, the relative delays t between adjacent loudspeakers in array 102 may be calculated. d (e.g., speed of sound*r). Relative delay t d The time delay of applying data representing the audio signal to the signal input of an amplifier coupled to the speaker may be represented so that a calculated value of the relative delay may be obtained for each speaker from a first speaker in the array 102 (e.g., Fig.11 C1 in) to the last speaker (e.g. Fig.11 For example, the relative delay of channel C2 is greater than the relative delay of channel C1, and so on.

[0097] At stage 1307, the spacing between adjacent speakers (e.g., Fig.11 If the distances d are not equal, then the “no” branch can be taken to reach stage 1311. At stage 1311, the audio signal (e.g., data representing audio signal 1306) sent to each speaker can be delayed according to its distance d (e.g., t d =f(d)). Then, stage 1311 may transition to stage 1313.

[0098] If the distances d are equal, then the “yes” branch may be taken to stage 1309. At stage 1309, the audio signal (e.g., data representing the audio signal 1306) sent to each speaker (e.g., via its respective amplifier) ​​may be delayed by a relative delay t d Stage 1309 may then transition to stage 1313.

[0099] At stage 1313, a determination may be made as to whether array shading is to be applied to data representing signal gains (e.g., G) for one or more channels in array 102. If the "no" branch is taken, process 1300 may transition to stage 1317. If the "yes" branch is taken, process 1300 may transition to stage 1315. At stage 1315, for example, a signal gain G for a selected speaker positioned at an edge portion of array 102 may be reduced (e.g., relative to a gain G for a speaker positioned at a middle portion of array 102). As an example, data representing a first gain for a first portion of speakers positioned at a first portion of acoustic beamsteering array 102 (e.g., at a first edge of the array) may be calculated, data representing a second gain for a second portion of speakers positioned at a second portion of acoustic beamsteering array 102 (e.g., at a second edge of the array) may be calculated, and data representing a third gain for a third portion of speakers positioned at a third portion of acoustic beamsteering array 102 (e.g., at a middle portion of the array) may be calculated. The magnitude of the signal gain of the data representing the third gain may be greater than the magnitude of the signal gain of the data representing the first gain and the data representing the second gain (e.g., the third gain>the first gain and the third gain>the second gain). The magnitude of the signal gain of the data representing the first gain and the data representing the second gain may be the same or may be different. In some examples, array shading may be implemented to reduce side lobes in the guided acoustic energy beam 104. Reducing side lobes may effectively reduce or eliminate audio signals from being audibly perceived by persons other than the desired object to which the beam 104 is directed (e.g., via the coordinates of the object). Stage 1315 may then transition to stage 1317.

[0100] At stage 1317, a gain G and delay D specific to each channel in array 102 may be applied to the signal and gain inputs of the amplifiers in each channel. For example, in channel C1, data representing the gain G calculated for channel C1 may be applied to the gain input of the amplifier in channel C1 (e.g., as a voltage). Data representing the audio signal may be applied to the signal input of the amplifier in channel C1 with a time delay D (e.g., a time delay in milliseconds) determined by the data representing the signal delay calculated for channel C1.

[0101] Fig.14 An example of a flowchart 1400 for implementing adaptive acoustic beam steering in an acoustic beam steering array of an autonomous vehicle is depicted. Fig.14 Adaptive beam steering is calculated using the equation depicted in example 1450 of , where the gain R can be determined based on the angle Φ of the beam to be directed to the object (e.g., object coordinates expressed as angles). Thus, R can express the gain as a function of various values ​​of the angle Φ.

[0102] In example 1450, variable k represents a wave number that can be expressed as k=2*(π / λ), where variable λ is the wavelength of the sound wave (e.g., the wavelength of beam 104). Variable Φ is the angle to which beam 104 is to be directed (e.g., data representing object coordinates). Variable x represents the position of a speaker in array 102, such that for an array having 320 speakers, x can have values ​​ranging from x=1 for the first speaker in array 102 to x=320 for the last speaker in array 102. Variable A represents the complex amplitude of a speaker located at position x in array 102. Variable d represents the distance between adjacent speakers in array 102 (e.g., Fig.11 The variable L represents the width 104 of the beam.

[0103] In process 1400, at stage 1401, data representing the position of object 1408 (e.g., coordinate angle φ) and data representing the width of the beam (e.g., variable L) may be received and may be used to calculate a gain R (e.g., R(sinφ)) as a function of the sine of angle φ according to the equation of example 1450. Stage 1401 may output data representing gain 1402 (e.g., the value of R(sinφ)).

[0104] At stage 1403, the inverse transform of R(sinφ) may be calculated to determine the complex amplitude A (e.g., A(x)) for each speaker position x in array 102. As an example, the inverse transform of R(sinφ) may be implemented using an inverse Fourier transform. Stage 1403 may output data representing the complex amplitude 1404 (e.g., A(x)) for each speaker position x in array 102. For example, if there are 54 speakers in array 102, stage 1403 may calculate and output values ​​for each of the 54 speaker positions.

[0105] At stage 1405, data representing complex amplitudes 1404 and data representing audio signals 1412 may be received, and a complex gain G at each speaker position x may be applied to a gain input of an amplifier of a speaker located at position x (e.g., at a channel located at position x in array 102), and a delay D at each speaker position x may be applied to the audio signal, and the delayed audio signal may be applied to the signal input of the amplifier of the speaker located at position x. Stage 1405 may output 1407 data representing gains Gn for signal n channels and signal delays D for n channels.

[0106] Fig.15 An example of a block diagram 1500 of an acoustic beam steering array is depicted. Fig.15 In the embodiment, the processor 1550 may receive data including but not limited to: the position 1501 (e.g., coordinates) of an object, an audio signal 1503, a microphone signal 1505, an ambient signal 1507 (e.g., from Figure 8 877) and trigger signal 1509. In some examples, processor 1550 may be electrically coupled to acoustic beam steering array 102 such that each acoustic beam steering array 102 is electrically coupled to its own dedicated processor 1550. In other examples, processor 1550 may be electrically coupled to multiple acoustic beam steering arrays 102.

[0107] The processor may implement functions for operating one or more acoustic beam steering arrays 102, and the implemented functions may include, but are not limited to: a delay (D) calculator 1511, a gain (G) calculator 1513, a beam steering algorithm 1517 (e.g., Fig.13 1300), adaptive beam steering algorithm 1519 (e.g., Fig.141400), an environment compensator 1521, an audio signal modulator 1523, an ambient noise compensator 1525, and a signal converter 1515. The processor 1550 may access data including, but not limited to, speaker (S) data 1531 (e.g., spacing d, width W, number of speakers S, speaker compensation data for calculating gain G and delay D, etc.) and data storage device 1530 (e.g., for storing data, algorithms, etc.). The processor 1550 may be implemented in an algorithm, software, firmware, logic, circuit, or some combination thereof.

[0108] The data and signals received by the processor 1550 may be converted from one format to another using a signal converter 1515. For example, the signal converter 1515 may convert digital data to analog signals using a digital-to-analog converter (DAC), and may convert analog signals to digital data using an analog-to-digital converter (ADC). The processing of data representing one or more audio signals 1503, data representing one or more microphone signals 1505, and data representing one or more ambient signals 1507 may be processed in the analog domain using a DAC, in the digital domain using an ADC, or both.

[0109] The environmental compensator 1521 may process data representing one or more environmental signals and output data representing compensated environmental data, such as the speed of sound c (eg, compensated for temperature, altitude, etc.) used by the beam steering algorithms (1517, 1519).

[0110] The ambient noise compensator 1525 may receive a signal representing ambient noise (e.g., from Figure 8 The data representing the gain compensation may be indicative of the ambient noise level in the environment outside the automatic vehicle 100. A high level of ambient noise may require an increase in the gain level G applied to the amplifier A in one or more channels to compensate for the high level of ambient noise. The gain calculator 1513 may receive the data representing the gain compensation and may increase the gain G or decrease the gain G in one or more of the channels of the array 102.

[0111] The audio signal modulation 1523 may receive data representing an audio signal and data representing a modulated signal (e.g., from Figure 8 The data representing the modulated signal may be based on the regenerative braking noise generated by the drive system of the autonomous vehicle 100. Figure 8The signal from microphone 871 in can be the signal source of data representing the modulated signal, and the amplitude of the data representing the modulated signal can be used to modulate data representing the audio signal (e.g., from a digital audio file). Audio signal modulation 1523 can process the data representing the audio signal and the data representing the modulated signal in the analog domain (e.g., using a DAC in signal converter 1515) or in the digital domain (e.g., using an ADC in signal converter 1515).

[0112] The gain calculator 1513 and the delay calculator 1511 may calculate channel gains G and channel delays D for one or more arrays 102, such as beam steering algorithm 1517 or adaptive beam steering algorithm 1519, for example, using an algorithm specific to the type of beam steering algorithm implemented for the array. One or more processors 1550 may receive data representing the trigger signal 1509 and generate gains G 1563 for n channels and delays D 1565 for n channels for each array 102 electrically coupled to the processor 1550. There may be a single array 102 or multiple arrays 102 electrically coupled to the processor 1550, indicated as 1561. The gain calculator 1513 may implement array shadowing (e.g., Fig.13 For example, after applying array shading, the gain Gm is greater than the gain Ge (see, for example, Fig.16 The speaker positions x taken from the speaker data 1531 can be used to determine which speaker positions x in the array 102 have the gain Ge applied to their amplifiers and which positions x in the array 102 have the gain Gm applied to their amplifiers. As another example, if the array 102 has 32 speakers S, such that there are 32 speaker positions x, then the 8 speakers located at each edge portion can have the gain Ge applied to their corresponding amplifiers A, and the 16 speakers located in the middle portion can have the gain Gm applied to their corresponding amplifiers A. The gain Gm can vary between the speakers S in the middle portion. The gain Ge can vary between the speakers S in the edge portion.

[0113] Fig.16 An example of array shadowing in an acoustic beam steering array is depicted 1600. Fig.16In example 1600 of , the object in the environment at which the directed acoustic energy beam is directed is person 1601. A curve 1620 of sound pressure level (e.g., in dB) as a function of angle θ of directed acoustic energy beam 104 is depicted as having its maximum sound pressure level in propagation direction 106. When person 1601 is positioned at a location where the sound pressure level has its maximum, person 1601 should be able to audibly perceive 1602 beam 104 (e.g., hear audio information included in the audio signal). To the left and right of propagation direction 106, the sound pressure level of beam 104 decreases to a sound pressure level that is less than the maximum value, but can still be audibly perceived (1604 and 1606) by persons 1603 and 1605 positioned in sidelobe regions 1621 and 1623 of beam 104. Person 1601 for whom the acoustic alert is desired by the beam may be located at the coordinates of the object (e.g., at a location indicated by angle θ); however, due to the conical expansion of beam 104, persons 1603 and 1605 at angular locations within the positive "+" or negative "-" angle θ may also perceive (1604 and 1606) the audio information in beam 104 and may react thereto in an unpredictable manner. Thus, in some examples, by applying shading (e.g., as shown in dashed curve 1630) to the gain G of the speaker S in array 102, the side lobes in beam 104 may be reduced from 1623 to 1633 and from 1621 to 1631. In curve 1630, the application of shading may result in persons 1603 and 1605 not being able to audibly perceive (1604 and 1606) the audio information in beam 104, or perceiving (1604 and 1606) the audio information in beam 104 at a lower sound pressure level.

[0114] exist Fig.16, speakers located at edge portions 1671 and 1673 of array 102 (e.g., speakers positioned toward the ends of array 102) may have their gain values ​​set to Ge; while speakers located in middle portion 1670 of array 102 (e.g., speakers positioned toward the center of 102) may have their gains set to Gm, where gain Gm is greater than gain Ge (e.g., Gm>Ge). Creating a gain difference between speakers in edge portions 1671 and 1673 and middle portion 1670 may reduce side lobes, resulting in a reduction in the sound pressure level perceived by persons 1603 and 1605. In some examples, the side lobe reduction may increase the perceived sound pressure level at the location of person 1601 due to the application of increased gain Gm to the amplifiers of speakers located in middle portion 1670 and / or the application of reduced gain Ge to the amplifiers of speakers located in edge portions 1671 and 1673. The speaker position data x may be used to determine which speakers S in array 102 have gain Ge applied to their amplifiers and which speakers have gain Gm applied to their amplifiers. The gain Gm may vary between speakers in the middle portion S. The gain Ge may vary between speakers in edge portion 1671, edge portion 1673, or both.

[0115] Fig.17 Examples of speakers and speaker enclosures in an acoustic beam steering array are depicted. In example 1700, each speaker S can include a frame 1701 and a sound radiating structure 1703 (e.g., a sound emitting surface of a piezoelectric or piezoelectric ceramic transducer). In example 1700, speakers S1, S2-Sn are depicted as having a square shape for frame 1701 and sound radiating structure 1703; however, the actual shape, size, and configuration of the speakers are not limited to example 1700. For example, the speaker and / or its sound radiating structure can have a circular shape, a rectangular shape, a nonlinear shape, or an arcuate shape, etc.

[0116] The x-dimension X-dim and y-dimension Y-dim of the sound radiating structure 1703 may define the sound radiating surface area of ​​the speaker. The spacing between adjacent speakers (e.g., between S1 and S3) may be d as measured from the center point "x" on the sound radiating structure 1703 (as shown in the front and top plan views in example 1700). Another reference point on the speaker may be selected and is not limited to the center point "x" depicted in example 1700. The width W of the array 102 may be measured from the center point "x" of the first speaker S1 in the array 102 to the center point "x" of the last speaker Sn.

[0117] In example 1740, the speakers may be mounted to a baffle, housing, or enclosure for array 102, labeled structure 1750. Frame 1701 of each speaker may be connected to structure 1750 using fasteners, glue, adhesive, welding, or other joining techniques. For autonomous vehicles 100 having multiple arrays 102, the size (e.g., width W) and number of speakers in each array may be the same or may vary between multiple arrays 102. For example, an array 102 positioned on the side of vehicle 100 (e.g., Figure 8-10B Arrays C and D in the example above may have a longer width W due to the longer sides of the vehicle 100; and thus arrays C and D may, for example, have more speakers within the width W than arrays A and B. Surface 1751 of array 102 may engage with a surface of vehicle 100 (e.g., the roof or surface 100u) to mount array 102 to vehicle 100. Structure 1750 to which the speakers are mounted need not be a planar surface and the construction of structure 1750 is not limited to Fig.17 As an example, the structure may be a non-linear surface or a curved surface.

[0118] In example 1780, two or more arrays 102 can be stacked on top of each other. The upper array 102U and the lower array 102L can have respective guided acoustic energy beams 104 directed to different objects in the environment, and the arrays 102U and 102L can operate independently of each other. The center points of the stacked arrays 102U and 102L do not need to have the same orientation and can face different directions. The upper array 102U can be joined to the lower array 102L using fasteners, glue, adhesive, welding or other joining techniques. The surface 1751 of the lower array 102L can be joined to the surface of the vehicle 100 (e.g., the roof or surface 100u) to mount the stacked arrays 102U and 102L to the vehicle 100.

[0119] The x-dimension X-dim and y-dimension Y-dim of the acoustic radiation structure 1703 can be determined in part by the wavelength (e.g., sound frequency) for which the array 102 is designed. As an example, the array 102 can be designed to emit sounds in a frequency range of about 250 Hz to about 8 kHz or in a range of about 400 Hz to about 4 kHz. For frequencies below about 400 Hz, bandpass filtering of the audio signal (e.g., using a low pass filter LPF) can be used to attenuate frequencies below about 400 Hz. Similarly, for frequencies above about 4 kHz, bandpass filtering of the audio signal (e.g., using a high pass filter HPF) can be used to attenuate frequencies above about 4 kHz. For speakers selected to emit sounds at frequencies audible to humans such as pedestrians, the x-dimension X-dim and y-dimension Y-dim can be in the range of about 20 mm to about 60 mm.

[0120] In other examples, the array 102 may include speakers designed to emit sound at ultrasonic frequencies beyond the range of human hearing. For speakers selected to emit sound at ultrasonic frequencies, the x-dimension X-dim and the y-dimension Y-dim may be in the range of about 2 mm to about 8 mm. An array 102 having ultrasonic transducers for its speakers may have 300 or more (e.g., n 300) ultrasonic transducers in the array 102, as opposed to about one-tenth the number of speakers for an array 102 having transducers emitting sound at frequencies compatible with human hearing.

[0121] Sound 104 transmitted at ultrasonic frequencies may be distorted downward in the atmosphere to human audible frequencies (e.g., below about 20 kHz) at the object toward which the beam 104 is directed, such that at the location of the object, the sound waves are still audibly perceived as an acoustic alarm even though the original beam source is at ultrasonic frequencies. For beams 104 transmitted at ultrasonic frequencies, the beam width at the location of the object may be narrower (e.g., from about 4 degrees to about 10 degrees). In contrast, beams 104 transmitted at frequencies within the range of human hearing may have beam widths in the range of about 20 degrees to about 35 degrees.

[0122] Sample Clauses

[0123] 1. A method comprising:

[0124] Compute the trajectory of the autonomous vehicle in the environment;

[0125] determining a location of an object disposed in the environment, the object having an object type;

[0126] predicting an object trajectory of the object based at least in part on the object type and the location of the object;

[0127] estimating a threshold position in the environment based at least in part on the object trajectory and the trajectory of the autonomous vehicle, the threshold position associated with an acoustic alert;

[0128] selecting an audio signal to be associated with the acoustic alarm;

[0129] detecting that the position of the object is consistent with the threshold position;

[0130] determining a propagation direction to transmit a guided acoustic energy beam based at least in part on the position of the object; and

[0131] An acoustic beam steering array is caused to transmit the guided acoustic energy beam indicative of the audio signal into the environment along the propagation direction.

[0132] 2. The method of clause 1, further comprising identifying a change in driving operation of the autonomous vehicle; and wherein selecting the audio signal is based at least in part on the change in driving operation.

[0133] 3. The method of clause 2, further comprising:

[0134] capturing sounds associated with the driving operation of the autonomous vehicle, and

[0135] One or more characteristics of the audio signal are adjusted based at least in part on the sound.

[0136] 4. The method of clause 1, wherein the threshold position comprises a plurality of threshold positions, and wherein selecting the audio signal comprises selecting the audio signal based at least in part on a threat level.

[0137] 5. The method of clause 1, further comprising:

[0138] detecting a second object in the environment;

[0139] identifying an object type of the second object;

[0140] associating a priority with the object and the second object based at least in part on the object type of the object and the object type of the second object; and

[0141] Determining to process data associated with the object before processing data associated with the second object is based at least in part on the object having a higher priority than the second object.

[0142] 6. The method of clause 5, wherein associating the priority comprises determining one or more of the motion of the object and the second object, the position of the object and the second object, the predicted motion of the object and the second object, and the trajectory of the autonomous vehicle.

[0143] 7. The method of clause 1, further comprising activating a system of the autonomous vehicle based at least in part on the trajectory of the object, wherein the system is one or more of a braking system, a propulsion system, and a signaling system.

[0144] 8. The method of clause 1, wherein causing the acoustic beam steering array to emit the guided acoustic energy beam comprises selecting the acoustic beam steering array from at least the acoustic beam steering array and a second acoustic beam steering array based at least in part on a direction of travel of the autonomous vehicle, wherein the autonomous vehicle is configured for bidirectional travel, and wherein the acoustic beam steering array is disposed on a first half of the autonomous vehicle and the second acoustic beam steering array is disposed on a second half of the autonomous vehicle.

[0145] 9. The method of clause 1, further comprising:

[0146] calculating a first signal delay associated with a first speaker disposed in the acoustic beamsteering array and a second signal delay associated with a second speaker disposed in the acoustic beamsteering array; and

[0147] The audio signal is delayed one or more times applied to a signal input of one or more amplifiers coupled to the first speaker and the second speaker based at least in part on one or more of the first delay signal or the second delay signal.

[0148] 10. The method of clause 9, wherein the first speaker is adjacent to the second speaker, and wherein the first speaker is separated from the second speaker by a wavefront distance, and wherein the signal delay is calculated based at least in part on the wavefront distance and the speed of sound.

[0149] 11. The method of clause 1, further comprising:

[0150] accessing speaker data associated with a first speaker in the acoustic beamsteering array and a second speaker in the acoustic beamsteering array, the speaker data comprising a first speaker position in the acoustic beamsteering array and a second speaker position in the acoustic beamsteering array;

[0151] calculating a first gain associated with the first speaker and a second gain associated with the second speaker, wherein one or more of the first speaker or the second speaker is coupled to an output of an amplifier having a gain input; and

[0152] One or more of the first gain or the second gain is applied to the gain input.

[0153] 12. The method of clause 1, further comprising:

[0154] accessing driving operation sounds of the autonomous vehicle; and

[0155] The audio signal is modulated using the driving operation sound.

[0156] 13. The method of clause 1, further comprising:

[0157] generating said position of said object at a perception system;

[0158] receiving the position of the object at the acoustic beamsteering array, the acoustic beamsteering array in communication with the perception system;

[0159] receiving, in a planner system in communication with a perception system, said position of said object, object tracking, and data representing a classification of said object;

[0160] accessing, by the planner system, an object type;

[0161] comparing the object classification and the object tracking to data representative of the object type at the planner system to determine the object type;

[0162] accessing object dynamics by the planner system; and

[0163] The object type and the position of the object are compared at the planner system with data representing dynamics of the object to determine the object trajectory.

[0164] 14. A system comprising:

[0165] a sensor operable to generate sensor data of an autonomous vehicle located within the environment;

[0166] an acoustic beam steering array operable to transmit guided acoustic energy into the environment; and

[0167] One or more processors configured to perform actions comprising:

[0168] determining a location of an object disposed in the environment based at least in part on the sensor data;

[0169] selecting an audio signal to be associated with the acoustic alarm;

[0170] determining a propagation direction to transmit a guided acoustic energy beam, wherein the direction is based at least in part on the position of the object; and

[0171] The acoustic beam steering array is caused to emit the guided beam of acoustic energy indicative of the audio signal into the environment along the propagation direction.

[0172] 15. The system of clause 14, wherein the actions further comprise:

[0173] predicting a trajectory of the object based at least in part on an object type associated with the object and the location of the object;

[0174] estimating a threshold position in the environment based at least in part on the object trajectory; and

[0175] Wherein causing the acoustic beam steering array to transmit the guided acoustic energy beam is based at least in part on detecting that the object is consistent with the threshold location.

[0176] 16. The system of claim 14, further comprising one or more microphones operable to capture sound; and wherein the action further comprises identifying a change in driving operation of the autonomous vehicle based at least in part on the sound; and wherein selecting the audio signal is based at least in part on the change in driving operation.

[0177] 17. The system of clause 14, further comprising one or more microphones operable to capture sound; and wherein the actions further comprise adjusting one or more characteristics of the audio signal based at least in part on the sound.

[0178] 18. The system of clause 14, wherein the threshold positions include a first threshold position associated with a first threat level and a second threshold position associated with a second threat level, and wherein selecting the audio signal includes selecting the first audio signal based at least in part on identifying the first threat level and selecting the second audio signal based at least in part on identifying the second threat level.

[0179] 19. The system of clause 14, wherein the acoustic beamsteering array comprises one or more amplifiers coupled to a plurality of speakers comprising a first speaker and a second speaker.

[0180] 20. The system of clause 19, wherein the actions further comprise:

[0181] determining a first signal delay associated with the first speaker;

[0182] determining a second signal delay associated with the second speaker; and

[0183] The audio signal delay is applied one or more times to a signal input of at least one of the one or more amplifiers based at least in part on one or more of the first signal delay or the second signal delay.

[0184] 21. The system of clause 19, wherein the first speaker is adjacent to the second speaker, and wherein the first speaker is separated from the second speaker by a wavefront distance, and wherein the signal delay is calculated based at least in part on the wavefront distance.

[0185] 22. The system of clause 19, wherein the actions further comprise:

[0186] accessing speaker data associated with the first speaker and the second speaker, the speaker data comprising a first speaker position in the acoustic beamsteering array and a second speaker position in the acoustic beamsteering array;

[0187] calculating a first gain associated with the first speaker and a second gain associated with the second speaker, wherein one or more of the first speaker or the second speaker is coupled to an output of an amplifier having a gain input; and

[0188] One or more of the first gain or the second gain is applied to the gain input.

[0189] 23. The system of clause 14, wherein the actions further comprise:

[0190] accessing driving operation sounds of the autonomous vehicle; and

[0191] The audio signal is modulated using the driving operation sound.

[0192] 24. The system of clause 14, wherein the actions further comprise:

[0193] determining an object type of the object;

[0194] accessing object dynamics based at least in part on the object type; and

[0195] determining a trajectory of the object based at least in part on the object dynamics; and

[0196] Wherein determining the propagation direction to transmit the guided acoustic energy beam is based at least in part on the trajectory.

[0197] 25. The system of clause 14, wherein the actions further comprise:

[0198] detecting a second object in the environment;

[0199] identifying an object type of the second object;

[0200] associating a priority with the object and the second object based at least in part on the object type of the object and the object type of the second object; and

[0201] Determining to process data associated with the object before processing data associated with the second object is based at least in part on the object having a higher priority than the second object.

[0202] 26. The system of clause 25, wherein associating the priority comprises determining one or more of the motion of the object and the second object, the position of the object and the second object, the predicted motion of the object and the second object, and the trajectory of the autonomous vehicle.

[0203] 27. The system of clause 15, wherein the action further comprises activating a system of the autonomous vehicle based at least in part on the trajectory of the object, wherein the system is one or more of a braking system, a propulsion system, and a signaling system.

[0204] 28. The system of clause 14, further comprising a second acoustic beam steering array disposed on a second half of the autonomous vehicle, wherein the acoustic beam steering array is disposed on a first half of the autonomous vehicle, and wherein the autonomous vehicle is configured for bidirectional travel.

[0205] Although the foregoing examples have been described in some detail for the purpose of clear understanding, the above-described conceptual technology is not limited to the details provided. There are many alternative ways to implement the above-described conceptual technology. The disclosed examples are illustrative rather than restrictive.

Claims

1. A system for implementing an acoustic beam steering array, comprising: an acoustic beamsteering array comprising a plurality of speakers configured to output directional audio; and One or more processors configured to perform operations comprising: receiving a first signal indicative of an object type and an object location, and optionally a first signal indicative of an object trajectory or a predicted intersection of the object trajectory and a vehicle trajectory, the first signal being associated with a first object; determining, based at least in part on the first signal, a direction in which to transmit a beam of acoustic energy via at least a portion of the plurality of speakers of an acoustic beamsteering array; receiving a second signal indicative of a second object, the second object being different from the first object; determining, based at least in part on the first signal and the second signal, a first gain setting associated with acoustic energy beams emitted by a first portion of the at least a portion of the plurality of speakers and a second gain setting associated with acoustic energy beams emitted by a second portion of the at least a portion of the plurality of speakers, wherein the first gain setting is different from the second gain setting; and causing said at least a portion of said plurality of speakers to emit a beam of acoustic energy indicative of an alarm in said direction and at a first gain setting and a second gain setting, The first gain setting and the second gain setting are configured to cause an acoustic pressure of the acoustic energy beam at the first object to be different from an acoustic pressure of the acoustic energy beam at the second object.

2. The system of claim 1, wherein: The first object is a first pedestrian, and the second object is at least one of a second pedestrian or a vehicle.

3. The system of claim 1, wherein: A first portion of the transmitted acoustic energy beam is associated with a first object and a second portion of the transmitted acoustic energy beam is associated with a second object.

4. The system of claim 1, wherein the operations further comprise: receiving an audio signal determined based at least in part on the first signal; as well as The at least a portion of the plurality of speakers is caused to emit audio signals as acoustic energy beams.

5. The system of claim 4, wherein: The first signal is indicative of at least a first object type, wherein the audio signal is a first audio signal based at least in part on the first signal, and wherein the acoustic energy beam is a first acoustic energy beam, the operations further comprising: receiving a second audio signal, determining the second audio signal based at least in part on a third signal indicative of at least a third object type different from the first object type; and At least a portion of the plurality of speakers is caused to emit a second audio signal as a second acoustic energy beam.

6. The system of claim 1, wherein: The second object is outside a main lobe of transmitted acoustic energy of the acoustic beamsteering array; The first signal indicates at least the position of the object, An acoustic beam steering array is coupled to the vehicle, and Determining the direction includes determining an angle between the direction of vehicle travel and the location of the object.

7. The system of claim 1, wherein the operations further comprise: determining, for a first speaker of the plurality of speakers, a first gain and a first delay associated with a first signal output via the first speaker; as well as determining, for a second speaker of the plurality of speakers, a second gain and a second delay associated with a second signal output via the second speaker; wherein the first gain, the second gain, the first delay, and the second delay are based at least in part on the direction in which the acoustic energy beam is transmitted; and Wherein the first gain and the second gain are based at least in part on a second signal indicative of a second object.

8. The system of claim 1, wherein the operations further comprise: receiving an audio signal determined based at least in part on the first signal; determining to apply one or more bandpass filters to the audio signal to produce a filtered audio signal; as well as At least a portion of the plurality of speakers is caused to transmit the filtered audio signal as an acoustic energy beam.

9. The system of claim 1, wherein: The direction is a first direction, and the operations further include: determining a second direction in which to transmit a beam of acoustic energy via at least a portion of the plurality of speakers of the acoustic beamsteering array; determining a first gain in one or more channels associated with the at least a portion of the plurality of speakers, the first gain being associated with a first direction; and A third gain in the one or more channels associated with the at least a portion of the plurality of speakers is determined, the third gain being associated with a second direction.

10. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform actions comprising: receiving, at an acoustic beam steering array including a plurality of speakers, a first signal indicative of an object type and an object location, and optionally a first signal indicative of an object trajectory or a predicted intersection between the object trajectory and a vehicle trajectory, the first signal being associated with a first object; determining a direction in which to transmit a beam of acoustic energy via at least a portion of the plurality of speakers of an acoustic beamsteering array; receiving a second signal indicative of a second object, the second object being different from the first object; determining, based at least in part on the first signal and the second signal, a first gain setting associated with acoustic energy beams emitted by a first portion of the at least a portion of the plurality of speakers and a second gain setting associated with acoustic energy beams emitted by a second portion of the at least a portion of the plurality of speakers, wherein the first gain setting is different from the second gain setting; and causing said at least a portion of said plurality of speakers to emit a beam of acoustic energy indicative of an alarm in said direction and at a first gain setting and a second gain setting, The first gain setting and the second gain setting are configured to cause an acoustic pressure of the acoustic energy beam at the first object to be different from an acoustic pressure of the acoustic energy beam at the second object.

11. The one or more non-transitory computer-readable media of claim 10, wherein: The first signal is indicative of at least a location of the object, and wherein the direction in which the acoustic energy beam is transmitted is based at least in part on the location of the object.

12. The one or more non-transitory computer-readable media of claim 10, the actions further comprising: determining, for a first speaker of the plurality of speakers, a first gain and a first delay associated with a first signal output via the first speaker; as well as determining, for a second speaker of the plurality of speakers, a second gain and a second delay associated with a second signal output via the second speaker; wherein the first gain, the second gain, the first delay, and the second delay are based at least in part on the direction in which the acoustic energy beam is transmitted; and Wherein the first gain and the second gain are based at least in part on a second signal indicative of a second object.

13. The one or more non-transitory computer-readable media of claim 10, the actions further comprising: receiving an audio signal determined based at least in part on the first signal; determining to apply one or more bandpass filters to the audio signal to produce a filtered audio signal; as well as The at least a portion of the plurality of speakers is caused to transmit the filtered audio signal as an acoustic energy beam.

14. The one or more non-transitory computer-readable media of claim 10, wherein: The direction is a first direction, and the action further includes: determining a second direction in which to transmit a beam of acoustic energy via at least a portion of the plurality of speakers of the acoustic beamsteering array; determining a first gain in one or more channels associated with the at least a portion of the plurality of speakers, the first gain being associated with a first direction; and A third gain in the one or more channels associated with the at least a portion of the speakers is determined, the third gain being associated with a second direction.

15. A method for implementing an acoustic beam steering array, comprising: receiving, by one or more processors, a first signal indicative of an object type and an object location, and optionally a first signal indicative of an object trajectory or a predicted intersection between the object trajectory and a vehicle trajectory, the first signal being associated with a first object; Receiving, by the one or more processors, a direction in which a beam of acoustic energy is emitted via at least a portion of a plurality of speakers of an acoustic beamsteering array; receiving, by the one or more processors, a second signal indicative of a second object, the second object being different than the first object; determining, by the one or more processors and based at least in part on the first signal and the second signal, a first gain setting associated with acoustic energy beams emitted by a first portion of the at least a portion of the plurality of speakers and a second gain setting associated with acoustic energy beams emitted by a second portion of the at least a portion of the plurality of speakers, wherein the first gain setting is different from the second gain setting; and causing, by the one or more processors, the at least a portion of the plurality of speakers to emit a beam of acoustic energy indicative of an alarm in the direction and at the first gain setting and the second gain setting, The first gain setting and the second gain setting are configured to cause an acoustic pressure of the acoustic energy beam at the first object to be different from an acoustic pressure of the acoustic energy beam at the second object.

16. The method of claim 15, further comprising: receiving an audio signal determined based at least in part on the first signal; as well as The at least a portion of the plurality of speakers is caused to emit audio signals as acoustic energy beams.

17. The method of claim 15, wherein: The second object is outside a main lobe of transmitted acoustic energy of the acoustic beamsteering array; The first signal indicates at least the position of the object, an acoustic beam steering array coupled to the vehicle, and Receiving the direction includes receiving an angle between the direction of vehicle travel and the location of the object.

18. The method of claim 15, further comprising: receiving an audio signal determined based at least in part on the first signal; determining to apply one or more bandpass filters to the audio signal to produce a filtered audio signal; as well as The at least a portion of the plurality of speakers is caused to transmit the filtered audio signal as an acoustic energy beam.

19. The method of claim 15, wherein: The direction is a first direction, and the method further comprises: determining a second direction in which to transmit a beam of acoustic energy via at least a portion of the plurality of speakers of the acoustic beamsteering array; determining a first gain in one or more channels associated with the at least a portion of the plurality of speakers, the first gain being associated with a first direction; and A third gain in one or more channels associated with the at least a portion of the speakers is determined, the third gain being associated with a second direction.

Images