Vehicle sensor module with external audio receiver

Abstract

Example embodiments relate to vehicle sensor modules with external audio receivers. An example sensor module can include a sensor and can be coupled to a roof of a vehicle with a first microphone positioned proximate a front of the sensor module. The sensor module can also include a second microphone extending into a first side of the sensor module such that the second microphone is configured to detect audio originating from an environment located on a first side of the vehicle, and a third microphone extending into a second side of the sensor module such that the third microphone is configured to detect audio originating from an environment located on a second side of the vehicle, where the second side is opposite the first side.

Description

Technical Field

[0001] This disclosure relates to a vehicle sensor module with an external audio receiver. Background Technology

[0002] Advances in computing, sensors, and other technologies have enabled vehicles to navigate safely and autonomously between locations without requiring input from a human driver. By processing sensor measurements of the surrounding environment in near real-time, autonomous vehicles can safely transport passengers or objects (e.g., cargo) between locations while avoiding obstacles, complying with traffic rules, and performing other actions typically performed by a driver. Delegating both decision-making and control of the vehicle to the vehicle system allows passengers to focus their attention on tasks other than driving. Summary of the Invention

[0003] The example embodiments described herein relate to a vehicle sensor module with an external audio receiver. By strategically arranging a recessed external audio receiver on the vehicle sensor module, the vehicle system can use audio measurements of the surrounding environment captured by the receiver to locate alarms and other noises relative to the vehicle during navigation. This strategic arrangement allows the audio receiver to clearly convert external audio into electrical signals while minimizing unwanted noise interference from wind and vibrations of vehicle components.

[0004] In one aspect, an example system is provided. The system includes a vehicle and a sensor module coupled to the roof of the vehicle, such that a gap is formed between a portion of the bottom surface of the sensor module and the roof. The sensor module includes one or more sensors. The system also includes a microphone assembly coupled to the sensor module. The microphone assembly includes: (i) a first microphone positioned adjacent to the front of the sensor module, wherein the first microphone extends into a given portion of the bottom surface of the sensor module adjacent to the gap; (ii) a second microphone extending into a first side of the sensor module, such that the second microphone is configured to detect audio originating from an environment extending from the first side of the vehicle; and (iii) a third microphone extending into a second side of the sensor module, such that the third microphone is configured to detect audio originating from an environment extending from the second side of the vehicle. The second side is opposite to the first side.

[0005] In another aspect, an example sensor module is provided. The sensor module includes one or more sensors and a microphone set, the microphone set including a first microphone positioned adjacent to the front of the sensor module. The first microphone extends into a given portion of the bottom surface of the sensor module. The microphone set also includes a second microphone extending into a first side of the sensor module, such that the second microphone is configured to detect audio originating from the environment extending from the first side of the sensor module, and a third microphone extending into a second side of the sensor module, such that the third microphone is configured to detect audio originating from the environment extending from the second side of the sensor module, wherein the second side is opposite to the first side.

[0006] In another aspect, an example method is provided. This method involves receiving audio data at a computing device from a set of microphones located on a sensor module coupled to the roof of a vehicle. The audio data represents one or more sounds originating from the vehicle's environment. The microphone set includes a first microphone positioned adjacent to the front of the sensor module, wherein the first microphone extends into a given portion of the bottom surface of a gap adjacent to the sensor module; a second microphone extending into a first side of the sensor module, such that the second microphone is configured to detect audio originating from the environment extending from the first side of the vehicle; and a third microphone extending into a second side of the sensor module, such that the third microphone is configured to detect audio originating from the environment extending from the second side of the vehicle, wherein the second side is opposite to the first side. The method also involves determining the direction of a particular sound relative to the vehicle based on the audio data; and controlling the vehicle based on determining the direction of the particular sound relative to the vehicle.

[0007] The foregoing overview is merely illustrative and is not intended to be limiting in any way. Further aspects, embodiments, and features will become clear from the accompanying drawings and the following detailed description, in addition to the illustrative aspects, embodiments, and features described above. Attached Figure Description

[0008] Figure 1 This is a functional block diagram illustrating a vehicle according to an example implementation.

[0009] Figure 2A A side view of a vehicle according to one or more example embodiments is shown.

[0010] Figure 2B A top view of a vehicle according to one or more example embodiments is shown.

[0011] Figure 2C A front view of a vehicle according to one or more example embodiments is shown.

[0012] Figure 2D A rear view of a vehicle according to one or more example embodiments is shown.

[0013] Figure 2E Additional views of a vehicle according to one or more example embodiments are shown.

[0014] Figure 3 It is a simplified block diagram of a computing system according to one or more example embodiments.

[0015] Figure 4 A vehicle sensor module with an external audio receiver is shown according to one or more example embodiments.

[0016] Figure 5 A vehicle equipped with a vehicle sensor module is shown according to one or more example embodiments.

[0017] Figure 6 A scenario is illustrated where a vehicle uses an external audio receiver to locate an emergency vehicle, according to one or more example embodiments.

[0018] Figure 7A Another vehicle sensor module with an external audio receiver is shown according to one or more example embodiments.

[0019] Figure 7B A vehicle sensor module with an external audio receiver, positioned on a truck, is shown according to one or more exemplary embodiments.

[0020] Figure 7C An external audio receiver cover is shown according to one or more example embodiments.

[0021] Figure 8 This is a flowchart of a method according to one or more example embodiments.

[0022] Figure 9 It is a schematic diagram of a computer program according to one or more example embodiments. Detailed Implementation

[0023] In the following detailed description, reference is made to the accompanying drawings, which form part of the description. In the drawings, similar symbols generally identify similar components unless the context indicates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that, as generally described herein and shown in the drawings, aspects of this disclosure can be arranged, substituted, combined, separated, and designed in a variety of different configurations, all of which are explicitly contemplated herein.

[0024] Some vehicles can use sensors to measure and navigate based on elements in their surrounding environment, such as road obstacles, traffic signals and signs, pedestrians, and other vehicles. To navigate in autonomous or semi-autonomous modes, vehicle systems can perform a variety of tasks typically handled by human drivers. For example, a vehicle system might determine the location of nearby objects and predict their future state, taking traffic signals and road boundaries as factors into account as part of determining the vehicle's control strategy.

[0025] While many navigation operations are performed based on spatial information representing the location of boundaries and objects in the surrounding environment, vehicle systems can also utilize audio measurements from the environment to determine the vehicle's control strategy. For example, microphones or other types of audio receivers can be positioned outside the vehicle to convert external sounds into electrical signals for use by the vehicle systems. The vehicle systems can use incoming audio information to identify various sounds in the surrounding environment, such as emergency vehicle sirens, vehicle horns, and audio warnings from traffic controllers. However, when the vehicle is navigating, wind and vibrations of vehicle components can add unwanted noise to the audio measurements captured by external audio receivers.

[0026] The example embodiments described herein relate to vehicle sensor modules that incorporate external audio receivers in their arrangement, minimizing the effects of wind, vibrations from vehicle components, and other undesirable noise sources during navigation. Some example vehicle sensor modules include multiple microphone modules strategically positioned at different locations on the sensor module to allow the vehicle system to extract additional information about the environment based on ambient sound. For example, a computing device can use sound detection to identify which microphones are receiving a specific sound (e.g., an emergency siren) and compare the power levels of the detected sound to pinpoint the source of the sound relative to the vehicle. In some cases, the vehicle system can use audio information captured by the microphones to detect when and from which direction the emergency vehicle is approaching before the camera system captures an image of the emergency vehicle. As an example, the vehicle system can perform a pull-over maneuver or another control strategy to clear the path for the emergency vehicle.

[0027] Example embodiments of sensor modules with external audio receivers may vary in configuration and may depend on the specific application of the sensor module. Some exemplary sensor modules presented herein are designed to add sensors for passenger vehicles, while other exemplary sensor modules have configurations implemented for use on trucks and other large vehicles. Furthermore, a human driver can use an external microphone during navigation to hear sounds occurring around the vehicle. For example, the vehicle system may reduce or temporarily suspend the vehicle's internal media system in response to detecting the presence of a nearby alarm, so that the driver can clearly hear the alarm and respond accordingly. In some examples, the audio receiver on the sensor module may also be bidirectional. In particular, the driver or other passengers can speak through an audio system built into the sensor module for other drivers and pedestrians to hear. The audio system can amplify and increase the volume of the voice provided by the driver or passenger. In some examples, the vehicle system may also implement a feature that discards recordings obtained by an external microphone.

[0028] In some cases, vehicle sensor modules can be configured to be positioned on the roof of the vehicle. For example, a vehicle sensor module may include one or more cameras, lidar, radar, and / or other types of sensors that operate according to the field of view of being positioned on the top of the vehicle's roof. By incorporating one or more microphones or other types of audio receivers into a vehicle sensor module that can be mounted on the roof, the vehicle system can detect ambient sound without requiring microphones mounted in other external locations on the vehicle. Furthermore, external placement of one or more microphones can allow for better detection of external sounds compared to microphones placed inside the vehicle. In other examples, sensor modules with external audio receivers can be configured to be coupled to other parts of the vehicle.

[0029] In some embodiments, multiple microphone modules are located at different positions on a vehicle sensor module. A microphone module, also referred to herein as a microphone, means one or more audio receivers configured to convert sound into electrical signals representing audio information that a computing device can analyze and use. For example, a vehicle sensor module may include microphone modules arranged at different locations, each oriented towards a specific area of ​​the environment relative to the vehicle for primary detection. The computing device can use incoming audio measurements from the different microphones to locate a specific sound relative to the vehicle. In some cases, the computing device may also estimate the rate at which a specific sound travels towards or away from the vehicle. Continuous measurements of external sounds can provide information related to the rate of travel of a specific sound. For example, the detection of an alarm may increase in volume when an emergency vehicle approaches a vehicle equipped with a sensor module having external audio receivers, and decrease in volume when the emergency vehicle navigates away from the vehicle.

[0030] For example, a roof-mountable sensor module may include four microphones strategically positioned on different parts of the sensor module. When mounted on a vehicle, one microphone may be positioned relative to the front of the sensor module, which extends closest to the front of the vehicle, while another microphone may be positioned on the rear of the sensor module, which extends towards the rear of the vehicle. This allows the microphones to clearly detect sounds originating from the front and rear areas of the vehicle, respectively. The sensor module may also include microphones located on the sides of the module, which allows for even clearer detection of sounds originating from the sides of the vehicle. Although in some cases, microphones on the sensor module can detect sound regardless of its location, the orientation of the microphones on the sensor module can allow some of them to better receive external sounds originating from specific areas relative to the vehicle. This may be due to the cavities, angles, and / or orientations of the microphone modules on the sensor module, as well as certain parts of the sensor module and the vehicle, which partially affect the reception of sound waves at each microphone module.

[0031] In some examples, the sensor module can be coupled to the vehicle roof in such a way that a gap (open space) is formed between a portion of the sensor module and the roof. This allows one or more microphones (e.g., a front microphone module) to be positioned on the bottom surface of the sensor module relative to the gap, enabling sound waves to enter the microphone module through the gap. When the sensor module is mounted on the vehicle, by positioning the front microphone module and / or other microphone modules near the gap, the microphones can detect audio with less impact from wind generated during forward navigation.

[0032] To further enhance the reception of sound from the external environment, in some example embodiments, one or more microphones may have recessed locations on the sensor module. For example, a recessed location may involve extending the microphone's position into a cavity positioned on the surface of the sensor module. Furthermore, the recessed location may include one or more microphones that are slightly subflushed within the sensor module. For example, one or more microphones may be positioned slightly below the surface of the sensor module. In some examples, the depth, angle, shape, and configuration of the cavity may have different parameters, depending on the cavity's position on the sensor module and the type of microphone used. For example, the cavity may place the microphone module 5-15 mm into the outer surface of the sensor module. Other depths may exist.

[0033] In some examples, the depth and configuration of the cavity used to position the microphone module (e.g., one or more audio receivers) may depend on the protective layer material integrated to protect the microphone module. This protective layer material can be designed to prevent water and debris from contacting and potentially damaging the audio receivers. Therefore, a thicker protective layer material may require a deeper cavity for the microphone module. Furthermore, in some implementations, the distance and shape of the recessed space can create unwanted resonances. To avoid unwanted resonances, the cavity can be designed via simulation and computational testing, which may involve comparing audio detection levels and intelligibility under different test conditions.

[0034] In some examples, the cavity used to position the microphone for better detection of sound originating from the front of the vehicle can be different from the cavity used to position the microphone for better detection of sound originating from the rear of the vehicle. Wind experienced during navigation, vibrations of vehicle components, the location of fans used to cool the vehicle sensors, and / or other factors can be analyzed to select the microphone's location on the vehicle sensor module. Therefore, when constructing an example vehicle sensor module with the external audio receiver described herein, wind tunnel testing, simulation, and real-world driving testing can be used.

[0035] In some examples, computing devices can use microphones on the vehicle's sensor modules to record ambient noise in the environment in which the vehicle is operating. This helps the vehicle identify and locate external objects, such as emergency vehicles, railway crossings, horns from other vehicles, and / or passenger detection. As an example, the vehicle system can perform appropriate actions during navigation. In addition to external objects, audio sensor modules can monitor the integrity of the platform vehicle, which may involve detecting sounds indicating loose parts or vibrations. Weather monitoring by analyzing road noise and raindrop patterns can help the vehicle understand the surface conditions it is navigating. In our trucking applications, the integration of microphone arrays close to the towing load helps detect loose cargo in containers or trailers.

[0036] In some examples, the vehicle sensor module may have a structure configured to couple to a semi-truck or another large vehicle. For example, the sensor module may be shaped like a surfboard or another elongated structure that, when mounted, spans the roof of the semi-truck. The shape and length of the sensor module can contribute to the aerodynamic operation of the truck when traveling in the forward direction. The sensor module can be coupled to the truck via at least one coupling component attached to either side of the truck cab roof. In this embodiment, a collection of microphones may be embedded within a curved cover piece, also mounted on the coupling component. The shape and curves of the cover piece can reduce the effects of wind during forward movement of the vehicle.

[0037] The example systems within the scope of this disclosure will now be described in more detail. The example systems may be implemented in or take the form of automobiles, but other example systems may be implemented in or take the form of other vehicles, such as cars, trucks, motorcycles, buses, boats, airplanes, helicopters, lawnmowers, bulldozers, boats, snowmobiles, aircraft, recreational vehicles, amusement park vehicles, farm equipment, construction equipment, trams, golf carts, trains, handcarts, and robotic equipment. Other vehicles are also possible.

[0038] Now refer to the attached diagram, Figure 1 This is a functional block diagram illustrating vehicle 100, representing a vehicle capable of operating fully or partially in an autonomous mode. More specifically, vehicle 100 can operate in an autonomous mode without human interaction (or with reduced human interaction) by receiving control commands from a computing system (e.g., a vehicle control system). As part of operating in autonomous mode, vehicle 100 can use sensors (e.g., sensor system 104) to detect and possibly identify objects in the surrounding environment for safe navigation. In some embodiments, vehicle 100 may also include a subsystem that enables a driver (or remote operator) to control the operation of vehicle 100.

[0039] like Figure 1 As shown, vehicle 100 includes various subsystems, such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, a data storage device 114, and a user interface 116. The subsystems and components of vehicle 100 can be interconnected in various ways (e.g., wired or secure wireless connections). In other examples, vehicle 100 may include more or fewer subsystems. Furthermore, the functionality of vehicle 100 described herein can be divided into additional functions or physical components, or combined into fewer functions or physical components in embodiments.

[0040] The propulsion system 102 may include one or more components operable to provide powered motion to the vehicle 100, and may include an engine / motor 118, an energy source 119, a transmission 120, and wheels / tires 121, as well as other possible components. For example, the engine / motor 118 may be configured to convert the energy source 119 into mechanical energy, and may correspond to an internal combustion engine, one or more electric motors, a steam engine, or a Stirling engine, as well as one or a combination of other possible options. For example, in some embodiments, the propulsion system 102 may include multiple types of engines and / or motors, such as gasoline engines and electric motors.

[0041] Energy source 119 refers to an energy source that can power one or more systems of vehicle 100 (e.g., engine / motor 118), either wholly or partially. For example, energy source 119 may correspond to gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and / or other power sources. In some embodiments, energy source 119 may include a combination of a fuel tank, battery, capacitor, and / or flywheel.

[0042] The transmission 120 can send mechanical power from the engine / motor 118 to the wheels / tires 121 and / or other possible systems of the vehicle 100. Therefore, the transmission 120 may include a gearbox, clutch, differential, and drive shaft, as well as other possible components. The drive shaft may include an axle connected to one or more wheels / tires 121.

[0043] In an exemplary embodiment, the wheels / tires 121 of vehicle 100 can have various configurations. For example, vehicle 100 can exist as a unicycle, bicycle / motorcycle, tricycle, or four-wheeled car / truck, among other possible configurations. Therefore, the wheels / tires 121 can be attached to vehicle 100 in various ways and can be made of different materials, such as metal and rubber.

[0044] Sensor system 104 may include various types of sensors, such as a Global Positioning System (GPS) 122, an Inertial Measurement Unit (IMU) 124, one or more radar units 126, a laser rangefinder / LIDAR unit 128, a camera 130, a steering sensor 123, and a throttle / brake sensor 125, as well as other possible sensors. In some embodiments, sensor system 104 may also include sensors configured to monitor the internal systems of vehicle 100 (e.g., O2 monitor, fuel gauge, engine oil temperature, braking status).

[0045] GPS 122 may include a transceiver operable to provide information about the position of vehicle 100 relative to the Earth. IMU 124 may be configured to use one or more accelerometers and / or gyroscopes and to sense changes in the position and orientation of vehicle 100 based on inertial acceleration. For example, when vehicle 100 is stationary or moving, IMU 124 may detect the pitch and yaw of vehicle 100.

[0046] Radar unit 126 may represent one or more systems configured to use radio signals (e.g., radar signals) to sense objects within the local environment of vehicle 100, including the speed and direction of the objects. Therefore, radar unit 126 may include one or more radar units equipped with one or more antennas configured to transmit and receive radar signals as described above. In some embodiments, radar unit 126 may correspond to an mountable radar system configured to acquire measurements of the surrounding environment of vehicle 100. For example, radar unit 126 may include one or more radar units configured to be coupled to the underside of the vehicle.

[0047] The laser rangefinder / LIDAR 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components, and may operate in a coherent mode (e.g., using heterodyne detection) or an incoherent detection mode. The camera 130 may include one or more devices (e.g., a still camera or a video camera) configured to capture images of the environment of the vehicle 100.

[0048] Steering sensor 123 can sense the steering angle of vehicle 100, which may involve measuring the angle of the steering wheel or measuring an electrical signal representing the angle of the steering wheel. In some embodiments, steering sensor 123 can measure the angles of the wheels of vehicle 100, such as detecting the angle of the wheel relative to the front axle of vehicle 100. Steering sensor 123 can also be configured to measure a combination (or subset) of the steering wheel angle, an electrical signal representing the steering wheel angle, and the angles of the wheels of vehicle 100.

[0049] Throttle / brake sensor 125 can detect the throttle position or brake position of vehicle 100. For example, throttle / brake sensor 125 can measure the angle of both the accelerator pedal (throttle) and the brake pedal, or it can measure an electrical signal that represents, for example, the angle of the accelerator pedal (throttle) and / or the angle of the brake pedal. Throttle / brake sensor 125 can also measure the angle of the throttle body of vehicle 100, which may include a portion of a modulated physical mechanism (e.g., a butterfly valve or carburetor) that provides energy source 119 to engine / motor 118. Furthermore, throttle / brake sensor 125 can measure the pressure of one or more brake pads on the rotor of vehicle 100, or a combination (or subset) of the angle of the accelerator pedal (throttle) and brake pedal, an electrical signal representing the angle of the accelerator pedal (throttle) and brake pedal, the angle of the throttle body, and the pressure exerted by at least one brake pad on the rotor of vehicle 100. In other embodiments, the throttle / brake sensor 125 may be configured to measure the pressure applied to a vehicle pedal, such as a throttle or brake pedal.

[0050] The control system 106 may include components configured to assist vehicle 100 in navigation, such as a steering unit 132, a throttle valve 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a navigation / path system 142, and an obstacle avoidance system 144. More specifically, the steering unit 132 is operable to adjust the direction of vehicle 100, and the throttle valve 134 can control the operating speed of the engine / motor 118 to control the acceleration of vehicle 100. The braking unit 136 can decelerate vehicle 100, which may involve using friction to slow down the wheels / tires 121. In some embodiments, the braking unit 136 can convert the kinetic energy of the wheels / tires 121 into electrical current for subsequent use by one or more systems of vehicle 100.

[0051] Sensor fusion algorithm 138 may include Kalman filters, Bayesian networks, or other algorithms capable of processing data from sensor system 104. In some embodiments, sensor fusion algorithm 138 may provide evaluations based on incoming sensor data, such as evaluations of individual objects and / or features, evaluations of specific situations, and / or evaluations of the potential impact of a given situation.

[0052] Computer vision system 140 may include hardware and software operable to process and analyze images in an effort to determine objects, environmental objects (e.g., parking lights, road boundaries, etc.), and obstacles. Therefore, computer vision system 140 may use object recognition, structure-of-motion (SFM), video tracking, and other algorithms used in computer vision, such as object recognition, environment mapping, object tracking, and estimation of object velocity, etc.

[0053] Navigation / path system 142 can determine the driving path of vehicle 100, which may involve dynamically adjusting navigation during operation. Therefore, navigation / path system 142 can use data from sensor fusion algorithm 138, GPS 122, maps, and other sources to navigate vehicle 100. Obstacle avoidance system 144 can evaluate potential obstacles based on sensor data and enable vehicle 100's system to avoid or otherwise successfully negotiate obstacles.

[0054] like Figure 1 As shown, vehicle 100 may also include peripheral devices 108, such as a wireless communication system 146, a touchscreen 148, a microphone 150, and / or a speaker 152. Peripheral devices 108 can provide control or other elements for a user to interact with user interface 116. For example, touchscreen 148 can provide information to the user of vehicle 100. User interface 116 can also accept input from the user via touchscreen 148. Peripheral devices 108 can also enable vehicle 100 to communicate with devices such as other vehicle equipment.

[0055] The wireless communication system 146 can securely communicate wirelessly with one or more devices, either directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as WiMAX or LTE. Alternatively, the wireless communication system 146 can communicate with a wireless local area network (WLAN) using WiFi or other possible connections. The wireless communication system 146 can also communicate directly with devices using, for example, an infrared link, Bluetooth, or ZigBee. In the context of this disclosure, other wireless protocols, such as various vehicle communication systems, are also possible. For example, the wireless communication system 146 may include one or more dedicated short-range communication (DSRC) devices, which can include public and / or private data communication between vehicles and / or roadside stations.

[0056] Vehicle 100 may include a power source 110 for supplying power to components. In some embodiments, power source 110 may include a rechargeable lithium-ion or lead-acid battery. For example, power source 110 may include one or more batteries configured to provide power. Vehicle 100 may also use other types of power sources. In an example embodiment, power source 110 and energy source 119 may be integrated into a single energy source.

[0057] Vehicle 100 may also include a computer system 112 to perform operations such as those described herein. Therefore, computer system 112 may include at least one processor 113 (which may include at least one microprocessor) operable to execute instructions 115 stored in a non-transitory computer-readable medium, such as a data storage device 114. In some embodiments, computer system 112 may represent multiple computing devices that can be used to control individual components or subsystems of vehicle 100 in a distributed manner.

[0058] In some embodiments, the data storage device 114 may include instructions 115 (e.g., program logic) executable by the processor 113 to perform various functions of the vehicle 100, including those described above. Figure 1 The functions described. The data storage device 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the propulsion system 102, sensor system 104, control system 106, and peripheral devices 108.

[0059] In addition to instruction 115, data storage device 114 can store data such as road maps, route information, and other information. This information can be used by vehicle 100 and computer system 112 while vehicle 100 is operating in autonomous, semi-autonomous, and / or manual modes.

[0060] Vehicle 100 may include a user interface 116 for providing information to or receiving input from a user of vehicle 100. User interface 116 may control or enable control over the content and / or layout of interactive images that may be displayed on touchscreen 148. Furthermore, user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as wireless communication system 146, touchscreen 148, microphone 150, and speaker 152.

[0061] Computer system 112 can control the functions of vehicle 100 based on input received from various subsystems (e.g., propulsion system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can use input from sensor system 104 to estimate the outputs produced by propulsion system 102 and control system 106. Depending on the embodiment, computer system 112 can be operable to monitor many aspects of vehicle 100 and its subsystems. In some embodiments, computer system 112 can disable some or all functions of vehicle 100 based on signals received from sensor system 104.

[0062] The components of vehicle 100 can be configured to operate in a manner that interconnects with other components, either internally or externally to their respective systems. For example, in an example embodiment, camera 130 can capture multiple images that may represent information about the environmental state of vehicle 100 operating in autonomous mode. The environmental state may include parameters of the road on which the vehicle is operating. For example, computer vision system 140 may be able to identify slope (incline) or other features based on multiple images of the road. Furthermore, a combination of GPS 122 and features identified by computer vision system 140 can be used with map data stored in data storage device 114 to determine specific road parameters. Additionally, radar unit 126 can also provide information about the vehicle's surroundings.

[0063] In other words, the combination of various sensors (which may be referred to as input indication and output indication sensors) and computer system 112 can interact to provide input indications or indications of the vehicle's surrounding environment that are provided for controlling the vehicle.

[0064] In some embodiments, computer system 112 can determine various objects based on data provided by systems other than radio systems. For example, vehicle 100 may have lasers or other optical sensors configured to sense objects in the vehicle's field of view. Computer system 112 can use the outputs from various sensors to determine information about objects in the vehicle's field of view, and can determine distance and orientation information to various objects. Computer system 112 can also determine whether an object is desired or undesirable based on the outputs from various sensors. Furthermore, vehicle 100 may also include a telematics control unit (TCU) 160. TCU 160 can enable vehicle connectivity and in-flight passenger equipment connectivity via one or more wireless technologies.

[0065] although Figure 1 Various components of vehicle 100, namely wireless communication system 146, computer system 112, data storage device 114, and user interface 116, are integrated into vehicle 100; however, one or more of these components may be installed separately from or associated with vehicle 100. For example, data storage device 114 may exist partially or entirely independent of vehicle 100. Therefore, vehicle 100 can be provided in the form of device elements, which may be placed separately or together. The device elements comprising vehicle 100 may be communicatively coupled together in a wired and / or wireless manner.

[0066] Figure 2A , Figure 2B , Figure 2C , Figure 2D and Figure 2E Different views of the physical configuration of vehicle 100 are shown. Various views are included to depict example sensor locations 202, 204, 206, 208, and 210 on vehicle 100. In other examples, the sensors may be located at different positions on vehicle 100. Although vehicle 100 is... Figures 2A to 2E The vehicle is depicted as a van, but vehicle 100 in the example may have other configurations, such as truck, car, semi-trailer truck, motorcycle, bus, shuttle bus, golf cart, off-road vehicle, emergency vehicle, robotic equipment or agricultural vehicle, and other possible examples.

[0067] As described above, vehicle 100 may include sensors coupled to various external locations, such as sensor locations 202-210. Vehicle sensors include one or more types of sensors, each configured to capture information from the surrounding environment or perform other operations (e.g., communication links, obtaining overall positioning information). For example, sensor locations 202-210 may serve as locations for any combination of one or more cameras, radar, LiDAR, rangefinders, radios (e.g., Bluetooth and / or 802.11), acoustic sensors, and other possible types of sensors.

[0068] When in Figures 2A to 2E When coupled at the example sensor locations 202-210 shown, various mechanical fasteners, including permanent and non-permanent fasteners, can be used. For example, bolts, screws, clips, pins, rivets, anchors, and other types of fasteners can be used. In some examples, the sensor can be coupled to the vehicle using adhesive. In further examples, the sensor can be designed and manufactured as part of a vehicle component (e.g., part of a vehicle rearview mirror).

[0069] In some implementations, one or more sensors may be positioned at sensor locations 202-210 using a movable base operable to adjust the orientation of the sensors. The movable base may include a rotating platform capable of rotating the sensors to acquire information from multiple directions around the vehicle 100. For example, a sensor located at sensor location 202 may use a movable base capable of rotating and scanning within a specific angular and / or azimuth range. Therefore, the vehicle 100 may include mechanical structures enabling the mounting of one or more sensors on the top of the roof of the vehicle 100. Furthermore, other mounting locations are possible in this example. In some cases, sensors coupled to these locations may provide data that can be used by a remote operator to assist the vehicle 100.

[0070] Figure 3This is a simplified block diagram illustrating computing device 300, showing some of the components that may be included in a computing device arranged to operate according to embodiments herein. Computing device 300 may be a client device (e.g., a device actively operated by a user (e.g., a remote operator), a server device (e.g., a device providing computing services to client devices), or some other type of computing platform. In some embodiments, computing device 300 may be implemented as computer system 112, which may be located on vehicle 100 and perform processing operations related to vehicle operation. For example, computing device 300 may be used to process sensor data received from sensor system 104. Alternatively, computing device 300 may be located remotely from vehicle 100 and communicate via secure wireless communication. For example, computing device 300 may operate as a remotely located device that a remote operator can use to communicate with one or more vehicles.

[0071] exist Figure 3 In the example embodiment shown, computing device 300 includes processing system 302, memory 304, input / output unit 306, and network interface 308, all of which can be coupled via system bus 310 or a similar mechanism. In some embodiments, computing device 300 may include other components and / or peripheral devices (e.g., removable storage devices, sensors, etc.).

[0072] Processing system 302 can be one or more of any type of computer processing element, such as a central processing unit (CPU), a coprocessor (e.g., a math, graphics, or cryptographic coprocessor), a digital signal processor (DSP), a network processor, and / or an integrated circuit or controller that performs processor operations. In some cases, processing system 302 can be one or more single-core processors. In other cases, processing system 302 can be one or more multi-core processors with multiple independent processing units. Processing system 302 may also include register memory for temporarily storing instructions being executed and associated data, and cache memory for temporarily storing recently used instructions and data.

[0073] Memory 304 can be any form of computer-usable memory, including but not limited to random access memory (RAM), read-only memory (ROM), and non-volatile memory. This can include flash memory, hard disk drives, solid-state drives, rewritable optical discs (CDs), rewritable digital video discs (DVDs), and / or magnetic tape storage devices, to name just a few examples.

[0074] Computing device 300 may include fixed memory and one or more removable memory units, including but not limited to various types of secure digital storage (SD) cards. Therefore, memory 304 may represent both main memory units and long-term storage devices. Other types of memory may include biological memory.

[0075] Memory 304 may store program instructions and / or data on which the program instructions can operate. For example, memory 304 may store such program instructions on a non-transitory computer-readable medium such that the instructions are executable by processing system 302 to perform any method, process, or operation disclosed in this specification or the accompanying drawings.

[0076] like Figure 3 As shown, memory 304 may include firmware 314A, kernel 314B, and / or application 314C. Firmware 314A may be some or all of the program code used to boot or otherwise start computing device 300. Kernel 314B may be an operating system, including modules for memory management, process scheduling and management, input / output, and communication. Kernel 314B may also include a device driver that allows the operating system to communicate with hardware modules of computing device 300, such as memory units, network interfaces, ports, and buses. Application 314C may be one or more user-space software programs, such as web browsers or email clients, and any software libraries used by these programs. In some examples, application 314C may include one or more neural network applications and other deep learning-based applications. Memory 304 may also store data used by these and other programs and applications.

[0077] Input / output unit 306 facilitates interaction between users and peripheral devices and computing device 300 and / or other computing systems. Input / output unit 306 may include one or more types of input devices, such as a keyboard, mouse, one or more touchscreens, sensors, biosensors, etc. Similarly, input / output unit 306 may include one or more types of output devices, such as a screen, monitor, printer, speaker, and / or one or more light-emitting diodes (LEDs). Additionally or alternatively, computing device 300 may communicate with other devices using, for example, a Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) port interface. In some examples, input / output unit 306 may be configured to receive data from other devices. For example, input / output unit 306 may receive sensor data from vehicle sensors.

[0078] like Figure 3As shown, the input / output unit 306 includes a GUI 312, which can be configured to provide information to a remote operator or another user. The GUI 312 may involve one or more display interfaces, or another type of mechanism for conveying information and receiving input. In some examples, the representation of the GUI 312 may vary depending on the vehicle context. For example, the computing device 300 may provide the GUI 312 in a specific format, such as one with a single optional option for a remote operator to select from.

[0079] Network interface 308 may take the form of one or more wired interfaces, such as Ethernet (e.g., Fast Ethernet, Gigabit Ethernet, etc.). Network interface 308 may also support communication over one or more non-Ethernet media, such as coaxial cable or power lines, or communication over wide area media, such as Synchronous Fiber Network (SONET) or Digital Subscriber Line (DSL) technologies. Network interface 308 may additionally take the form of one or more wireless interfaces, such as IEEE 802.11 (Wifi), BLUETOOTH®, Global Positioning System (GPS), or wide area wireless interfaces. However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols may be used on network interface 308. Furthermore, network interface 308 may include multiple physical interfaces. For example, some embodiments of computing device 300 may include Ethernet, BLUETOOTH®, and Wifi interfaces. In some embodiments, network interface 308 may enable computing device 300 to connect to one or more vehicles to allow the remote assistance technologies presented herein.

[0080] In some embodiments, one or more instances of computing device 300 may be deployed to support a cluster architecture. The exact physical location, connectivity, and configuration of these computing devices may be unknown and / or insignificant to the client devices. Therefore, the computing device may be referred to as a “cloud-based” device, which can be housed in various remote data center locations. Furthermore, computing device 300 enables the performance capabilities of the embodiments described herein, including the efficient distribution and processing of sensor data.

[0081] Figure 4 A vehicle sensor module 400 with external audio receivers 402, 404, 406, and 408 is illustrated according to one or more example embodiments. In the example embodiment, the vehicle sensor module 400 is shown in a bottom perspective view to show the example locations of the audio receivers 402-408. Figure 4As shown, the vehicle sensor module 400 includes a housing 401, audio receivers 402-408, coupling components 410A and 410B, an extension 411, sensors 412 and 414, and other components. In other examples, the vehicle sensor module 400 may have other configurations with different arrangements of components, such as audio receivers 402-408 and sensors 412-414. Furthermore, in this example, the vehicle sensor module 400 and other components may differ in size, shape, and material. The front side 416 of the vehicle sensor module 400 is configured to be positioned closest to the front of the vehicle, and the rear side 418 of the vehicle sensor module 400 is configured to be positioned closest to the rear of the vehicle.

[0082] The vehicle sensor module 400 is configured to be mounted on the roof of a vehicle via coupling components 410A and 410B. When the sensor module is coupled to the vehicle, the front side 416 of the sensor module 400 is aligned with the front of the vehicle, while the rear side 418 of the sensor module 400 is aligned with the rear of the vehicle. When facing this direction, the audio receiver 404 can be positioned relative to the right-hand side of the vehicle, and the audio receiver 406 can be positioned relative to the left-hand side of the vehicle. In other examples, the vehicle sensor module 400 can be configured to be mounted in other locations on the vehicle, such as on one side of the vehicle, near the front of the vehicle, or near the rear of the vehicle. In some examples, the vehicle sensor module 400 can be configured to be coupled to the roof of the vehicle such that a gap is formed between a portion of the bottom surface 405 of the vehicle sensor module 400 and the roof of the vehicle via coupling components 410A and 410B. For example, when the vehicle sensor module 400 is mounted to the roof of the vehicle, coupling components 410A and 410B can be configured to extend from a first side of the vehicle to a second side of the vehicle. As an example, a gap can be formed between a portion of the bottom surface 405 of the vehicle sensor module 400 and the roof of the vehicle, allowing cool air to flow and cool the sensors 412-414. For example, airflow from navigation can enter the space between the bottom surface 405 of the vehicle sensor module 400 and the roof of the vehicle from the front and sides. In some examples, the coupling assembly 410B can be further positioned to allow warm air heated by sensor operation to flow out toward the rear of the vehicle. This airflow configuration can increase the reception of external audio by the audio receivers 402-408 while allowing warm air to be removed from the sensor module to keep the sensors operating in cooler conditions and prevent overheating.

[0083] Other coupling configurations are also possible. For example, in another example embodiment, the vehicle sensor module 400 can be coupled to the vehicle using a different number of coupling components, which may have different structures in some implementations. For example, the coupling components do not necessarily have to be as follows: Figure 4 Instead of the drawn long rectangular bar, it can be replaced by something like a round rod, a solid platform, or other possible mountable shapes.

[0084] Audio receivers 402, 404, 406, and 408 represent one or more devices capable of converting external sounds from the vehicle's environment into electrical signals for analysis by the vehicle's systems. In some examples, one or more audio receivers 402-408 may include one or more microphones. For example, in one embodiment, audio receiver 402 may be a single microphone. However, in another embodiment, external audio receiver 402 may include a collection of two or more microphones contained in a single module.

[0085] Audio receiver 402 is shown positioned in front of the first coupling assembly 410A and extending into a cavity in the bottom surface of the vehicle sensor module 400 adjacent to the gap between the vehicle sensor module 400 and the roof of the vehicle. Furthermore, audio receivers 404 and 406 are shown positioned in front of the second coupling assembly 410B, angled into the respective lower side of the vehicle sensor module 400 near the gap between the vehicle sensor module 400 and the roof of the vehicle. Additionally, external audio receiver 404 is configured to detect audio originating from an environment extending from the right side of the vehicle, and external audio receiver 406 is configured to detect audio originating from an environment extending from the left side of the vehicle. The coupling assembly 410B also includes an extension 411 adjacent to the rear of the vehicle sensor module 400 and extending toward the rear of the vehicle, with external audio receiver 408 positioned on the extension 411. In this position and orientation on the extension 411, audio receiver 408 can be configured to detect audio originating from the rear of the vehicle.

[0086] The vehicle sensor module 400 is also shown having sensors 412-414, which may represent one or more types of sensors configured to capture information from the surrounding environment or perform other operations (e.g., communication links, obtaining overall positioning information). For example, sensors 412 and / or 414 may include one or more cameras, radar, lidar, rangefinders, wireless devices (e.g., Bluetooth and / or 802.11), acoustic sensors, and any combination of other possible types of sensors. Sensors 412-414 may also require other components to be attached to the vehicle sensor module 400, such as a cooling fan, processing unit, etc. Therefore, audio receivers 402-408 may be positioned at a distance of at least a threshold distance from the cooling fan and other equipment that causes noise during external sound reception.

[0087] Figure 5A vehicle 500, depicted from the left-hand side of the vehicle, is shown with external audio receiver locations 502, 504 (not shown), 506, and 508 on the vehicle sensor module 510, according to one or more example embodiments.

[0088] An external audio receiver is configured to be recessed into the vehicle sensor module 510 at a position 502 facing forward of the vehicle sensor module 510, at a position 504 (not shown) on the rear right side of the vehicle sensor module 510, at a position 506 on the rear left side of the vehicle sensor module 510, and at a position 508 facing the rear end of the vehicle sensor module 510.

[0089] Location 502 for an external audio receiver is recessed upward into the bottom surface of the gap between the vehicle sensor module 510 and the roof of the vehicle 500. The external audio receiver at location 502 may be one or more microphones configured to collect sound data originating from the area around, in front of, or near the front of the vehicle 500 (as shown in area 512).

[0090] The location 504 (not shown) for the external audio receiver is angled into the lower right-hand side of the gap between the vehicle sensor module 510 and the roof of the vehicle 500. Furthermore, the external audio receiver at location 504 is configured to detect audio originating from the environment located on the right-hand side relative to the vehicle 500. Additionally, the external audio receiver at location 504 may be one or more microphones configured to collect sound data originating from the area around, on, or near the right-hand side of the vehicle 500.

[0091] The location 506 for the external audio receiver is angled into the lower left side of the gap between the vehicle sensor module 510 and the roof of the vehicle 500. Furthermore, the external audio receiver at location 506 is configured to detect audio originating from portions of the environment near the left side of the vehicle 500. Additionally, the external audio receiver at location 506 may be one or more microphones configured to collect sound data originating from the area around, on, or near the left side of the vehicle 500 (as shown in area 514).

[0092] Location 508 for an external audio receiver is adjacent to the rear of vehicle sensor module 510 and extends toward the rear of vehicle 500, wherein the external audio receiver at location 508 is configured to detect audio originating from the environment extending toward the rear of the vehicle. Furthermore, the external audio receiver at location 508 may be one or more microphones configured to collect sound data originating from the area around, at, or near the rear of vehicle 500 (as shown in area 516).

[0093] While it is possible for all external audio receivers to receive audio from every particular direction, an external audio receiver positioned towards the direction of the noise's origin may receive the strongest noise data due to its orientation.

[0094] The vehicle sensor module 510 is shown in a left-hand perspective view, with the front of the vehicle closest to the leftmost portion of the view and the rear of the vehicle closest to the rightmost portion of the view. The vehicle sensor module 510 is configured to be coupled to the roof of the vehicle 500 such that a gap is formed between a portion of the bottom surface of the vehicle sensor module 510 and the roof of the vehicle.

[0095] In addition, the vehicle sensor module 510 may also include sensors, which may be one or more types of sensors configured to capture information from the surrounding environment or perform other operations (e.g., communication links, obtaining overall positioning information). For example, sensors may include one or more cameras, radar, lidar, rangefinders, wireless devices (e.g., Bluetooth and / or 802.11), acoustic sensors, and any combination of other possible types of sensors. Sensors may also require other components attached to the vehicle sensor module 510, such as cooling fans, processing units, etc.

[0096] Figure 6 A scenario 600 is illustrated involving a vehicle 620 using an external audio receiver, according to one or more example embodiments. Scenario 600 depicts a crossroads where the vehicle 620 is currently navigating on road 602, which extends vertically from the bottom to the top of the figure, after complying with traffic rules (including stopping at stop sign 604). A vehicle 606 and an emergency vehicle 608 equipped with a siren 610 are also depicted, both operating on a segment of the crossroads extending from the left to the right of the figure, in a direction generally perpendicular to the vehicle 620.

[0097] Vehicle 620 may include vehicle sensor module 622, which may be equipped with an external audio receiver and other sensors. For example, vehicle sensor module 622 may be similar to Figure 4 The sensor module 400 shown or Figure 5 The sensor module 500 is shown. An external audio receiver set within the vehicle sensor module 612 on top of the vehicle 620 is configured to collect sound data generated by the environment at the intersection depicted in scene 600. This sound data is then processed and analyzed to inform the vehicle 620 of its surroundings and to help guide the vehicle 620 to navigate its environment safely.

[0098] As shown in scenario 600, when a control strategy is determined and executed, vehicle 620 may use audio information received from one or more audio receivers located on sensor module 622. In some cases, the one or more audio receivers may capture audio information of a specific sound, such as the siren 610 on emergency vehicle 608. A computing device may process the audio information received from the one or more audio receivers to locate emergency vehicle 608 relative to vehicle 620 based on the sound level measured for siren 610.

[0099] In some examples, when an emergency vehicle 608 approaches vehicle 620 from the right side, external audio receivers recessed in the front and right-hand portions of the vehicle sensor module 622 can initially capture the sound emitted by the siren 610. This initial detection can cause vehicle 620 to remain in a stopped position at stop sign 604 until the siren ceases to sound, or to pull over to the side of the road until the siren is no longer detected. Vehicle 620 can perform other actions based on the detection of siren 610. After stopping or pulling over, vehicle 620 can remain stopped for a period of time until the external audio receivers in vehicle sensor module 612 indicate that it is safe to proceed. This period of time can instruct each external audio sensor in vehicle sensor module 612 to cooperate with each other to locate the siren 610 on emergency vehicle 608 as it approaches vehicle 620 from the right side, until emergency vehicle 608 has passed a safe distance to the left side of vehicle 620.

[0100] In some examples, when the siren 610 on emergency vehicle 608 is detected as vehicle 608 operates from the right side of vehicle 620 to the left side of vehicle 620, the audio data picked up by the external audio receiver in vehicle sensor module 612 can change dynamically accordingly. Specifically, because emergency vehicle 608 is approaching from the right side in front of vehicle 620, the right-side external audio receiver may initially have the strongest siren detection. As emergency vehicle 608 begins to pass vehicle 620, the signal strength detected by the right-side external audio receiver may begin to attenuate. However, at this point, the front external audio receiver may have the strongest siren detection, and the left-side external audio receiver may begin to pick up sound.

[0101] As emergency vehicle 608 passes in front of vehicle 620, the signal strength detected by the front external audio receiver will begin to attenuate, and the left-hand external audio receiver may have the strongest alarm detection. After emergency vehicle 608 has completely passed vehicle 620, the signal strength obtained by the left-hand external audio receiver may also attenuate, indicating that it is safe for vehicle 620 to resume operation and navigation in the forward direction.

[0102] While the diagram illustrates an emergency vehicle crossing an intersection from right to left, the external audio receiver can detect other scenarios. For example, scenarios could include an emergency vehicle approaching from behind and passing in front of vehicle 620, vehicle 620 merging onto a highway, or multiple emergency vehicles operating within the environment of vehicle 620. As shown, vehicle 620 can perform autonomous or semi-autonomous operation based on audio information representing its surroundings. In other examples, a driver could control vehicle 620 and perform actions using audio information provided by a receiver on sensor module 622.

[0103] Figure 7A A vehicle sensor module 700 with an external audio receiver is shown according to one or more example embodiments. The vehicle sensor module 700 is as follows... Figure 7A The configuration shown can be coupled to the truck via mounting rod 702. Mounting rod 702 is configured to attach to the top front of the truck cab, but other configurations and placements are also possible. Mounting rod 702 is depicted as an elongated rod to aid in the truck's aerodynamic operation during forward navigation.

[0104] An external audio receiver cover 704 is coupled to the mounting rod 702. In this embodiment, the external audio receiver cover 704 is shaped like an end portion of a surfboard. The external audio receiver cover 704 is designed to contain one or more external audio receivers. Each external audio receiver may include one or more microphones for collecting noise data collected around the truck in operation. The external audio receiver cover 704 is designed to reduce wind exposure of the external audio receivers within it without suppressing all sound reaching the external audio receivers. Other configurations of the external audio receiver cover are also possible. A sensor 706 is also coupled to the mounting rod 702. The sensor 706 may include one or more cameras, radar, lidar, rangefinders, wireless devices (e.g., Bluetooth and / or 802.11), acoustic sensors, and any combination of other possible types of sensors.

[0105] Figure 7BA truck cab 710 with a mounted vehicle sensor module 712, equipped with an external audio receiver, is shown according to one or more example embodiments. The vehicle sensor module 712 is mounted to the roof of the truck cab 710 via an extending mounting rod 714. At least one external audio receiver cover (not shown) is mounted on the mounting rod 714 and configured to include at least one external audio receiver. The external audio receiver is configured to acquire noise data of the environment surrounding the truck cab 710 during operation. A sensor 716 is also mounted on the mounting rod. The sensor 716 may include one or more cameras, radar, lidar, rangefinders, radios (e.g., Bluetooth and / or 802.11), acoustic sensors, and any combination of other possible types of sensors. Optionally, additional external audio receivers 718 and 719 may be embedded in the upper part of both sides of the truck cab to detect noise originating from the respective sides of the truck. For example, an additional external audio receiver 718 can be placed on the upper right side of the truck, while two additional external audio receivers 719 can be placed on the upper left side of the truck. These additional audio receivers 718 and 719 can be strategically placed in the truck cab to simultaneously maximize sound detection while minimizing interfering sounds, such as wind noise from truck components (e.g., tires) and unwanted vibrations.

[0106] Figure 7B The results of wind simulation tests conducted in the cab of truck 710 are also depicted. These results of the wind test simulations identify the areas in the cab of truck 710 where wind stagnation is greatest, thereby determining the optimal location for placing external audio receivers to minimize the negative impact of wind noise in the collected sound data, as shown in area 721, which is only shown as an example.

[0107] Figure 7C An external audio receiver cover 720 according to one or more exemplary embodiments is shown. In the exemplary embodiment, the external audio receiver cover 720 is shown having four holes 722A, 722B, 722C, and 722D. One or more holes may contain one or more audio receivers for measuring sound from the surrounding environment. Although the cover 720 is shown with four holes 722A-722D, other examples may include a different number of holes.

[0108] The cover 720 can be configured to reduce direct wind contact with the external audio receiver contained therein. In some cases, the cover 720 can also prevent water and other debris from reaching the audio receiver and / or other components protected by the cover 720. Therefore, the cover 720 can be made of various materials that provide protection for internal components. In some examples, the cover 720 can prevent water ingress, thereby improving the performance of the audio receiver during vehicle operation. In some cases, sound can be detected by the internal audio receiver via four holes 722A, 722B, 722C, and 722D.

[0109] Figure 8 This is a flowchart of a method for operating a vehicle based on audio data, according to an example implementation. Method 800 represents an example method that may include one or more operations, functions, or actions depicted as in one or more of blocks 802, 804, and 806, each of which may be performed by... Figures 1 to 7C This can be performed by any system, device, and / or vehicle, as well as other possible systems. For example, Figure 4 The system 400 described herein can implement the execution of method 800.

[0110] Those skilled in the art will understand that the flowcharts described herein illustrate the functionality and operation of certain embodiments of this disclosure. In this regard, each block of the flowchart may represent a module, a fragment or portion of program code, which includes one or more instructions executable by one or more processors for implementing a specific logical function or step in the process. The program code may be stored on any type of computer-readable medium, such as storage devices including disks or hard disk drives.

[0111] Furthermore, each block may represent a circuit that is wired to perform a specific logical function in the process. Alternative implementations are included within the scope of the exemplary implementations of this application, wherein, as those skilled in the art will understand, functions may not be performed in the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved.

[0112] At block 802, method 800 relates to receiving audio data from a set of microphones located on a sensor module coupled to the roof of a vehicle. The audio data may represent one or more sounds originating from the vehicle's environment. The microphone set may include a first microphone positioned adjacent to the front of the sensor module. In some embodiments, the first microphone may extend into a given portion of the bottom surface of a gap adjacent to the sensor module, formed by coupling the sensor module to the roof of the vehicle. The microphones may also include a second microphone extending into a first side of the sensor module such that the second microphone is configured to detect audio originating from an environment extending from the first side of the vehicle, and a third microphone extending into a second side of the sensor module such that the third microphone is configured to detect audio originating from an environment extending from the second side of the vehicle. The second side may be opposite to the first side. Furthermore, the sensor module may include other microphones, such as a fourth microphone oriented to detect audio from the rear of the vehicle.

[0113] In box 804, method 800 relates to determining the direction of a specific sound relative to a vehicle based on audio data. For example, a computing device can determine the direction of an emergency vehicle relative to other vehicles.

[0114] In some examples, the computing device can perform a comparison between audio data received from each microphone and determine the direction of a particular sound relative to the vehicle based on that comparison.

[0115] In box 806, method 800 relates to controlling a vehicle based on determining the direction of a specific sound relative to the vehicle. For example, a computing device could cause the vehicle to pull over to the side of the road to provide a path for emergency vehicles to pass through.

[0116] Figure 9 This is a schematic diagram of a computer program according to an example embodiment. In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a non-transitory computer-readable storage medium, or as computer program instructions encoded on other non-transitory media or articles of art.

[0117] exist Figure 9 In the illustrated embodiment, a signal bearer medium 902 is used to provide a computer program product 900. The signal bearer medium 902 may include one or more programming instructions 904, which, when executed by one or more processors, can provide the above-mentioned reference. Figures 1 to 8 The described function or part of the function.

[0118] Signal-bearing medium 902 may encompass non-transitory computer-readable media 906, such as, but not limited to, hard disk drives, optical discs (CDs), digital video discs (DVDs), digital magnetic tapes, memory, and components of remote storage (e.g., in the cloud). In some embodiments, signal-bearing medium 902 may encompass computer-recordable media 908, such as, but not limited to, memory, read / write (R / W) CDs, R / W DVDs, etc.

[0119] In some implementations, signal-bearing medium 902 may encompass communication medium 910, such as, but not limited to, digital and / or analog communication media (e.g., optical fiber, waveguide, wired communication link, wireless communication link, etc.). Similarly, signal-bearing medium 902 may correspond to a remote storage device (e.g., the cloud). The computing system can share information with the cloud, including sending or receiving information. For example, the computing system can receive additional information from the cloud to enhance information obtained from a sensor or another entity. Therefore, for example, signal-bearing medium 902 may be transmitted by communication medium 910 in the form of a wireless medium.

[0120] One or more programming instructions 904 can be, for example, computer-executable and / or logically implemented instructions. In some examples, such as Figure 1 The computer system 112 shown or Figure 3 The computing device 300 shown can be configured to provide various operations, functions, or actions in response to one or more programming instructions 904 conveyed to the computer system by a computer-readable medium 906, a computer-recordable medium 908, and / or a communication medium 910. The non-transitory computer-readable media can also be distributed across multiple data storage elements and / or the cloud (e.g., remotely), which may be geographically distant from each other. The computing device executing some or all of the storage instructions can be a vehicle. Alternatively, the computing device executing some or all of the storage instructions can be another computing device, such as a server.

[0121] The above detailed description, with reference to the accompanying drawings, illustrates various features and functions of the disclosed systems, apparatus, and methods. While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent. The various aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting; the true scope is indicated by the appended claims.

Claims

1. A system for detecting audio, comprising: vehicle; A sensor module, coupled to the roof of a vehicle, such that a gap is formed between a portion of the bottom surface of the sensor module and the roof, wherein the sensor module includes one or more sensors; and A microphone set coupled to a sensor module, wherein the microphone set includes: (i) A first microphone, positioned adjacent to the front of the sensor module, wherein the first microphone extends into a given portion of the bottom surface of the sensor module adjacent to the gap; (ii) A second microphone, extending into the first side of the sensor module, such that the second microphone is configured to detect audio originating from the environment extending from the first side of the vehicle, and (iii) A third microphone, extending into the second side of the sensor module, such that the third microphone is configured to detect audio originating from the environment extending from the second side of the vehicle, wherein the second side is opposite to the first side. The sensor module further includes a first coupling component and a second coupling component. The second coupling component is arranged in an orientation aligned with the direction extending from the first side of the vehicle to the second side of the vehicle. The first coupling component and the second coupling component are configured to couple the sensor module to the roof of the vehicle. The second coupling component further includes an extension that is adjacent to the rear of the sensor module and extends toward the rear of the vehicle. The microphone set further includes a fourth microphone extending into the end side of the extension, such that the fourth microphone is configured to detect audio originating from the environment extending from the rear of the vehicle.

2. The system according to claim 1, wherein, The first coupling component is arranged in an orientation aligned with the direction extending from the first side of the vehicle to the second side of the vehicle.

3. The system according to claim 2, wherein, The first microphone is positioned adjacent to the first coupling component, and The second and third microphones are located adjacent to the second coupling component.

4. The system according to claim 1, wherein, The vehicle in question is a semi-truck. The sensor module has a slender structure, and Each microphone includes a curved cap.

5. The system according to claim 1, wherein, The one or more sensors include: A lidar unit coupled to the top surface of the sensor module.

6. The system according to claim 1, wherein, The one or more sensors include: At least one camera is coupled to the front of an adjacent sensor module.

7. The system according to claim 1, further comprising: One or more fans are located inside the sensor module, wherein each fan is configured to reduce the operating temperature of one or more sensors coupled to the sensor module. Each microphone is coupled to the sensor module at least a threshold distance from one or more fans.

8. The system according to claim 1, wherein, The second and third microphones are coupled to the sensor module at a downward angle relative to a horizontal plane extending parallel to the road surface.

9. The system according to claim 1, wherein, Each microphone includes a pair of audio receivers.

10. A sensor module coupled to the roof of a vehicle, such that a gap is formed between a portion of the bottom surface of the sensor module and the roof, the sensor module comprising: One or more sensors; as well as The microphone set includes: (i) A first microphone, positioned adjacent to the front of the sensor module, wherein the first microphone extends into a given portion of the bottom surface of the sensor module adjacent to the gap; (ii) A second microphone, extending into the first side of the sensor module, such that the second microphone is configured to detect audio originating from the environment extending from the first side of the vehicle, and (iii) A third microphone, extending into the second side of the sensor module, such that the third microphone is configured to detect audio originating from the environment extending from the second side of the vehicle, wherein the second side is opposite to the first side; A first coupling assembly and a second coupling assembly are configured to couple the sensor module to the roof of the vehicle, wherein the second coupling assembly is arranged in an orientation aligned with a direction extending from a first side of the vehicle to a second side of the vehicle. The second coupling component further includes an extension that extends adjacent to the rear of the sensor module, and The microphone assembly further includes a fourth microphone extending into the end side of the extension, such that the fourth microphone is configured to detect audio originating from the environment extending from the rear of the sensor module.

11. The sensor module according to claim 10, wherein, The first coupling component is arranged in an orientation aligned with the direction extending from the first side of the vehicle to the second side of the vehicle.

12. The sensor module according to claim 11, wherein, The first microphone is positioned adjacent to the first coupling component, and The second and third microphones are positioned on the first and second sides of the sensor module, respectively, adjacent to the second coupling component.

13. The sensor module according to claim 10, wherein, The one or more sensors include at least a camera and a lidar.

14. The sensor module according to claim 10, further comprising: One or more fans are located inside the sensor module, wherein each fan is configured to reduce the operating temperature of one or more sensors coupled to the sensor module. Each microphone is coupled to the sensor module at least a threshold distance from one or more fans.

15. A method for detecting audio, comprising: Audio data is received at a computing device from a set of microphones on a sensor module, wherein the sensor module is coupled to the roof of the vehicle such that a gap is formed between a portion of the bottom surface of the sensor module and the roof, wherein the audio data represents one or more sounds originating from the vehicle's environment, and wherein the set of microphones includes: (i) A first microphone, positioned adjacent to the front of the sensor module, wherein the first microphone extends into a given portion of the bottom surface of the sensor module adjacent to the gap; (ii) A second microphone, extending into the first side of the sensor module, such that the second microphone is configured to detect audio originating from the environment extending from the first side of the vehicle, and (iii) A third microphone, extending into the second side of the sensor module, such that the third microphone is configured to detect audio originating from the environment extending from the second side of the vehicle, wherein the second side is opposite to the first side; and Based on audio data, determine the direction of a specific sound relative to the vehicle; and Vehicle control is based on determining the direction of a specific sound relative to the vehicle. The sensor module further includes a first coupling component and a second coupling component. The second coupling component is arranged in an orientation aligned with the direction extending from the first side of the vehicle to the second side of the vehicle. The first coupling component and the second coupling component are configured to couple the sensor module to the roof of the vehicle. The second coupling component further includes an extension that is adjacent to the rear of the sensor module and extends toward the rear of the vehicle. The microphone set further includes a fourth microphone extending into the end side of the extension, such that the fourth microphone is configured to detect audio originating from the environment extending from the rear of the vehicle.

16. The method according to claim 15, wherein, Determining the direction of a specific sound relative to a vehicle includes: Determine the direction of the emergency vehicle relative to the vehicle.

17. The method of claim 15, further comprising: Perform a comparison between the audio data received from each microphone; as well as Determining the direction of a specific sound relative to the vehicle includes: The direction of a particular sound relative to the vehicle is determined based on the comparison.

Images