Window vibration monitoring for voice command recognition

Abstract

A method and apparatus for monitoring vehicle window vibrations for voice command recognition are disclosed. An exemplary vehicle includes a vehicle window; an exterior panel; a vibration sensor coupled to the vehicle window to detect audio vibrations; an audio actuator coupled to the exterior panel to vibrate the exterior panel; and a controller. The controller is configured to detect a voice command from a user via the vibration sensor, identify an audio response based on the voice command, and transmit the audio response to the user via the audio actuator.

Description

Technical Field

[0001] The present disclosure relates generally to voice command recognition and, more particularly, to vehicle window vibration monitoring for voice command recognition. Background Art

[0002] Typically, a vehicle includes a plurality of features and / or functions that are controlled by an operator (e.g., a driver). Typically, a vehicle includes a plurality of input devices to support the operator in controlling the vehicle features and / or functions. For example, a vehicle may include buttons, control knobs, an instrument panel, a touch screen, and / or a touchpad to support the operator in controlling the vehicle features and / or functions. Additionally, in some cases, a vehicle includes a communication platform that is communicatively coupled to a mobile device located within the vehicle to support the operator and / or another occupant in interacting with the vehicle features and / or functions via the mobile device. Summary of the Invention

[0003] This disclosure summarizes various aspects of the embodiments. Other implementations are contemplated based on the techniques described herein, which will be apparent to one of ordinary skill in the art upon examination of the following drawings and detailed description, and are intended to fall within the scope of this application.

[0004] An exemplary embodiment of monitoring vehicle window vibrations for voice command recognition is disclosed. The exemplary disclosed vehicle includes a vehicle window; an exterior panel; a vibration sensor coupled to the vehicle window to detect audio vibrations; an audio actuator coupled to the exterior panel to vibrate the exterior panel; and a controller. The controller is configured to detect a voice command from a user via the vibration sensor, identify an audio response based on the voice command, and transmit the audio response to the user via the audio actuator.

[0005] An exemplary disclosed method includes detecting a user's voice command via a vibration sensor. The vibration sensor is coupled to a window of a vehicle to detect audio vibrations. The exemplary disclosed method also includes identifying, via a processor, an audio response based on the voice command, and transmitting the audio response to the user via an audio actuator. The audio actuator is coupled to an exterior layer of the vehicle to vibrate the exterior layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] For a better understanding of the present invention, reference may be made to the embodiments illustrated in the following drawings. The components in the drawings are not necessarily drawn to scale, and related elements may be omitted or, in some cases, may be exaggerated in proportion to emphasize and clearly illustrate the novel features described herein. Furthermore, as is known in the art, the system components may be arranged differently. Additionally, in the drawings, like reference numerals designate corresponding parts throughout the several views.

[0007] Figure 1 An exemplary vehicle according to the teachings herein is shown.

[0008] Figure 2 yes Figure 1 Block diagram of the vehicle's electronic components.

[0009] Figure 3 is a flow chart for monitoring vehicle window vibrations to recognize voice commands according to the teachings herein. DETAILED DESCRIPTION

[0010] While the invention may be embodied in various forms, certain exemplary and non-limiting embodiments will be shown in the drawings and described below, it being understood that this disclosure is to be considered as illustrative of the invention and is not intended to limit the invention to the particular embodiments shown.

[0011] Typically, a vehicle includes a plurality of features and / or functions that are controlled by an operator (e.g., a driver). Typically, a vehicle includes a plurality of input devices to support the operator in controlling the vehicle features and / or functions. For example, a vehicle may include buttons, control knobs, an instrument panel, a touch screen, and / or a touchpad to support the operator in controlling the vehicle features and / or functions. Additionally, in some cases, a vehicle includes a communication platform that is communicatively coupled to a mobile device located within the vehicle to support the operator and / or another occupant in interacting with the vehicle features and / or functions via the mobile device.

[0012] Recently, some vehicles include microphones that allow an operator located inside the vehicle's cabin to audibly interact with vehicle features and / or functions (e.g., via a digital personal assistant). However, when the operator is outside the vehicle's cabin, the microphone is located inside the cabin and may not be able to detect audio commands from the vehicle operator. Additionally, the microphone may not be located outside the vehicle's cabin due to potential damage from inclement weather conditions and / or contact with other objects.

[0013] Example methods and apparatus disclosed herein include a vehicle system that supports voice activation of vehicle features by a user located outside a vehicle cabin. The vehicle system includes a communication node that detects when a key fob of a user associated with the vehicle approaches the vehicle. The vehicle system includes an accelerometer positioned on a vehicle window, enabling the window to act as a microphone. The accelerometer is activated to receive a signal when the communication node detects the key fob's proximity. The accelerometer transmits the received signal to a controller for speech recognition analysis. The controller is configured to identify a wake-up indicator (e.g., "Hey SYNC") and subsequent commands within the signal. The vehicle system causes the vehicle to provide information (e.g., factory-recommended tire pressure, factory-recommended fuel type (e.g., premium fuel), factory-recommended engine oil, identification of a fuse box location) and / or perform a vehicle function (e.g., starting the vehicle, sending a text message, etc.) based on the recognized command. For example, the vehicle system includes actuators (e.g., actuators of a SoundBug device) located on a vehicle window and / or other exterior surface. The actuator vibrates the vehicle windows and / or other exterior layers to cause these surfaces to act as speakers providing requested information to a user located outside the vehicle.

[0014] Returning to the attached figure, Figure 1 An exemplary vehicle 100 according to the teachings herein is shown. Vehicle 100 can be a standard gasoline-powered vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle, and / or any other type of vehicle for mobility. Vehicle 100 includes components related to mobility, such as a powertrain having an engine, a transmission, a suspension, a drive shaft, and / or wheels. Vehicle 100 can be non-autonomous, semi-autonomous (e.g., some conventional mobility functions are controlled by vehicle 100), or autonomous (e.g., mobility functions are controlled by vehicle 100 without direct driver input).

[0015] In the illustrated example, vehicle 100 includes tires 102 and an engine 104. For example, each of tires 102 is coupled to the body of vehicle 100 at a different corresponding wheel well of vehicle 100 to support vehicle 100 in traveling along a road surface. Engine 104 of the illustrated example is an internal combustion engine, an electric motor, and / or any other power source that propels vehicle 100. In some examples, engine 104 is initially activated when receiving energy from a starter battery pack and is subsequently powered via energy received from an alternator.

[0016] The vehicle 100 of the illustrated example also includes a cabin 106, a front side 108, a rear side 110, a driver's side 112, and a passenger side 114. The front side 108 includes a front windshield 116 and a front panel 118 (also referred to as a hood), and the rear side 110 includes a rear windshield 120 and a rear panel 122 (also referred to as a tailgate panel and / or trunk panel). The vehicle 100 includes doors 124 that include side windows 126 and door panels 128. For example, the driver's side 112 of the vehicle 100 includes one of the doors 124 (e.g., the driver's side door), including one of the side windows 126 and one of the door panels 128. Additionally, the passenger side 114 includes one of the doors 124 (e.g., the passenger's side door), including another of the side windows 126 and another of the door panels 128. Figure 1 As shown in FIG, the front windshield 116, front panel 118, rear windshield 120, rear panel 122, side windows 126, and door panels 128 at least partially form the exterior of vehicle 100. That is, the exterior of vehicle 100 includes the front windshield 116, front panel 118, rear windshield 120, rear panel 122, side windows 126, and door panels 128. Furthermore, vehicle cabin 106 is at least partially defined by the front windshield 116, rear windshield 120, and doors 124. In some examples, the front windshield 116 is formed of laminated glass or safety glass, and the rear windshield 120 and side windows 126 are formed of non-laminated, tempered glass. Furthermore, the front panel 118, rear panel 122, and door panels 128 are formed of, for example, steel, aluminum, other metallic materials, plastic, carbon fiber, fiberglass, other composite materials, and / or any combination thereof.

[0017] like Figure 1 As shown in FIG, vehicle 100 includes one or more communication nodes 130. In the illustrated example, communication node 130 is configured to communicatively connect to a key fob 132 of a user 134 of vehicle 100. As used herein, a "key fob" refers to an electronic device that wirelessly communicates with vehicle 100 (e.g., via one or more of communication nodes 130) to unlock and / or lock one or more of vehicle doors 124, open and / or close one or more of vehicle doors 124, and / or activate the engine 104 of vehicle 100. In the illustrated example, user 134 carries key fob 132 for wireless communication with communication node 130. In other examples, user 134 carries a mobile device that acts as a phone-as-key for wireless communication with communication node 130. As used herein, "phone-as-key" refers to a mobile device (e.g., a smartphone, wearable device, smartwatch, tablet, etc.) that includes hardware and / or software to act as a key fob.

[0018] The communication node 130 includes hardware and firmware to establish a wireless connection with the key card 132. For example, the communication node 130 is a short-range wireless module that wirelessly communicates with the key card and / or phone key of the user of the vehicle 100 (e.g., the key card 132 of the user 134) via a short-range wireless communication protocol. In some examples, the communication node 130 implements and / or Bluetooth Low Energy (BLE) protocol. Special Interest Groups maintain Volume 6 of the 4.0 specification (and subsequent versions) describes In other examples, the communication node 130 may utilize WiFi, WiMax, NFC, UWB (Ultra Wideband), and / or any other communication protocol that enables the communication node 130 to be communicatively coupled to the key fob 132 .

[0019] Before communicating with the key card 132, one or more of the communication nodes 130 can be utilized to authenticate the key card 132 in order to communicate with the communication node 130. When authenticated, the key card 132 is paired with the vehicle 100. For example, to authenticate the key card 132, one or more of the communication nodes 130 intermittently broadcasts a beacon (e.g., a low-power beacon such as a

[0014] ). The key fob 132 may be a Bluetooth Low Energy (BLE) beacon. When the key fob 132 is within the broadcast range of one or more of the communication nodes 130, the key fob 132 receives the beacon and subsequently transmits the key. One or more of the communication nodes 130 authenticate the key fob 132 to communicate with the communication node 130 upon receiving the key from the key fob 132. In other examples, the key fob 132 broadcasts the beacon, and one or more of the communication nodes 130 subsequently receives the beacon to authenticate communication between the key fob 132 and the communication node 130.

[0020] Additionally, the vehicle 100 of the illustrated example includes a communication module 136 that includes a wired or wireless network interface to support communication with an external network (e.g., Figure 2The communication module 136 also includes hardware (e.g., processor, memory, storage device, antenna, etc.) and software to control the wired or wireless network interface. In the example shown, the communication module 136 includes one or more communication controllers for standard-based networks (e.g., Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), Code Division Multiple Access (CDMA), WiMAX (IEEE802.16m); Near Field Communication (NFC); Local Area Wireless Networks (including IEEE 802.11a / b / g / n / ac or others), Dedicated Short Range Communication (DSRC) and Wireless Gigabit (IEEE 802.11ad), etc.). In some examples, the communication module 136 includes a wired or wireless interface (e.g., auxiliary port, Universal Serial Bus (USB) port, The vehicle 100 may be coupled to a mobile device (e.g., a smart phone, a wearable device, a smartwatch, a tablet, etc.) to communicate with the mobile device. In the example, the vehicle 100 may communicate with the external network via the coupled mobile device. The external network may be a public network such as the Internet; a private network such as an intranet; or a combination thereof, and may utilize a variety of networking protocols currently available or later developed, including but not limited to TCP / IP-based networking protocols.

[0021] The vehicle 100 also includes vibration sensors 138 coupled to the exterior of the vehicle 100 to detect audio vibrations. For example, each of the vibration sensors 138 is securely mounted to an interior surface and / or an exterior surface of the exterior of the vehicle 100. The vibration sensors 138 can be coupled to the interior surface of the exterior of the vehicle 100 to prevent contact between the vibration sensors 138 and other objects. In other examples, one or more of the vibration sensors 138 are embedded within the exterior of the vehicle 100.

[0022] Additionally, in the example shown, each side of the vehicle 100 includes one vibration sensor 138. That is, one vibration sensor 138 is coupled to each of the front side 108, rear side 110, driver side 112, and passenger side 114 of the vehicle 100. In other examples, more or fewer vibration sensors 138 may be coupled to one or more of the front side 108, rear side 110, driver side 112, and / or passenger side 114 of the vehicle 100. Additionally, in the example shown, one vibration sensor 138 is coupled to each of the front windshield 116, rear windshield 120, and side windows 126. In other examples, more or fewer vibration sensors 138 may be coupled to one or more of: the front windshield 116; the rear windshield 120; and / or the side windows 126. Additionally, in some examples, one or more of the vibration sensors 138 are coupled to one or more of: the front panel 118 ; the rear panel 122 ; and / or the door panel 128 .

[0023] The vibration sensor 138 of the illustrated example is configured to measure audio vibrations of various portions of the exterior of the vehicle 100 to which the vibration sensor 138 is coupled. The audio vibrations are caused by sound waves impinging on the exterior. For example, sound waves traveling through a medium (e.g., one or more of the front windshield 116, the front panel 118, the rear windshield 120, the rear panel 122, the side windows 126, or one or more of the door panels 128) cause the medium to oscillate and / or vibrate. When the sound waves impinge on corresponding portions of the exterior of the vehicle 100, the vibration sensor 138 measures these audio vibrations caused by the sound waves and generates an electrical signal corresponding to the measured audio vibrations. The electrical signal can then be processed and / or analyzed to determine the sound waves impinging on the exterior of the vehicle 100. For example, the vibration sensor 138 can be an accelerometer (e.g., a single-axis accelerometer, a three-axis accelerometer, a micromachined accelerometer, a piezoelectric accelerometer, etc.), a piezoelectric contact film, a microphone, and / or any other device capable of detecting sound by monitoring the audio vibrations of the exterior. In the example shown, the vibration sensor 138 is configured to measure audio vibrations perpendicular to the surface to which the vibration sensor 138 is coupled. Additionally, the vibration sensor 138 is configured to measure a wide frequency range of sounds, for example to support detection of voice commands provided by the user 134.

[0024] In other words, the outer layers of the vehicle 100 and the vibration sensors 138 form microphones. For example, the portions of the outer layers coupled to the vibration sensors 138 act as the diaphragms of the microphones. In the illustrated example, the front windshield 116 and the vibration sensor 138 coupled to the front windshield 116 form one microphone, the rear windshield 120 and the vibration sensor 138 coupled to the rear windshield 120 form another microphone, and the side windows 126 and the vibration sensors 138 coupled to the side windows 126 form other microphones. In other examples, the front panel 118 forms a microphone when one of the vibration sensors 138 is coupled to the front panel 118, the rear panel 122 forms a microphone when one of the vibration sensors 138 is coupled to the rear panel 122, and / or one or more of the door panels 128 and corresponding one or more of the vibration sensors 138 form microphones.

[0025] like Figure 1 As shown in FIG, the vehicle 100 also includes an audio actuator 140 (also referred to as a vibration speaker audio actuator) coupled to the outer layer of the vehicle 100 to vibrate the outer layer. For example, each of the audio actuators 140 (e.g., the audio actuators 140 of the SoundBug device) is securely mounted to an interior surface and / or an exterior surface of the outer layer of the vehicle 100. The audio actuators 140 can be coupled to the interior surface of the outer layer of the vehicle 100 to prevent the audio actuators 140 from coming into contact with other objects. In other examples, one or more of the audio actuators 140 are embedded within the outer layer of the vehicle 100.

[0026] Additionally, in the illustrated example, each side of the vehicle 100 includes one audio actuator 140. That is, one of the audio actuators 140 is coupled to each of the front side 108, rear side 110, driver side 112, and passenger side 114 of the vehicle 100. In other examples, more or fewer audio actuators 140 may be coupled to one or more of the front side 108, rear side 110, driver side 112, and / or passenger side 114 of the vehicle 100. Additionally, in the illustrated example, one of the audio actuators 140 is coupled to each of the front windshield 116, rear windshield 120, and side windows 126. In other examples, more or fewer audio actuators 140 may be coupled to one or more of: the front windshield 116; the rear windshield 120; and / or the side windows 126. Additionally, in some examples, one or more of the audio actuators 140 are coupled to one or more of: the front panel 118 ; the rear panel 122 ; and / or the door panels 128 .

[0027] The audio actuators 140 of the illustrated example are configured to emit sound by vibrating a corresponding portion of the outer layer of the vehicle 100 to which they are coupled. For example, each of the audio actuators 140 includes an arm that actuates upon receiving an electrical signal. For each of the audio actuators 140, when the arm actuates to cause the outer layer to vibrate, the arm contacts or causes another actuator component to contact an adjacent surface of the outer layer. In other words, each of the audio actuators 140 functions as a tuning fork by actuating an adjacent portion of the outer layer, causing that portion of the outer layer to form a sound baffle for the speaker. In other words, the outer layer of the vehicle 100 and the audio actuators 140 form a speaker. In the example shown, the front windshield 116 and one of the audio actuators 140 coupled to the front windshield 116 form one speaker, the rear windshield 120 and one of the audio actuators 140 coupled to the rear windshield 120 form another speaker, and the side windows 126 and those of the audio actuators 140 coupled to the side windows 126 form other speakers. In other examples, the front panel 118 forms a speaker when one of the audio actuators 140 is coupled to the front panel 118, the rear panel 122 forms a speaker when one of the audio actuators 140 is coupled to the rear panel 122, and one or more of the door panels 128 and corresponding one or more of the audio actuators 140 form speakers.

[0028] The vehicle 100 of the illustrated example also includes a voice command controller 142. For example, the voice command controller 142 is communicatively coupled (e.g., wired and / or wirelessly) to the communication node 130, the communication module 136, the vibration sensor 138, the audio actuator 140, and / or other electrical components of the vehicle 100. The voice command controller 142 is configured to detect a voice command from the user 134 via one or more of the vibration sensors 138 and transmit an audio response to the user 134 and / or perform a vehicle function for the user 134 based on the voice command.

[0029] In operation, the voice command controller 142 utilizes signals retrieved from the vibration sensors 138 to detect a voice command that has been provided by the user 134. For example, the voice command controller 142 and the vibration sensors 138 are configured to detect a voice command provided by the user 134 when the user 134 is outside of, but in the vicinity of, the cabin 106 of the vehicle 100. For example, to identify a voice command, the voice command controller 142 initially receives an audio signal detected via one or more of the vibration sensors 138. The voice command controller 142 then utilizes speech recognition (e.g., via speech recognition software) to identify a word or phrase within the audio signal and associates the word or phrase with a set of commands or requests corresponding to the vehicle 100 (e.g., stored in the system). Figure 2If the identified word or phrase corresponds to one of the predefined commands or requests, the voice command controller 142 detects the voice command within the audio signal detected via one or more of the vibration sensors 138.

[0030] In some instances, vibration sensor 138 is triggered to detect a voice command from user 134 . For example, voice command controller 142 triggers vibration sensor 138 to detect a voice command in response to voice command controller 142 identifying, via one or more of vibration sensors 138 , that user 134 has provided a wake-up term preceding a voice command. That is, user 134 will provide a wake-up term prior to providing the voice command to trigger detection of the voice command. The wake-up term can be any word or phrase preselected by the manufacturer or driver, such as an uncommon word (e.g., “SYNC”), an uncommon name (e.g., “Clara”), and / or an uncommon phrase (e.g., “Hey SYNC,” “Hey Clara”). For example, to identify the wake-up term, voice command controller 142 initially receives an audio signal detected via one or more of vibration sensors 138 . Voice command controller 142 then utilizes speech recognition (e.g., via speech recognition software) to identify the word or phrase within the audio signal and compares the word or phrase to predefined wake-up terms for vehicle 100 (e.g., stored in database 214 ). Upon identifying that the audio signal includes a predefined wake-up term, the voice command controller 142 triggers the vibration sensor 138 to detect the voice command following the wake-up term.

[0031] Additionally, in some examples, the voice command controller 142 activates the vibration sensor 138 to detect a wake-up term in response to detecting that the key fob 132 and / or the user's 134 phone, i.e., key fob, is within the communication range of the vehicle 100. For example, the voice command controller 142 detects, via one or more of the communication nodes 130 of the vehicle 100, whether the key fob 132 to which it has been paired is within the communication range of the vehicle 100. In the illustrated example, the voice command controller 142 is configured to detect the distance between the key fob 132 and the vehicle 100 based on a received signal strength indicator (RSSI) of a signal between the key fob 132 and one or more of the communication nodes 130. When the key fob 132 is detected to be within the communication range of the vehicle 100, the voice command controller 142 activates the vibration sensor 138 to detect the wake-up term.

[0032] Additionally or alternatively, the voice command controller 142 determines whether the detected wake-up term and / or voice command was provided by an authorized source. For example, the voice command controller 142 confirms that the wake-up term and / or voice command was provided by the user 134 carrying the key fob 132 paired with the vehicle 100.

[0033] In some examples, voice command controller 142 utilizes voice recognition (e.g., via voice recognition software). That is, voice command controller 142 compares the pronunciation characteristics and / or biometric characteristics of the detected audio signal including the wake-up term and / or voice command with known pronunciation characteristics and / or biometric characteristics of user 134 (e.g., stored in database 214) to determine whether user 134, who is authorized to utilize vehicle 100, has provided the wake-up term and / or voice command.

[0034] Additionally, in some examples, the voice command controller 142 determines whether the detected wake-up term and / or voice command was provided by an authorized source by comparing the location of the key fob 132 with the location of the source of the detected wake-up term and / or voice command. The voice command controller 142 identifies the location of the key fob 132 via the communication nodes 130 based on the RSSI of signals between the key fob 132 and the plurality of communication nodes 130. For example, the voice command controller 142 utilizes trilateration to determine the location of the key fob 132 based on the RSSI of the signals. The voice command controller 142 identifies the direction from which the wake-up term and / or voice command was provided by comparing the signal strength of the vibration sensors 138. For example, if the signal strength of the wake-up term and / or voice command detected by one of the vibration sensors 138 on the driver's side 112 of the vehicle 100 is greater than the signal strength of one of the vibration sensors 138 on the passenger's side 114, the voice command controller 142 detects that the wake-up term and / or voice command was provided from the driver's side 112 of the vehicle 100. Similarly, if the signal strength of the wake-up term and / or voice command detected by one of the vibration sensors 138 on the front side 108 of the vehicle 100 is greater than the signal strength of one of the vibration sensors 138 on the rear side 110, then the voice command controller 142 detects that the wake-up term and / or voice command is being provided from the front side 108 of the vehicle 100. In response to determining that the position of the key fob 132 corresponds to the direction from which the wake-up term and / or voice command was provided, the voice command controller 142 determines that the detected wake-up term and / or voice command is being provided by an authorized source.

[0035] Additionally, in some examples, the voice command controller 142 determines which of the vibration sensors 138 to activate to detect the wake-up term and / or voice command based on the identified location of the key fob 132. For example, the voice command controller 142 activates one or more of the vibration sensors 138 located toward the front side 108 when it detects that the key fob 132 is located in front of the vehicle 100.

[0036] Upon detecting a voice command provided by user 134 , voice command controller 142 determines whether the voice command includes a request for information and / or an instruction to perform a vehicle function.

[0037] In response to voice command controller 142 determining that the voice command is a request for information, voice command controller 142 identifies an audio response based on the request for information of the voice command. In some instances, the request for information includes a request for information included in an owner's manual for vehicle 100, such as factory-recommended tire pressures for tires 102, and the audio response includes the information in the owner's manual, such as the factory-recommended tire pressures. Additionally, in some instances, the request for information includes a request for vehicle sensor readings, such as measured tire pressures for one or more of tires 102, and the audio response includes the vehicle sensor readings, such as the measured tire pressures. In other instances, the request for information includes a request for information from an external network (e.g., Figure 2 The audio response includes information such as the forecasted weather conditions for the day that is retrieved from the external network 216, and the audio response includes information such as the forecasted weather conditions that is retrieved from the external network.

[0038] When an audio response is identified, the voice command controller 142 transmits the audio response to the user 134 located outside the cabin 106 of the vehicle 100 via one or more of the audio actuators 140 and / or any other vehicle speakers configured to transmit sound to an area located outside the cabin 106 of the vehicle 100. In some examples, the voice command controller 142 determines which of the audio actuators 140 to utilize to transmit the audio response based on the identified location of the key fob 132. For example, when the key fob 132, and therefore the user 134, is detected to be located in front of the vehicle 100, the voice command controller 142 transmits the audio response using one or more of the audio actuators 140 positioned toward the front side 108 of the vehicle.

[0039] In response to the voice command controller 142 determining that the voice command is an instruction to perform a vehicle function, the voice command controller 142 identifies the vehicle function to be performed and sends a signal to perform the identified vehicle function. In some examples, the vehicle function includes starting the engine 104, locking or unlocking one or more of the doors 124, opening or closing one or more of the side windows 126, sending a text message via the communication module 136, adding an item to a to-do or grocery list (e.g., stored in the database 214), sending a signal via the communication module 136 requesting off-site service to be performed, etc. In addition, in some examples, the vehicle function includes initiating and facilitating a phone call, wherein one or more of the vibration sensors 138 receives audio of the phone call from the user 134 and one or more of the audio actuators 140 provides the audio of the phone call to the user 134.

[0040] Figure 2 is a block diagram of electronic components 200 of vehicle 100. In the example shown, electronic components 200 include an onboard computing platform 202, communication node 130, communication module 136, audio actuator 140, sensor 204, electronic control unit (ECU) 206, and vehicle data bus 208.

[0041] The onboard computing platform 202 includes a microcontroller unit, controller, or processor 210; a memory 212; and a database 214. In some examples, the processor 210 of the onboard computing platform 202 is configured to include the voice command controller 142. For example, the voice command controller 142 retrieves information stored in the database 214 to identify an audio response to be transmitted to the user 134 via one or more of the audio actuators 140. Alternatively, in some examples, the voice command controller 142 is incorporated into another electronic control unit (ECU) having its own processor 210, memory 212, and database 214.

[0042] Processor 210 may be any suitable processing device or collection of processing devices, such as, but not limited to, a microprocessor, a microcontroller-based platform, an integrated circuit, one or more field programmable gate arrays (FPGAs), and / or one or more application-specific integrated circuits (ASICs). Memory 212 may be volatile memory (e.g., RAM including non-volatile RAM, magnetic RAM, ferroelectric RAM, etc.), non-volatile memory (e.g., disk storage, flash memory, EPROM, EEPROM, memristor-based non-volatile solid-state memory, etc.), immutable memory (e.g., EPROM), read-only memory, and / or high-capacity storage device (e.g., hard disk drive, solid-state drive, etc.). In some examples, memory 212 includes multiple types of memory, particularly volatile memory and non-volatile memory.

[0043] The memory 212 is a computer-readable medium on which one or more sets of instructions, such as software for operating the methods of the present disclosure, may be embedded. The instructions may embody one or more of the methods or logic described herein. For example, the instructions may reside entirely or at least partially within any one or more of the memory 212, the computer-readable medium, and / or within the processor 210 during execution of the instructions.

[0044] The terms "non-transitory computer-readable medium" and "computer-readable medium" include a single medium or multiple media, such as a centralized or distributed database and / or associated cache memory and servers that store one or more sets of instructions. Additionally, the terms "non-transitory computer-readable medium" and "computer-readable medium" include any tangible medium that can store, encode, or carry a set of instructions for a processor to execute or cause a system to perform any one or more of the methods or operations disclosed herein. As used herein, the term "computer-readable medium" is expressly defined to include any type of computer-readable storage device and / or storage disk, and excludes propagating signals.

[0045] Additionally, in the example shown, the communication module 136 of the vehicle 100 is in wireless communication with the network 216. For example, the communication module 136 is communicatively coupled to the network 216 to enable the voice command controller 142 to retrieve information from the network 216. The voice command controller 142 utilizes the information retrieved from the network 216 to identify an audio response to be transmitted to the user 134 via one or more of the audio actuators 140.

[0046] Sensors 204 are arranged in and around vehicle 100 to monitor properties of vehicle 100 and / or the environment in which vehicle 100 is located. One or more of sensors 204 can be installed to measure properties of the external surroundings of vehicle 100. Additionally or alternatively, one or more of sensors 204 can be installed within the cabin of vehicle 100, or within the body of vehicle 100 (e.g., engine compartment, wheel wells, etc.) to measure properties of the interior of vehicle 100. For example, sensors 204 include accelerometers, odometers, tachometers, pitch and yaw sensors, wheel speed sensors, microphones, tire pressure sensors, biometric sensors, and / or any other suitable type of sensor. In the illustrated example, sensors 204 include vibration sensors 138 that detect voice commands provided by user 134.

[0047] The ECU 206 monitors and controls the subsystems of the vehicle 100. For example, the ECU 206 is a discrete collection of electronic devices, including its own circuitry (e.g., integrated circuits, microprocessors, memory, storage devices, etc.) and firmware, sensors, actuators, and / or mounting hardware. The ECU 206 communicates and exchanges information via a vehicle data bus (e.g., vehicle data bus 208). In addition, the ECUs 206 can communicate properties (e.g., ECU 206 status, sensor readings, control status, error and diagnostic codes, etc.) to each other and / or receive requests from each other. For example, the vehicle 100 may have 70 or more ECUs 206 located in various locations around the vehicle 100 and communicatively coupled by the vehicle data bus 208.

[0048] In the example shown, the ECU 206 includes an engine control unit 218 and a body control module 220. For example, the engine control unit 218 controls the operation of the engine 104 of the vehicle 100 (e.g., remote start). In addition, the body control module 220 controls one or more subsystems throughout the vehicle 100, such as power windows, power locks, an anti-theft system, power mirrors, etc. For example, the body control module 220 includes circuits for driving one or more of relays (e.g., to control wiper fluid, etc.), brushed direct current (DC) motors (e.g., to control power seats, power locks, power windows, wipers, etc.), stepper motors, LEDs, etc.

[0049] The vehicle data bus 208 communicatively couples the communication node 130, the communication module 136, the audio actuator 140, the onboard computing platform 202, the sensor 204, and the ECU 206. In some examples, the vehicle data bus 208 includes one or more data buses. The vehicle data bus 208 can be configured in accordance with the Controller Area Network (CAN) bus protocol defined by the International Organization for Standardization (ISO) 11898-1, the Media Oriented Systems Transport (MOST) bus protocol, the CAN Flexible Data (CAN-FD) bus protocol (ISO 11898-7), and / or the K-line bus protocol (ISO 9141 and ISO 14230-1), and / or Ethernet. TM It is implemented by bus protocols such as IEEE802.3 (since 2002).

[0050] Figure 3 is a flow chart of an exemplary method 300 for monitoring audio vibrations to recognize voice commands. Figure 3 The flowchart of represents machine readable instructions, which are stored in a memory (such as, Figure 2 212) and includes one or more programs, when the one or more programs are executed by a processor (such as Figure 2The processor 210) executes the vehicle 100 to achieve Figure 1 and Figure 2 The exemplary voice command controller 142 of FIG. Figure 3 The flowchart shown in describes an exemplary process, but many other methods of implementing the exemplary voice command controller 142 may alternatively be used. For example, the order in which the boxes are executed may be rearranged, changed, eliminated, and / or combined to perform the method 300. In addition, because the method 300 is combined with the Figures 1 to 2 The method 300 is disclosed by the components of FIG. 3 , so some functions of these components will not be described in detail below.

[0051] Initially, at block 302, the voice command controller 142 detects whether a key fob 132 corresponding to the vehicle 100 is within the communication range of the vehicle 100. For example, the voice command controller 142 detects whether the key fob 132 is within the communication range of the vehicle 100 via one or more of the communication nodes 130 of the vehicle 100. In response to the voice command controller 142 detecting that the key fob 132 is not within the communication range of the vehicle 100, the method 300 remains at block 302. Otherwise, in response to the voice command controller 142 detecting that the key fob 132 is within the communication range of the vehicle 100, the method proceeds to block 304, where the voice command controller 142 activates one or more of the vibration sensors 138 to detect a voice command from the user 134.

[0052] At block 306, the voice command controller 142 identifies whether a wake-up term has been detected via one or more of the vibration sensors 138 of the vehicle 100 (e.g., via speech recognition software of the vehicle 100). In response to the voice command controller 142 not detecting the wake-up term, the method 300 returns to block 302 (e.g., to determine whether the key fob 132 remains within communication range of the vehicle 100). Otherwise, in response to the voice command controller 142 detecting the wake-up term, the method 300 proceeds to block 308.

[0053] At block 308 , the voice command controller 142 determines whether the detected wake-up term was provided from an authorized source. For example, the voice command controller 142 determines whether the detected wake-up term was provided from the user 134 of the vehicle 100 corresponding to the key fob 132 .

[0054] In some examples, voice command controller 142 determines whether the detected wake-up term was provided by an authorized source using voice recognition (e.g., via voice recognition software of vehicle 100). That is, voice command controller 142 compares the pronunciation characteristics of the wake-up term detected by one or more of vibration sensors 138 with known pronunciation characteristics of user 134 to determine whether user 134 provided the wake-up term. In the described example, voice command controller 142 determines that the detected wake-up term was provided by an authorized source in response to identifying that the pronunciation characteristics of the wake-up term match the pronunciation characteristics of user 134.

[0055] Additionally or alternatively, the voice command controller 142 determines whether the detected wake-up term was provided by an authorized source by comparing the location of the key fob 132 with the location of the source of the wake-up term. For example, the voice command controller 142 identifies the location of the key fob 132 via the communication node 130 using RSSI and trilateration. The voice command controller 142 also identifies the location of the source of the wake-up term by comparing the strength of the signal detected by the vibration sensor 138. In this example, the voice command controller 142 determines that the detected wake-up term was provided by an authorized source in response to identifying that the location of the key fob 132 matches the location of the source of the wake-up term.

[0056] In response to the voice command controller 142 determining that the detected wake-up term is not provided from an authorized source, the method 300 returns to block 302. Otherwise, in response to the voice command controller 142 determining that the detected wake-up term is provided from an authorized source, the method 300 proceeds to block 310, where the voice command controller 142 triggers one or more of the vibration sensors 138 to detect a voice command provided by an authorized source (e.g., user 134). For example, the voice command controller 142 triggers the vibration sensors 138 to monitor for a voice command from an authorized source within a predetermined period of time after detecting the wake-up term.

[0057] At block 312, voice command controller 142 determines whether a voice command from an authorized source is detected by one or more of vibration sensors 138 (e.g., within a predetermined monitoring window after detecting the wake-up term). In response to voice command controller 142 not detecting a voice command from an authorized source, method 300 returns to block 302. Otherwise, in response to voice command controller 142 detecting a voice command from an authorized source, method 300 proceeds to block 314.

[0058] At block 314, voice command controller 142 determines whether the voice command is a request for information. In response to voice command controller 142 determining that the voice command is a request for information, the method proceeds to block 316, where voice command controller 142 identifies an audio response based on the voice command's request for information. At block 318, voice command controller 142 transmits the audio response to user 134 located outside cabin 106 of vehicle 100 via one or more of audio actuators 140 and / or any other vehicle speakers configured to transmit sound to an area outside cabin 106 of vehicle 100. Otherwise, in response to voice command controller 142 determining at block 314 that the voice command is not a request for information, the method proceeds to block 320.

[0059] At block 320, the voice command controller 142 determines whether the voice command is an instruction for the voice command controller 142 to perform a vehicle function. In response to the voice command controller 142 determining that the voice command is not an instruction to perform a vehicle function, the method 300 returns to block 302. Otherwise, in response to the voice command controller 142 determining that the voice command is an instruction to perform a vehicle function, the method 300 proceeds to block 322, where the voice command controller 142 identifies the vehicle function to be performed (e.g., one or more of sending a text message via the communication module 136, starting the engine 104, and opening the side window 126). At block 324, the voice command controller 142 performs the identified vehicle function.

[0060] In this application, the use of transitional conjunctions is intended to include conjunctions. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, reference to "the" object or "a and an" object is intended to also mean one of a possible plurality of the said objects. In addition, the conjunction "or" can be used to convey simultaneous features rather than mutually exclusive alternatives. In other words, the conjunction "or" should be understood to include "and / or". The terms "includes, including and include" are inclusive and have the same scope as "comprises, comprising and comprise", respectively. In addition, as used herein, the terms "module", "node" and "unit" refer to hardware having circuits, typically in combination with sensors, that provide communication, control and / or monitoring capabilities. "Module", "node" and "unit" may also include firmware executed on the circuit.

[0061] The above-described embodiments, and particularly any "preferred" embodiments, are possible examples of implementations and are set forth merely for a clear understanding of the principles of the present invention. Many variations and modifications may be made to the above-described embodiments without departing substantially from the spirit and principles of the technology described herein. It is intended that all modifications be included within the scope of this disclosure and protected by the following claims.

[0062] According to the present invention, a vehicle is provided, comprising: a vehicle window; an outer layer; a vibration sensor coupled to the vehicle window to detect audio vibrations; an audio actuator coupled to the outer layer to vibrate the outer layer; and a controller for detecting a voice command from a user via the vibration sensor, identifying an audio response based on the voice command, and transmitting the audio response to the user via the audio actuator.

[0063] According to an embodiment, the vehicle window is a windshield.

[0064] According to an embodiment, when the user is outside the vehicle cabin, the controller detects a user's voice command and transmits an audio response to the user.

[0065] According to an embodiment, the vibration sensor is an accelerometer.

[0066] According to an embodiment, an audio actuator actuates the outer layer to cause the outer layer to form an acoustic baffle.

[0067] According to an embodiment, the outer layer to which the audio actuator is coupled comprises a vehicle door panel.

[0068] According to an embodiment, the outer layer to which the audio actuator is coupled comprises a vehicle window.

[0069] According to an embodiment, the controller triggers detection of a voice command in response to detecting a wake-up term via the vibration sensor, the wake-up term preceding the voice command.

[0070] According to an embodiment, the voice command includes a request to identify a factory-recommended tire pressure, and the audio response includes the factory-recommended tire pressure.

[0071] According to an embodiment, the voice command includes an instruction for the controller to perform a vehicle function.

[0072] According to an embodiment, the instruction is selected from the group consisting of: sending a text message via the communication module, starting the engine, and opening a vehicle window.

[0073] According to an embodiment, the above invention also features a communication node for detecting when a user's key fob is within communication range of a vehicle.

[0074] According to an embodiment, the controller activates the vibration sensor in response to the communication node detecting that the key fob is within communication range.

[0075] According to an embodiment, the above invention is also characterized by a plurality of vibration sensors and a plurality of audio actuators located on the front side, rear side, driver side, and passenger side of the vehicle.

[0076] According to an embodiment, the above invention also features a communication node wirelessly coupled to a user's key fob that detects the location of the key fob via a received signal strength indicator.

[0077] According to an embodiment, the controller selects which of a plurality of vibration sensors to activate in order to detect the voice command based on the position of the key fob.

[0078] According to the present invention, a method is provided, comprising: detecting a user's voice command via a vibration sensor, the vibration sensor coupled to a window of a vehicle to detect audio vibrations; identifying, via a processor, an audio response based on the voice command; and transmitting the audio response to the user via an audio actuator, the audio actuator coupled to an exterior layer of the vehicle to vibrate the exterior layer.

[0079] According to an embodiment, the above invention is further characterized by triggering detection of a voice command in response to detecting a wake-up term via the vibration sensor, the wake-up term preceding the voice command.

[0080] According to an embodiment, the above invention is further characterized by activating the vibration sensor in response to detecting, via the vehicle's communication node, that the user's key fob is within communication range of the vehicle.

[0081] According to an embodiment, the above invention is further characterized by: detecting a location of a user's key fob via a communication node of a vehicle; and determining whether to activate the vibration sensor and the audio actuator based on the location of the key fob.

Claims

1. A vehicle, comprising: Car windows; outer layer; a vibration sensor coupled to the vehicle window to detect audio vibrations; an audio actuator coupled to the outer layer to vibrate the outer layer; as well as A controller configured to: detecting a voice command from a user via the vibration sensor; identifying an audio response based on the voice command; and transmitting the audio response to the user via the audio actuator; The vehicle further includes a communication node wirelessly coupled to the user's key fob, the communication node detecting a location of the key fob via the received signal strength indicator; as well as identifying a direction from which a voice command is provided by comparing signal strengths of the vibration sensors; Responsive to the position of the key fob corresponding to the direction of the voice command, it is determined that the detected voice command was provided by an authorized source.

2. The vehicle of claim 1, wherein the vibration sensor is an accelerometer.

3. The vehicle of claim 1 wherein the audio actuator actuates the outer layer to cause the outer layer to form a sound baffle.

4. The vehicle of claim 1 , wherein the outer layer to which the audio actuator is coupled comprises a door panel.

5. The vehicle of claim 1 wherein the outer layer to which the audio actuator is coupled comprises the vehicle window. 6 . The vehicle of claim 1 , wherein the controller triggers detection of the voice command in response to detecting a wake-up term via the vibration sensor, the wake-up term preceding the voice command.

7. The vehicle of claim 1 , wherein the voice command comprises an instruction for the controller to perform a vehicle function.

8. The vehicle of claim 1, further comprising a communication node for detecting when the user's key fob is within communication range of the vehicle.

9. The vehicle of claim 8, wherein the controller activates the vibration sensor in response to the communication node detecting that the key fob is within the communication range.

10. The vehicle of claim 1 further comprising a plurality of vibration sensors and a plurality of audio actuators located on the front, rear, driver, and passenger sides of the vehicle.

11. The vehicle of claim 1 wherein the controller selects which of the plurality of vibration sensors to activate to detect the voice command based on the position of the key fob.

12. A method for vehicle window vibration monitoring for voice command recognition, the method comprising: detecting a user's voice command via a vibration sensor coupled to a window of the vehicle to detect audio vibrations; identifying, via a processor, an audio response based on the voice command; as well as transmitting the audio response to the user via an audio actuator coupled to an outer layer of the vehicle to vibrate the outer layer, further comprising: detecting a location of the key fob via the received signal strength indicator; and identifying a direction from which a voice command is provided by comparing signal strengths of the vibration sensors; Responsive to the position of the key fob corresponding to the direction of the voice command, it is determined that the detected voice command was provided by an authorized source.

13. The method of claim 12, further comprising triggering detection of the voice command in response to detecting a wake-up term via the vibration sensor, the wake-up term preceding the voice command.

14. The method of claim 12, further comprising activating the vibration sensor in response to detecting, via a communication node of the vehicle, that the user's key fob is within communication range of the vehicle.

Images