Audio-based load control system

Abstract

A scalable, distributed load control system for home automation based on a microphone-based network can include control devices (e.g., load control devices) that can include microphones for monitoring the system and communicating audio data to a cloud server for processing. The control devices of the load control system can receive a single voice command and can be configured to select one of the load control devices to transmit the voice command to the cloud server. The load control devices can be configured to receive the voice command, control a connected load according to the voice command if the voice command is a verified command, and transmit the voice command to a voice service in the cloud if the voice command is not a verified command. The voice service to which the load control devices transmit audio data can be selectable.

Description

[0001] This application is a divisional application of the application patent application entitled "Audio-Based Load Control System" having an application date of February 7, 2018, application number 201880021333.9.

[0002] Cross Reference to Related Applications

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 455,973, filed February 7, 2017, the disclosure of which is incorporated herein by reference in its entirety. BACKGROUND

[0004] The ability to audibly (e.g., vocally and / or verbally) control devices is stimulating a technological revolution in smart home control. Voice integrated devices, such as Amazon Echo or Google Home devices, allow users to interact vocally with connected microphone / speaker devices by using a keyword (e.g., a wake word) to control other devices in a home environment or smart home network. For example, a user can integrate a voice integrated device with a lighting control system to control a light by using a keyword (e.g., "Alexa") followed by a voice command (e.g., "turn on the living room light").

[0005] Current voice integrated devices can be connected to a server (e.g., a voice service) via a network that performs speech recognition on acoustic data of a voice command after receiving a keyword. The connection can be wireless, such as a Wi-Fi enabled voice integrated device, or hardwired to a user's internet router via an Ethernet cable. After the voice service has interpreted the acoustic data, the voice integrated device can then send the interpreted data to one or more servers. The servers can communicate with a system controller that can command one or more load control devices to control an electrical load based on the interpretation of the acoustic data. The voice integrated device can also verbally respond to the user to provide an acknowledgement that the voice command was received and / or give the user confirmation of the device command sent to the smart home network. SUMMARY

[0006] There is great potential for extending the capabilities of microphone devices for residential and commercial environments. Described herein are scalable, distributed microphone device groups for integration into a home automation or load control system that includes control devices (e.g., load control devices configured to control electrical loads). While microphones can be used in standalone devices, described herein are load control devices that can include microphones for monitoring a system and transmitting measured data (e.g., audio data) to a server for processing. The server can be an internet server (i.e., a cloud server) or any type of server, for example. Control devices of a load control system that include microphones can be referred to as microphone devices or control devices or load control devices. However, it will be recognized that microphone devices do not necessarily need to be configured to control electrical loads.

[0007] A user can install one or more microphone devices to monitor audio data (e.g., acoustic data) in the surrounding environment, such as voice commands and ambient sounds. The microphone devices can be configured to learn and / or detect sounds over time and make intelligent decisions based on the recognized sounds. Alternatively, the decisions can be done remotely by a system controller or a cloud server. The cloud server can have machine learning capabilities whereby it takes passive inputs from microphone devices installed throughout the home and starts to correlate them with activities over time. In addition to the microphone devices, the system can use other inputs or information, such as occupancy status from occupancy sensors, time of day, day of week, inputs from user confirmation sounds or person recognition, etc.

[0008] A control device of a load control system can receive a single voice command and can be configured to select one of the load control devices to transmit the voice command to a voice service in the cloud. For example, the load control device can be configured to select one of the load control devices by determining which load control device heard the voice command best or highest quality.

[0009] One or more of the load control devices of a load control system can be configured to receive a voice command and either locally process the voice command if the voice command is a verified command or transmit the voice command to a voice service in the cloud if the voice command is not a verified command. The load control devices can also be configured to determine whether a voice command should be locally stored to the device as a verified command in response to repeatedly receiving the same voice command.

[0010] One or more of the load control devices of the load control system can be configured to receive a voice command, determine whether the voice command includes zone information, control a connected electrical load (such as a lighting load) according to the voice command if the voice command does not include zone information, and transmit the command to another load control device identified by the zone information if the voice command includes zone information.

[0011] One or more of the load control devices of the load control system can be configured to receive a voice command, and communicate the voice command to a system controller. The system controller can determine a zone or area from which the voice command originated according to the received voice command(s) and possible occupancy information, and control one or more electrical loads (such as lighting loads) in the area based on the voice command.

[0012] The voice service through which the load control devices of the load control system transmit audio data can be selectable. For example, the voice service can be selectable by a user through an application running on a processing device (such as a smart phone, tablet, laptop, or computer). Additionally, the voice service can be dynamically selectable by the system.

[0013] The above advantages and features are merely representative examples. They are not to be considered limiting. Additional features and advantages of the embodiments will become apparent in the following description in connection with the drawings and according to the claims. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 An example load control system with a microphone device is shown in an example user environment.

[0015] Figure 2A is an illustration of an example microphone device with an acoustic port.

[0016] Figure 2B is a cross-sectional view of the front of the microphone device of Figure 2A

[0017] Figure 3A is an example mobile application screen on a mobile device to allow a user to select a voice service.

[0018] Figure 3B is an example method that a microphone device can use to dynamically select a voice service.

[0019] Figure 3C is an example method that a microphone device can use to select a different voice service when the first selected voice service is experiencing significant latency.

[0020] Figure 4 ​These are example methods that can be executed by the server, for example, to associate received sounds with actions.

[0021] Figures 5-10 show examples of things that can be done by, for example Figure 1 A flowchart illustrating an example audio control process performed by the microphone device in a load control system.

[0022] Figure 11 This is a block diagram of an example control device (e.g., a wall-mounted keypad) that can be configured as a microphone device.

[0023] Figure 12 This is a block diagram of an example load control device (e.g., a wall-mounted lighting control device) that can be configured as a microphone device.

[0024] Figure 13 This is a block diagram of the example system controller.

[0025] Figure 14A This is an example method that a load control device can use to make a lighting load blink when it receives voice input.

[0026] Figure 14B This is an example method through which a microphone device, upon receiving voice input, can communicate with a load control device to cause a lighting load to blink. Detailed Implementation

[0027] Figure 1 This is a simplified diagram of an example load control system 100. (See diagram below.) Figure 1 As shown, the load control system 100 can be installed in a building having rooms 102, 104, and 106. The load control system 100 may include load control devices, i.e., devices configured to control one or more electrical loads. For example, the load control system 100 may include wall-mounted lighting control devices 120 (e.g., dimmer switches), light-emitting diode (LED) drivers 130, electric window covers 150, thermostats 160, and plug-in load control devices 140. The load control devices can control loads within the system, such as heating, ventilation, and air conditioning (HVAC) systems 162, lighting loads 122, 132, and 142, audio speakers 146, and other electrical equipment, such as televisions (TV) 144. In addition to switching controls (e.g., on / off controls), the load control devices may also be configured to control the level of their respective loads. For example, the lighting control device 120 and the LED driver 130 can adjust the intensity level of the corresponding lighting loads 122, 132, and 142; the electric window cover 150 can adjust the position level of the corresponding covering material 152; and the thermostat 160 can provide temperature control for the HVAC system 162.

[0028] The load control system can include a system controller 110, which can be connected to the Internet 112 via a router 114. The system controller 110 can be connected to the router 114 via either a wired connection (e.g., an Ethernet communication link) or a wireless connection (e.g., a Wi-Fi communication link). Servers on the Internet can allow for additional remote cloud processing, data storage, etc. The system controller 110 can communicate with devices in the load control system over wireless communication signals 108, which can use standard wireless protocols (e.g., ZigBee, Wi-Fi, Z-Wave, Bluetooth, Li-Fi, etc.) or a proprietary protocol (e.g., ClearConnect). The wireless protocol used by the system controller 110 to communicate with devices in the load control system 100 can be the same as or different from the wireless protocol used by the system controller to communicate with the router 114.

[0029] While the system has been described as a wireless system, alternatively, a wired system (Power over Ethernet, power line communication, CAT5 cable, etc.) for communication between devices can be implemented. Or, devices in the system can communicate wirelessly directly with the router 114 via Wi-Fi, without the need for a system controller.

[0030] The load control system 100 can also include input devices, such as a remote control 170, an occupancy sensor 172, a daylight sensor 174, and a wall keypad 120. These input devices can transmit wireless communication signals 108 to other load control devices in the load control system, either directly or through the system controller 110.

[0031] The load control devices can be configured to control respective electrical loads in response to one or more inputs. For example, the load control devices 120 can control the electrical loads 122 in response to user actuation, such as a button press. For example, a user can press a button on the load control device 120 to control the electrical loads 122. Additionally and / or alternatively, the load control devices can respond to input devices via the wireless communication signals 108. For example, the daylight sensor 174 can transmit a wireless communication signal 108 containing information about the amount of light in the room 102 to either the system controller 110 or the motorized window treatments 150 to change the level of the window coverings 152. The occupancy sensor 172 can transmit a wireless communication signal 108 including information about the occupancy status of the room in which the sensor is located to cause the respective lighting control devices 120 to automatically turn on (or off) the lighting loads 122 based on the occupancy status. For example, a user 180 can enter the room 102. The occupancy sensor 172 of the room 102 can detect that a user (i.e., the user 180) has entered the room 102. In response to the occupancy detection, the occupancy sensor 172 can transmit an occupancy command to the load control devices 120 and / or the system controller. In response to the occupancy command transmitted by the occupancy sensor, the system controller can then transmit the occupancy command to the load control devices 120, and / or the load control devices can receive the occupancy command from the occupancy sensor. In response to receiving the occupancy command, the load control devices 120 can control the lighting loads 122 by turning on the connected lighting loads 122.

[0032] Further, the occupant 180 of the residence 100 can control the loads locally or remotely by transmitting a message from a network device 182 (e.g., a communication device) to the load control device(s), such as a smartphone or tablet. For example, the occupant 180 can press a button in a mobile application on the network device 182 to turn on the lighting loads 122 in the room 102. The network device 182 can send a wireless command to the system controller 110, and the system controller can send a wireless communication signal 108 to devices in the load control system 100, such as the lighting control devices 120, to turn on the lights 122. Alternatively, the network device 182 can communicate with the system controller via the Internet 112. For example, the system controller can be connected to a router 114 via an Ethernet connection, and the network device 182 can have a Wi-Fi or data connection to the Internet 112 and / or the router 114.

[0033] Further, any of the methods previously described for controlling a load control device can also be used to control multiple load control devices to create a scene. For example, a user can press a button on a load control device, which can cause several load control devices to change the intensity of one or more lighting loads, adjust motorized window treatments, etc. Examples of scene-based load control are described in greater detail in U.S. Patent No. 6,803,728, entitled “System for control of devices,” issued to Balasubramanium et al. on October 12, 2004, which is incorporated by reference herein. Load control systems responsive to a system controller are described in greater detail as the broadcast controller by Donald Mosebrook et al. in U.S. Patent No. 9,337,943, entitled “Load control system having a broadcast controller with a diverse wireless communication system,” issued May 10, 2016, and as the system controller described in U.S. Patent Application No. 20170123390, entitled “COMMISSIONING LOAD CONTROL SYSTEMS,” published May 4, 2017, by Kyle Thomas Barco et al., which are incorporated by reference herein. These patents provide detailed examples of how the load control system 100 can operate. Other examples are possible.

[0034] Load control devices of the load control system 100 can include microphones and can be referred to herein as microphone devices. For example, the plug-in device 190, which can be mounted on a wall or flat surface or placed on a table, can include a microphone. Further, the load control system 100 can include microphone devices that can be separate, dedicated microphone devices in the load control system, i.e., the load control system can contain one or more microphone devices that do not control electrical loads.

[0035] Wall-mounted load control devices, such as the lighting control devices 120 and the keypads 176, which can be installed in electrical wallboxes, can include microphones and can have better reception of acoustic sound and / or noise because these devices are installed at a height above the floor and desk surfaces and can have reduced obstructions. In addition, electrical wallboxes are typically installed near doorways, thus enabling the microphone devices to easily receive acoustic sound and / or noise from occupants moving through the doorway and / or from the opening and closing of the door. In the case where microphones are integrated into wall-mounted load control devices in electrical wallboxes, the load control system 100 can provide a network of microphone devices located in several rooms of a house or building and can listen for voice commands and / or ambient sound. Each wall-mounted load control device can include a microphone (e.g., the lighting control device is a microphone device), or alternatively, the microphone device can be installed in the wallbox adjacent to the load control device but in a separate housing (e.g., the lighting control device and the microphone device are separate devices). Or, the microphone device can be installed in the faceplate of one or more load control devices. Other examples are possible.

[0036] When microphone devices are integrated in load control devices, the microphone devices can communicate on either a different protocol or a different frequency channel for acoustic data due to the higher data throughput requirements of audio signal processing as compared to system communications. For example, the microphone devices and / or system controllers can use a first protocol (e.g., Wi-Fi or BLE) for streaming acoustic data and a second protocol (e.g., ZigBee, Thread, ClearConnect, Z-Wave, etc.) for communications with the load control system, where the first protocol can have a higher data throughput than the second protocol. For example, the load control system can use the second protocol to communicate load control commands, and the first protocol can be reserved for communicating acoustic data. The microphone devices can be integrated with the load control system.

[0037] The control devices of the load control system 100 can be programmed (e.g., commissioned and / or configured) in response to received voice commands. For example, a user can associate a remote control device with a lighting control device by pressing a button on the remote control device and saying “associate remote.” In addition, a user can use voice commands to adjust operational settings of the control devices (e.g., high-end trim, low-end trim, fade time, delay time, occupancy sensor timeout, occupancy sensor sensitivity, etc.).

[0038] The use of microphone devices for voice recognition in a load control system can provide a number of benefits to a user, including but not limited to: control of loads; human voice interaction with web search engines for weather and traffic information, etc.; intercom or telephone calls, conversations, and voice recordings, etc. The microphone device can also be operable to replay conversations.

[0039] Figure 2A FIG. 1 is a front view of an example microphone device 200, which can be a load control device. For example, the microphone device 200 can have integrated lighting control capabilities, i.e., the microphone device 200 can control an electrical lighting load. The microphone device 200 can be mounted in an electrical wallbox (not shown) and can have a faceplate 202. The faceplate 202 can cover the electrical wallbox.

[0040] The microphone device 200 can also include a bezel 212. The bezel 212 can include one or more buttons. For example, the bezel 212 can have buttons 202, 206 that can turn an electrical lighting load on and off, respectively; a button 204 that can raise or lower a light level; and a button 210 for a preset light level. For example, the microphone device 200 can change a light intensity of an electrical lighting load in response to a button press on any of the buttons 202-210 of the microphone device 200. The preset button 210 can be a user-chosen preset light level that can be configurable by a user. Examples of presets for a lighting control device are described in greater detail in U.S. Patent No. 6,380,696, entitled “MULTI- SCENE PRESET LIGHTING CONTROLLER,” issued April 30, 2002, to Tarvinder Sembhi et al., the entire disclosure of which is incorporated by reference herein.

[0041] The microphone device 200 can include one or more input microphones (not shown) that are recessed behind the bezel 212. The bezel 212 can have holes 208 for allowing sound to pass through the bezel. Alternatively, the input microphones located within the housing of the microphone device 200 can be covered by a speaker grill, cloth, etc. to protect them from dust, debris, and damage. The input microphones can be recessed from the surface of the housing and acoustically directed to the surface of the housing via a tube, opening, or acoustic horn. For example, the input microphones can be configured to utilize a gap in the plastic, which can allow sound to reach the microphones without the need for holes, grills, etc. on the surface of the microphone device that would ruin the aesthetic design of the microphone device.

[0042] Figure 2Bis a side cross-sectional view of the microphone device 200 taken through the aperture 208. The aperture 208 can be configured such that the front top edge 208A of the aperture can be vertically at the same point or vertically at a lower point than the back bottom edge 208B of the aperture. For example, if the bezel 212 has a thickness t and the aperture has a diameter d, then the angle Θ can be greater than or equal to the arctangent of the aperture diameter divided by the material thickness according to the following formula:

[0043] Θ > arctan(d / t)

[0044] The upward tilt of the aperture can be used to reduce the amount of dust and / or debris that enters the conduit, as well as to improve the aesthetic appeal by making the aperture appear darker. While this structure is described for a single aperture, it will be understood that this can be used for multiple apertures, including an array of apertures.

[0045] Referring back to Figure 1 , the microphone device can receive acoustic sound and transmit acoustic data to cloud servers on the internet 112 to process the acoustic data. The internet 112 can be connected to a number of different cloud servers that process acoustic data. Each cloud server can host a service that processes acoustic data. For example, the internet 112 can be connected to one or more cloud servers that host a voice service for interpreting acoustic data that includes voice commands. For example, the voice service can include: a first cloud server 116 that hosts Amazon voice service; a second cloud server 117 that hosts Google voice service; and a third cloud server 118 that hosts other voice services, etc.

[0046] The microphone device can identify acoustic signatures and transmit them to remote cloud servers for additional processing. The acoustic signatures can be audio data of interest. For example, the acoustic signatures can be all acoustic signals sensed by the microphone device, or it can have some constraints, such as acoustic signals that are specifically intended for processing (e.g., as identified by a particular keyword or wake word), or acoustic signals that fall within or exceed one or more thresholds, such as frequency and / or amplitude thresholds, as will be described in greater detail herein.

[0047] A microphone device can detect specific human voice keywords, i.e., "wakeup words," by which the microphone device begins recording acoustic data and transmitting the acoustic data to a voice service after detecting the wakeup word. For example, a user can say "Alexa" to trigger the microphone device to start listening. Then, when a subsequent phrase is uttered (e.g., "what's the weather?"), the microphone device can then transmit the acoustic data. For example, the microphone device can then transmit the subsequent phrase to a cloud server for acoustic processing. Alternatively, the microphone device can transmit the data to the system controller 110. The system controller can then transmit the data to a voice service on a cloud server for acoustic processing, or the system controller can process the acoustic data locally.

[0048] Based on determining whether the signal is an acoustic signature (i.e., audio data of interest for additional processing), the acoustic processing and recognition can be done locally either at the microphone device or at the system controller. If the microphone device or the system controller determines that additional processing is necessary, the acoustic data can be sent to a server (i.e., a cloud server on the Internet) for processing. The microphone device can transmit the acoustic data to the system controller for sending to the cloud server through the router 114, or the microphone device can transmit the acoustic data directly to the router 114 via Wi-Fi (i.e., without a centralized system controller). Alternatively, the microphone device can transmit the acoustic data to the system controller, and the system controller can interpret and determine whether the data should be kept local or transferred to a cloud server for cloud processing. Although a cloud server has been described, it will be understood that any server can be used, e.g., a dedicated server. For example, instead of or in addition to a cloud server, the system controller can handle some or all of the acoustic processing.

[0049] One of the load control devices of the load control system 100 (e.g., one of the lighting control devices 120) can be configured to receive a voice command and transmit acoustic data to a cloud server for processing. The load control device can then receive a response from the cloud server, interpret the response to determine a command for controlling a connected electrical load, and control the connected electrical load in response to the received voice command.

[0050] The microphone device can monitor, for example, received audio input from a user and corresponding received responses from a cloud server, and over time learn responses to acoustic signatures that are commonly used for that space. For example, a command for "turn on light" that is the most frequent command received by the microphone device in the kitchen can allow the system (i.e., the microphone device and / or the system controller) to associate the acoustic signature "turn on light" with a load control command to turn on a light. In this way, the load control system can respond to these learned commands with local determinations, without the need for cloud processing.

[0051] Machine learning and pattern recognition of audio input can be processed locally at the device, system controller, or can be processed within a cloud server and transmitted to the device. For example, the cloud server can determine the most frequently used command phrases, and can "teach" the microphone device and / or the system controller the characteristic audio signatures to respond to. This local response optimization of frequently used commands for the area can help reduce lag in response time, as well as reduce system dependency on Wi-Fi and internet communication if external communication is down.

[0052] Cloud processing can use a cloud-based speech recognition software service, i.e., a voice service located on a remote cloud server, such as Amazon Alexa®, Amazon Voice Service, Siri, Cortana, Google, etc. The microphone device can be voice service agnostic, i.e., the voice service used for signal processing and speech recognition can be selectable and not preconfigured.

[0053] The voice service can be configured by the user. In a first example, the voice service can be configured at setup of the system. For example, the user can select a voice service to use with the microphone device as part of the configuration of the microphone device or the load control system. In a second example, the microphone device can dynamically select which voice service to use from two or more voice services each time a voice request is made. The dynamic selection of the voice service can be based on the type or content of the request, which can be set during configuration of the microphone device, as described herein.

[0054] The voice service can be chosen from a list of voice services in the mobile application at setup. For example, a user can set up a voice service for a load control system via a mobile application on a cell phone or tablet. The mobile application can communicate wirelessly with the microphone device. For example, the mobile application can communicate with the wireless device via Bluetooth or Wi-Fi. Alternatively, the mobile application can communicate with the system controller 110 via Bluetooth or Wi-Fi. The system controller can then communicate to the microphone device which voice service to use. Alternatively, the microphone device can transmit the acoustic data to the system controller, and then the system controller can transmit the acoustic data to the voice service, as previously described. Figure 3A An example mobile application on a mobile device 300 is shown, where a screen 302 allows a user to choose from a list 304 of voice services, e.g., Alexa 304A of Amazon voice service, Google Voice 304B by Google, or Siri 304C of Apple. This list can be extended to include other voice services, such as Microsoft Cortana, etc. Further, the user can have the option to skip 306 the selection, and can choose a default voice service.

[0055] Alternatively, the voice service used can be dynamic, i.e., dependent on the type or content of the user request. For example, the query "how is the current traffic" can be sent to a Google server for signal processing, while "tell me a joke" can be sent to an Amazon voice service. According to this embodiment, the recognition of the words of interest within the phrase can determine which cloud service to use after the initiation keyword, i.e., the wake word, has been spoken.

[0056] As described, the recognition of the spoken command can require a wake word or initiation keyword, such as "Alexa" or "Google". The wake word can determine which voice service to use, i.e., the voice service can be dynamically chosen from a plurality of voice services based on the wake word used. For example, the Amazon voice service can require the user to use the keyword "Alexa". As described above, the wake word can be used to choose the voice service for remote signal processing. For example, if the user speaks the initiation keyword "Alexa", then the voice data can be sent to the Amazon voice service, while if the wake word "Siri" is used, then the voice data can be sent to an Apple server for voice processing.

[0057] In addition to traditional wake words associated with specific voice services, a user can also set a generic wake word during configuration of the microphone device. The generic wake word can be any word chosen by the user and configured as a wake word at the time of setting. For example, the word "home" or "computer" can be used as a generic wake word. The user can use a mobile application to set the generic wake word. For example, the mobile application can communicate wirelessly with the microphone device either directly or via a system controller and / or router. The mobile application can prompt the user to enter or speak an initiation keyword that can then be used as the generic wake word.

[0058] Figure 3B is an example method 320 that a microphone device can use to dynamically choose a voice service. The microphone device can detect a wake word at step 330. After detecting the wake word, the microphone device can listen at step 332 to receive voice input from the user following the wake word. At 334, the microphone device can determine whether the wake word is a generic wake word. The generic wake word can be a word chosen by the user at the time of setting that is not necessarily associated with a particular voice service. For example, the wake word can be a word such as "home" or "computer."

[0059] If the wake word used is a generic wake word, then at 336, the microphone device can determine whether a word of interest has been detected within the voice input. The word of interest can be a keyword that triggers a specific voice service. The word of interest can be processed locally by the microphone device or a system controller. If the word of interest has been detected, then the microphone device can determine which voice service is associated with the word of interest at step 340. For example, the microphone device can have a lookup table containing a list of words of interest, where each word of interest is associated with a particular voice service. One or more words of interest can also be chosen by the user at the time of setting. For example, the user can enter one or more words of interest by typing the one or more words of interest or choosing the one or more words of interest from a list in a mobile application and choosing a voice service to use for each word. Alternatively, the list can be a default list that has been configured at the time of setting and the user can optionally change which voice service to use for each word of interest. For example, some example words of interest can be "traffic" or "jokes."

[0060] At step 344, the microphone device can transmit the voice input to the associated voice service for additional processing and respond to the user input. For example, the word "traffic" can trigger the use of Google servers while "jokes" can trigger the Amazon voice service. The method can then end.

[0061] If the word of interest is not detected within the voice input at step 336, the microphone device can transmit the voice input to a default voice service. The default voice service can be set by the user, or can be the voice service used when the user has not yet picked a voice service.

[0062] If the generic wake word has not been detected, the microphone device can determine whether the spoken wake word is associated with voice service A at step 352. For example, if voice service A is the Google voice service, and the wake word is "Hey, Google," the microphone device can determine that the wake word is associated with voice service A. The microphone device can transmit the voice input to the associated voice service (i.e., the Google voice service) at step 344. However, if the wake word is "Alexa," the microphone device can determine that the wake word is not associated with voice service A (i.e., the Google voice service) at step 352. The microphone device can then determine whether the wake word is associated with, for example, a second voice service (voice service B, such as the Amazon voice service) at step 354. If the wake word is associated with voice service B (e.g., the wake word is "Alexa" and voice service B is the Amazon voice service), the microphone device can transmit the voice input to the associated voice service at step 344. However, if the wake word is not associated with voice service B, the microphone device can then transmit the voice input to a default voice service (e.g., such as Microsoft Cortana). The method can then end.

[0063] It will be understood that the voice services described herein are for example purposes only, and that any voice service can be used. Furthermore, the microphone device can not be limited to three different voice services, but can use any number of different voice services. Also, while the method is shown in a particular order, it is contemplated that the steps of the method can be used in any order.

[0064] The flexibility of selecting and / or using a voice assistant (either by the user through manual picking, or dynamically during operation of the load control system) can provide several advantages. First, the user can be more flexible in selecting future home integrated devices without having to pre-select (i.e., at the time of purchase) a specific service that can not be compatible with future devices. Second, in the event that the default cloud server is down or is experiencing significant latency, the system (i.e., the microphone device or the system controller) can select a different voice service to process after the expected response time period has passed to ensure service continuity to the user.

[0065] For example, Figure 3Cis an example method 370 by which a microphone device can select a different voice service when the first selected voice service is experiencing significant latency. The method 370 can begin at 372 when the microphone device transmits data to a first voice service. At step 374, after transmitting the acoustic data to the voice service on the cloud server for processing, the microphone device can start a timer or counter. The microphone device can then determine whether a response has been received at step 376. If the microphone device receives a response from the voice service before the counter exceeds a threshold, then the microphone device can stop the timer at step 380 (e.g., reset the counter to zero and stop the counter). The method can then end.

[0066] However, if the microphone device does not receive a response from the voice service at step 384 and the counter exceeds the threshold, then the microphone device can transmit the acoustic data to a different voice service on a different cloud server for processing at step 388. At step 374, the device can either reset the counter or start a second counter / timer and repeat the process.

[0067] Alternatively, the microphone device can transmit the audio data to two or more voice services simultaneously and wait to receive a response from the faster voice service. While the methods herein have been described with respect to a microphone device, it can be appreciated that a system controller can instead transmit the audio data to one or more voice services for processing. For example, the microphone device can communicate with the system controller to determine which voice service to use. For example, the microphone device can transmit a wake word or a wake word and command to the system controller, and the system controller can interpret which voice service to use according to any of the preceding examples. The system controller can then tell the audio device which voice service to use, or the system controller can transmit the audio data directly to the voice services and subsequently provide the responses to the respective audio devices.

[0068] For example, the microphone device can transmit a wake word to the system controller. The system controller can interpret the wake word and determine which voice service from the set of voice services is associated with the wake word. The system controller can then receive acoustic data from the microphone device and transmit the acoustic data to the voice service associated with the wake word (i.e., the selected voice service).

[0069] In a second example, the system controller can select a voice service based on a wake word from a microphone device, as previously described. The system controller can then transmit which voice service is the selected voice service back to the microphone device. The microphone device can then transmit acoustic data to the selected voice service.

[0070] In addition to human voice commands and conversations, the microphone devices can also respond to other sounds. For example, the microphone devices can recognize: other user-generated sounds, such as screams or shouts; personal sounds, such as whistles, clapping, snaps; snoring; coughing / sneezing; laughing; etc. Any or all of these user-generated sounds can be used to recall a scene. For example, a scene can be triggered based on clapping, which can turn on all the lights in the space.

[0071] The microphone devices can also recognize other environmental sounds, such as sounds from: appliances, media, water, cooking, movement, device malfunctions, emergency sounds, health / mood, air flow, outside / outdoor sounds, and pet sounds, etc. The microphone devices can passively sense these other environmental sounds that can act as inputs for which a user can specify a desired output action through the load control system. In one example, a user can configure the load control system to integrate with non-“smart” enabled appliances and other home devices by learning the input sound and the desired action based on the input sound. In this way, a state change is sensed acoustically, rather than requiring the appliance to have additional processors and wireless communication capabilities. For example, upon receiving a particular acoustic signature, the microphone device 120 can determine that the dryer 176 has completed its cycle. The microphone device 120 can then initiate other actions in the system, such as causing the lights 122 in the room 102 that the user (i.e., the occupant 180) is occupying to flash, or sending a push notification to the mobile device 182 to let the occupant 180 know that the dryer has completed its cycle.

[0072] The load control system can also enable scenes in response to certain acoustic signatures. For example, the microphone devices can recognize a distinctive sound, such as the Deep Note audio trademark of THX playing on the television 144. The load control system can then automatically pick a movie scene, where the load control system can lower the shades 150 and dim the lights 132, 122 in the room 106. In another example, the microphone devices can listen to a combination of sounds to determine an action. For example, the microphone devices can recognize the acoustic signature of an alarm clock, and the load control system can cause the lights to fade up, or delay or extend the fade after a “snooze.” For example, if the microphone devices recognize the acoustic signature of an alarm clock, and the sound abruptly stops, the microphone devices can listen for additional sounds of the person getting up. If the microphone devices do not hear additional noise after the alarm clock abruptly stops, the load control system can determine that the user has pressed the snooze button. In a third example, the load control system can turn on a bathroom exhaust fan in response to the microphone devices recognizing water noise (i.e., from a shower) for an extended period of time (e.g., more than two minutes). The microphone devices can also detect leaks by the sound of water droplets, and can alert the user of the sound and location.

[0073] The user can set the load control system to provide various output responses based on the recognized acoustic input. That is, the load control system can learn to associate a particular action or load control command with a particular sound. The load control system can generate a warning such as for a water leak, send a push notification to the user, etc. The microphone device can also be integrated with other devices in the system, e.g., occupancy sensors.

[0074] Additional output responses can be enabled in a load control system with machine learning capabilities, which allows the system to adapt to a changing home environment by pattern mapping and correlating measured acoustic data with a database of known sounds, time of day, specific users and locations, specific activities, other system states. The discovery can be further enhanced by active reporting and feedback from the user, and can additionally include prompted queries to the user for confirmation and / or identification of events to provide additional information that then assists the system in pattern mapping and correlation. The user queries can be any one or combination of voice communication, mobile notification, audiovisual communication, etc.

[0075] The database can store and catalog sounds in a sound library. The sound library can be built over time by collecting and "crowdsourcing" sounds from various user inputs. Alternatively or additionally, the sound library can be built through a learning or commissioning period, where users actively teach different sounds to the database, or the database can passively learn the sounds by correlating the sounds with specific actions via machine learning. The sound library can be stored on a server, e.g., a cloud server 118 on the Internet 112, in the device's own memory, in the system controller, or any combination of these. Using machine learning algorithms on the cloud server during the learning period can allow the system to operate independently of the cloud after the learning period has been completed, i.e., without needing access to the Internet 112.

[0076] This can also be used, for example, during setup of the system to configure any word to be used as a universal wake word. For example, the user can repeatedly say the universal wake word during a training mode, which can be initiated during configuration of the microphone device or load control system. For example, the training can be done through a mobile application. The mobile application can communicate wirelessly with the system controller or cloud-based server. The microphone device and / or system controller can also be trained to recognize specific sounds. For example, the user can repeatedly open and / or close a door to train the microphone device to recognize the sound.

[0077] Users (e.g., occupant 180) can use a mobile application to associate sounds with specific responses from a load control system. For example, a user can manually train the load control system to turn on a light in response to a specific sound. The user can manually train the load control system to recognize specific sounds by repeatedly creating and teaching the sounds (e.g., via a mobile application) to the system. The user can then input the desired specific response from the load control system when the sound is recognized, or the user can choose from a list of suggested actions. For example, when a microphone device hears a door open, the load control device can turn on a light. This association between sound and corresponding actions of the load control system can also be used in conjunction with conditional logic (similar to, for example, If This Then That (IFTTT) logic). For example, if a microphone device hears a bedroom door open at night, the load control system can turn on the bathroom light to a dimly lit level.

[0078] Load control systems can also use machine learning to learn any sound. For example, a load control system can automatically learn a sound and begin associating it with actions. For instance, a load control system can be configured to log sounds and their corresponding actions, which can be processed by a server on the Internet. For example, the server could be such as... Figure 1 Servers such as "other servers" 118 shown may be capable of processing general acoustic sounds and may also be capable of machine learning and storing load control actions. For example, a microphone device may transmit audio data to system controller 110. The system controller may then transmit the audio data to the server. After (or before) the transmission of the audio data, a load control device may control an electrical load. The system controller may also transmit load control actions to the server. For example, a microphone device may record the sound of a bedroom door opening. The load control device may then turn on a bathroom light to a dim level. The sound of the bedroom door opening and the action of turning the bathroom light to a specific intensity level may be sent to server 118. For example, the microphone device and the load control device may transmit the acoustic signature and load control action directly to the server, or the microphone device and the load control device may transmit the acoustic signature and load control action to the system controller. The system controller may then transmit the acoustic signature and load control action to the system controller. As the server begins recording information over time, it may be configured to use machine learning to notice patterns and correlate them. For example, when a bedroom door noise is heard, the bathroom light should be turned on to a dim level, but only at night.

[0079] The load control system can also use machine learning to determine actions associated with specific users. For example, during or after a learning period, the system can determine that the home is comprised of three separate people. The microphone devices and / or system can then ask the users to confirm to identify that there are three separate people present, and upon hearing a sound or voice command, the system can guess the specific user and / or prompt one of the occupants to name an identity associated with that user. The microphone devices and / or system controller can query the user via audio communication, push notifications on a cell phone, a wearable device, a tablet or other mobile device, a television display, etc. Over time, as the load control system learns the sounds in the environment and can identify different users of the space, the load control system and / or microphone devices can also begin to predict the user’s path of movement and personalize the space for the user, which can include storing personalized preferences, predicting effects the user desires, or making suggestions to the user.

[0080] The use of a sound library and machine learning algorithms can allow the microphone devices to become aware of whether a sound in the space is unknown. The detection and recognition of the sound can be based on a confidence threshold for correct identification. When the confidence threshold has been reached for an unrecognized sound (i.e., the system has determined that the sound is not background noise), the microphone devices can alert the occupants of the detection of an abnormal sound. Further, sounds related to alarm sounds from a known alarm sound library, such as screams, breaking glass, falls or drops, etc. can be used to detect an emergency in a common area and provide actions based on the type of emergency. For example, if the load control system determines that one or more microphone devices have detected breaking glass, the load control system can issue a command to a security system or notify security personnel.

[0081] The load control system can also be able to detect health and emotional sounds and / or autonomously make adjustments in the space for occupant comfort and productivity. For example, if the load control system determines that productivity has decreased (e.g., the typing rate on a keyboard has slowed) and the ambient background noise is high, the load control system can provide focused noise reduction in the direction of the user’s typing. For example, one or more speakers in the load control system can output a sound to cancel out the noise in the room. Other sensor inputs can also be used to determine the user’s location, such as image sensing, local pressure sensors in the user’s chair, RF beacon technology, etc. Alternatively, if the load control system determines that productivity has decreased and the ambient background noise is low or normal, the load control devices can increase the brightness or color temperature of the lighting in the space to make the user more alert.

[0082] The load control system can also provide feedback, e.g., aggregate data measuring viewer responsiveness to movie previews or advertisements. Particularly in commercial office or manufacturing environments, the load control system can also be able to track productivity levels and OSHA noise hazards and report them to facility managers or supervisors. Depending on the room or area, different sound levels can indicate different levels of productivity. For example, loud machinery operation can indicate that the productivity level is sufficient; however, if the environment is quiet, the absence of machinery noise can indicate that the machines are experiencing downtime or the production line can be having issues. In a second example, loud background noise in an office space can indicate that there is a distraction, such as people talking, which can indicate a lower level of productivity. The load control system can be configured (i.e., programmed) with certain sound volume levels that are expected, which can be correlated to the described levels of productivity. For example, if the sound in a room is outside of the expected volume level or range (i.e., greater or less) for a period of time (which can be defined when configuring the load control system, e.g., 10 minutes), then the load control system can trigger a warning.

[0083] The load control system can also include speakers. The speakers can be integrated into one or more microphone devices and / or the speakers can be separate from the microphone devices. The speakers can provide focused noise reduction direction to specific areas in the room, e.g., using beamforming. Alternatively, the speakers can increase the amount of white noise in the space. Other sounds that the system can respond to can include: water sounds (leaks, drips, usage measurements), cooking, movement (walking, opening doors, opening windows, pulling blinds), device failure sounds (mechanical noises, rattling, clicking, etc., indicating that an appliance is reaching the end of its life), air flow, pet sounds, etc.

[0084] The system's response based on received acoustic inputs can be learned over time based on how users respond to the sounds and integrating the user responses into the system responses. For example, the system can learn that when a doorbell rings after sunset, the user will turn on the outside light when answering the door. The system can then learn to anticipate the user's response and perform the intended action for the user. The system responses can include control of loads, predicting user movements and / or paths, determining user's mood or activity, assessing the urgency of a situation, generating warnings, tracking activities, and generating reports.

[0085] When an expected sound does not occur, the load control system can alert the user that the expected sound (i.e., the expected activity) has not occurred. For example, when the system learns that a child comes home at a certain time, if the microphone device does not hear a door close, then the system can determine that this event has not occurred. When the system determines that the door was not closed, the system can alert the user that the door was not closed, and / or if the act of closing the door has been associated with the child coming home, then the system can remind the user that the child has not arrived home. As previously described, the user can teach the load control system which sounds are associated with which actions or events.

[0086] Figure 4 is an example method 400 that can be performed by a server to associate a load control action with an acoustic sound. The method can begin at step 402 when the server receives acoustic data. The acoustic data can be audio data of interest received from a microphone device. For example, the acoustic data can correspond to a user opening a door and walking into a room.

[0087] After receiving the acoustic data, the server can receive a notification at step 406 that a change in state (i.e., a load control action that can be defined by a command) has occurred in the space. For example, a user can press a button on a load control device, such as a light switch or dimmer. In response to pressing the button, the load control device can control (e.g., turn on) an electrical load, such as a light, for example. The load control device can transmit a notification to the server indicating the change in state (i.e., that the load control device has turned on the load). It will be understood that an external sensor can be used in place of the load control device to transmit the notification. For example, a light sensor can detect that a light has been turned on in the space, and the light sensor can transmit a notification of the change in state to the server. Further, the server can transmit a command to the load control device, for example, in response to receiving an input from a remote control device and / or a network device, such that the load control device does not need to transmit a notification to the server.

[0088] At step 408, the server can store the acoustic data, the notification (e.g., a command defining how the load control device controls the electrical load), the current day of the week, and / or the current time of day. Further, the server can store at 408 the time of day and / or the day of the week that the acoustic sound occurred. For example, the server can store the foregoing to a memory. At step 412, the server can determine an amount of time between receiving the acoustic data and receiving the notification. For example, the server can receive acoustic data of a door opening two seconds before receiving a notification that the light has been turned on. Further, the server can determine at step 412 whether the load control device controlled the electrical load within a period of time after the time of the acoustic sound occurred.

[0089] At step 414, the server can compare the recorded acoustic data and the notification to any previous recordings that can be stored on the server. For example, the server can store a door closing sound each time it hears the sound. Then, the server can compare the currently received acoustic data to each previous recording to determine which recordings match the currently received acoustic data. Then, the server can compare the notifications of the state change (e.g., commands) and confirm that the state change is the same for each notification.

[0090] Alternatively, the server can compare the currently received acoustic data to a library or database of known acoustic sounds, as previously described. The server can correlate the currently received acoustic data to one or a group of acoustic sounds to identify the sound. For example, the server can identify the currently received acoustic data with opening a door (or closing a door). This can allow the server to store the received acoustic data as opening a door (or closing a door) rather than storing all of the acoustic data in order to save space.

[0091] At step 416, the server can determine a pattern between the acoustic sound and the notification. For example, the server can use machine learning to determine that the acoustic sound and the notification occur in similar situations at a specific time of day and / or on a specific day of the week. Further, the server can determine at step 416 whether two acoustic sounds are similar and whether two related commands are the same. The server can use information from historical records to determine the pattern. After determining the pattern, the server can then associate an action with the acoustic data at step 418. For example, the server can associate the action of opening a door with the notification of a change in the state of a light from off to on (i.e., the action of turning on a light). Further, at step 418, the server can store a verified command as being associated with a verified acoustic sound, where the verified acoustic sound represents two acoustic sounds that have been determined to be similar and the verified command represents two commands that are related to the acoustic sound and have been determined to be the same. For example, one of the acoustic sounds can be stored as a verified acoustic sound and one of the commands can be stored as a verified command. The server can also store a verified time range and / or a verified day of the week at 418.

[0092] The server can associate the action with the acoustic data according to a condition. For example, the action can be associated with the acoustic data only during a certain time of day or day of the week according to the associated pattern. At step 420, the server can receive subsequent acoustic data. For example, the server can receive acoustic data corresponding to the opening of the door. Upon receiving the acoustic data, the server can determine in step 422 whether the acoustic data matches the pattern of acoustic data associated with the action. That is, the server can determine that the subsequent acoustic sound is similar to the verified acoustic sound before transmitting the verified command to the load control device. The server can then transmit the control command (i.e., the verified command) to perform the action at step 424 in response to receiving the acoustic data. The method 400 can then exit.

[0093] While this method has been described with the server as an example, it will be appreciated that the system controller can instead use the same described method or adaptations of the described method.

[0094] Designing each microphone device to continuously stream acoustic data to the cloud for processing can cause wireless congestion issues. In addition to requiring a keyword to initiate voice processing, there are several additional congestion mitigation options to reduce the bandwidth of acoustic data streaming from the microphone devices to the internet. The load control system can impose an amplitude or frequency limit on the acoustic data, which can be set by the user, initiated by the system controller, or set and stored in the microphone device itself. These limits can apply a threshold to the data such that either very quiet and / or noise outside of the frequency region of interest (i.e., very low frequency vibrations / sounds, or high frequency sounds) are filtered out by the microphone device and not transmitted to the cloud for processing, thereby constraining the amount of data.

[0095] Multiple microphone devices can receive or hear a single voice command. The microphone devices can be configured to communicate with each other to determine which microphone device(s) should respond to the voice command and / or which microphone devices should communicate with the voice service in the cloud. For example, the microphone devices can be configured to determine which microphone device best heard the voice command.

[0096] Microphone devices that are in close proximity to each other and receive the same acoustic signature can also communicate with each other to determine which device has received the highest quality acoustic signature and only enable one device to transmit the acoustic data for further processing. For example, several microphone devices in the same room can decide which microphone device should transmit the acoustic data to the internet for further processing based on the volume of the received sound (i.e., the amplitude of the acoustic signature).

[0097] For example, the microphone devices can determine which microphone device best hears the voice command (or has the highest quality) by one or more of the following criteria: volume, proximity to the sound source, minimum multipath interference (i.e., least amount of phase noise and / or acoustic echo), highest signal-to-interference ratio (i.e., signal-to-noise ratio or SNR ratio), and / or occupancy of the area. For example, proximity to the sound source can be determined based on phase delay of the received sound. For example, upon receiving the voice command, each microphone device can transmit an average volume of the heard command. Each microphone device can receive the average volume of the heard command from the other microphone devices. Each microphone device can then compare its own average volume reading to the average volumes received from the other microphone devices and determine whether its own average volume is the largest based on the comparison. The microphone device that determines that its average volume is the largest can then transmit acoustic data to the system controller and / or cloud server for voice processing.

[0098] As an alternative to the microphone devices analyzing the data and determining which microphone device best hears the voice command, the system controller 110 can be configured to analyze the audio data from the microphone devices to determine which microphone device best hears the voice command. Additionally, the microphone devices and / or the system controller 110 can cooperate (e.g., share audio data) to determine the content of the voice command.

[0099] The system can also use data from the occupancy sensors to determine which microphone devices are enabled to transmit acoustic data. For example, a user can configure a space such that acoustic data is enabled to be transmitted by the microphone devices in the room 102 only when the occupancy sensors in that space sense that the room is occupied. This can require the user to set the room in the system (e.g., via a mobile application) to configure the system controller and / or the microphone devices to respond to an occupancy signal from the occupancy sensors in the room 102.

[0100] In a first example, the occupancy sensor 172 of the room 102 can transmit an occupied command to the load control devices 120. For example, the occupancy sensor and the load control devices 120 can be associated, and the load control devices 120 can receive the command from the occupancy sensor. The load control devices 120 can be microphone devices. When a user 180 speaks an audio command, the audio command can be heard by one or more microphone devices in the rooms 102, 104, 106. The microphone devices can determine whether the room (i.e., the room 102, 104, or 106, respectively) is currently occupied based on the most recent occupied command. If the microphone devices determine that the room is not occupied, then the microphone devices can not transmit audio data based on the audio command. However, if the microphone devices determine that the room is occupied, then the microphone devices can transmit audio data based on the audio command. In this way, the number of microphone devices that transmit audio data can be reduced.

[0101] In a second example, the occupancy sensor 172 can transmit an occupied command to the system controller 110. The system controller can then also control the flow of audio data. For example, if the room in which a microphone device is located is currently occupied, then the microphone device can transmit audio data to the system controller, which can transmit the data to a server on the Internet 112. Alternatively, the system controller can instruct one or more microphone devices located in a room that is not occupied not to transmit audio data.

[0102] Alternatively, the microphone devices can contain occupancy sensors, and can use their own occupancy signals to determine whether to transmit audio data. For example, if a microphone device determines that the room is occupied, then it can transmit audio data. However, if the microphone device determines that the room is not occupied, then it can not transmit audio data.

[0103] Figure 5Ais a simplified flowchart of an example audio control process 500A that can be performed by a load control device of a load control system (e.g., one of the lighting control devices 120 of the load control system 100). The load control device can be configured to control an electrical load. The load control device can include a microphone to receive audio data (e.g., voice commands and / or sounds) and can be configured to directly communicate with a voice service in the cloud to allow control of an electrical load connected to the load control device. At step 510, the load control device can detect a wake word. After detecting the wake word at step 510, the load control device can receive a voice input including a voice command at step 512. The load control device can transmit the voice input to a voice service in the cloud (e.g., the Alexa voice service) at step 515 (e.g., via the router 109 and / or directly to the Internet via the system controller 110). The load control device can receive a response from the voice service at step 516 and can interpret the response at step 518 to determine a command for controlling the electrical load. At step 520, the load control device can then directly control the connected electrical load in response to the interpreted command received from the voice service before the audio control process 500 exits.

[0104] In addition to receiving and responding to voice load control commands from a user, the load control device can also respond to other voice requests. Figure 5B is a simplified flowchart of an example control process 500B that can be performed by a load control device of a load control system to receive and respond to both voice load control commands and other voice requests. Steps 510-518 can be the same as described above with respect to the audio control process 500A. Figure 5ASteps 510-518 described in the middle are the same, where the load control device receives the voice input, transmits the voice input to a voice service for interpretation, and then the load control device receives and interprets the response from the voice service. For example, the voice service on the cloud server can process the acoustic data and can determine an action based on the voice processing. Based on the action, the cloud server can send a response to the load control device. For example, the command can be "turn on the kitchen light." The voice service can determine the action "turn on" and the corresponding device for "kitchen light," i.e., a control device located in the kitchen. The cloud server can then transmit the command to the load control device (e.g., to a load control device with an integrated microphone device (i.e., the microphone device that sent the command)) to control the electrical load. At step 519, the load control device can determine whether the response from the voice service includes a load control command. If the load control device determines that the response includes a load control command, then the load control device can directly control the connected electrical load in step 520. For example, the load control command can include a digital code that includes one or more specific digits that the load control device can recognize as a load control command. For example, the code 1100100 can mean turn the light on to 100% intensity. It will be understood that analog control commands can alternatively be used.

[0105] If the load control device determines that the response does not include a load control command (i.e., the response is a stream of analog or digital data that is not recognized as a load control command), then the load control device can play the response received from the voice service at step 522. For example, the load control device can directly play the received response via one or more speakers. The speakers can be integrated with the load control and / or microphone device, or the speakers can be external devices. For example, the load control device or microphone device can receive the acoustic response from the cloud server and can transmit the response to one or more speakers to be played back to the user. For example, if the voice input was a request for the current weather, then the received response can be "it is currently raining," which can be played back by the speakers.

[0106] Alternatively, the system controller can receive the response and determine whether to send the response to the load control device or speakers, etc. Further, the system controller can receive the command (such as a lighting control command, e.g., 1100100) from the cloud server and can translate the command to a lighting control command in a different protocol. For example, the system controller can receive the command and can transmit the command to a lighting control device using the ClearConnect protocol. Other examples are possible.

[0107] Figure 6is a simplified flowchart of an example audio control process 600 that can be performed by a load control device of a load control system (e.g., one of the lighting control devices 120 of the load control system 100). In the audio control process 600, the load control device can be configured to locally process some voice commands and control connected electrical loads without transmitting the voice commands to a voice service in the cloud. This can reduce the latency of responses, and can also allow the load control system to process voice commands when the network (i.e., the cloud server) is down or unavailable. When the load control device receives an unauthenticated voice command for local processing (i.e., the voice command is not a command in a list of recognized commands), the load control device can be configured to analyze the voice command to determine whether the load control device should begin processing the voice command locally (e.g., if the voice command is a voice command that is often received).

[0108] Referring to Figure 6 At step 610, the load control device can detect a wake word, and at step 612, the load control device can receive a voice command. At step 614, the load control device can determine whether the voice command is an authenticated command. An authenticated command can be a voice command that the load control device recognizes, e.g., the authenticated command can be associated with a corresponding action. For example, the authenticated command can be the voice command “turn on the light,” which can be associated in memory with the action of turning on a light. For example, the load control device can compare the received voice command to one or more voice commands stored in memory. If the received voice command matches one of the voice commands stored in memory, then the load control device can recognize the voice command, i.e., the voice command can be an authenticated command. For example, the load control device can correlate the voice command to the voice commands stored in memory, and determine a match based on a percentage of correlation. If the percentage of correlation is above a threshold, then the load control device can determine that the voice command matches the stored voice command. For example, if the threshold is 90% and the voice command correlates to the stored command at 96%, then the load control device can determine that the voice command is an authenticated command. However, if the correlation is less than 90%, then the load control device can determine that the command is not an authenticated command.

[0109] If, at step 614, the received voice command is a verified command (e.g., as described, stored in memory in the load control device), the load control device can control the connected electrical load in response to the received voice command according to the action stored and associated with the verified command. The audio control process 600 can then exit. If, at step 614, the received voice command is not a verified command (i.e., not stored locally), the load control device can transmit the voice command to a voice service in the cloud at step 618 (e.g., directly to the Internet via the router 114 and / or via the system controller 110). The load control device can receive a response from the voice service at step 620 and interpret the response at step 622 to determine a load control command (i.e., action) for controlling the electrical load.

[0110] At step 624, the load control device can determine whether the voice command should be processed locally. For example, if the response from the voice service is a load control command, the load control device can determine whether the received load control command (i.e., action) is already stored in memory. For example, the load control command can be a simple intensity count, such as 00110101, for example, which can indicate that the load control device is to set the intensity to 100%. If one or more instances of the received load control command are already stored in memory, the load control device can compare the current voice command to each stored voice command in memory that is associated with the load control command, as previously described.

[0111] Different criteria can be used to determine whether the voice command should be processed locally. In a first example, if the voice command is not stored in memory, the load control device can store the voice command as a verified command with the load control command in memory at step 626. In this way, the voice command can only need to be interpreted by the voice service a single time before the load control device can determine at step 624 that the voice command with the corresponding load control command should be processed locally. The load control device can then store the voice command as a verified command and associate the corresponding action to be taken based on the load control command at step 626.

[0112] Alternatively, the load control device can require multiple instances of receiving the voice command before storing the voice command as a verified command. According to a second example, the load control device can require three instances of receiving the voice command. (It will be recognized that any number can be used.) Before storing the voice command as a verified command, the load control device can either store multiple instances of the voice command in memory or keep track of the number of times the voice command has been received. For example, when three instances of the voice command have been received and stored in memory, the load control device can then determine that the command should be handled locally, where each voice command has the same corresponding load control command. If the load control device determines at step 624 that the command should be handled locally, then the load control device can store the voice command as a verified command with the load control command in memory at step 626.

[0113] If the voice command does not match any stored voice command or if multiple stored commands are required to create a verified command, then the load control device can determine that the command should not be handled locally. Otherwise, the load control device can store data regarding the received voice command in memory at step 628 for use in determining whether the command should be handled locally in the future. After determining at step 624 whether the received voice command should be handled locally, before the audio control process 600 exits, the load control device can directly control the connected electrical load in response to the received voice command at step 616. Further, if the response is an acoustic response to the voice command (e.g., the user has asked a question and the voice service provides a response), then the load control device can determine that the voice command should not be handled locally (i.e., an internet search service can be required to handle the request), and the load control device can not store the voice command.

[0114] Figure 7 is a simplified flowchart of an example audio control process 700 that can be performed by one or more load control devices of a load control system (e.g., the lighting control devices 120 of the load control system 100). Multiple load control devices of a load control system can be configured to communicate with each other in order to determine how to communicate with a voice service in the cloud and control the respective electrical loads. One or more of the load control devices can detect a wake word at step 710 and receive a voice command at step 712.

[0115] The load control devices can communicate with each other at step 714 in order to determine which of the load control devices should transmit the voice command to the voice service. For example, the load control devices can determine at step 714 which load control device best heard the voice command (e.g., loudest or closest), as previously described for multiple microphone devices. For example, the microphone devices can transmit characteristics of the voice command (e.g., volume, multipath interference (e.g., echo), distance to the source of the sound, etc.) to each other. Each load control device can then compare the characteristic(s) to its own measured characteristic(s). If the load control device determines that its own characteristic(s) are not better than the received characteristic(s), then the load control device can not transmit the voice command to the server. However, if the load control device determines that its own characteristic(s) are better than the received characteristic(s), then the selected load control device can transmit the voice command to the voice service in the cloud at step 717 (e.g., directly to the Internet via the router 109 and / or via the system controller 110). Other examples are possible.

[0116] The selected load control device can receive the response from the voice service at step 718 and interpret the response at step 720 to determine a load control command for controlling the electrical load(s), as described with respect to FIG. 6. The selected load control device can then transmit the load control command to the electrical load(s) at step 722. Other examples are possible. Figure 5A 5B ​Similarly described). The selected load control devices can control their connected electrical loads directly (if needed) in response to the interpreted command at step 722, and / or can transmit the interpreted command to other load control devices at step 724, which can in turn control their loads. According to a first example, the load control devices can transmit the command to all load control devices that heard the voice command. For example, the load control devices can transmit the load control command to each load control device from which the communication was received at step 714. In a second example, the load control command can be a scene command. The load control devices can then transmit the scene command to other load control devices in the load control system. The other load control devices in the load control system can or can not be microphone devices. The other load control devices can determine whether to control their respective electrical loads based on the scene command. For example, the load control devices can transmit a “morning” scene command to one or more load control devices, such as motorized window treatments. The motorized window treatments can receive the scene command and can adjust the respective window coverings based on the scene command, for example, the motorized window treatments can raise the window coverings to fully open, while one or more lighting control devices can turn off their respective lighting loads in response to the “morning” scene. Other examples are possible. After controlling the electrical loads, the audio control process 700 can then exit.

[0117] Figure 8 is a flowchart of another example audio control process 800 that can be performed by one or more load control devices of a load control system, e.g., the lighting control devices 120 of the load control system 100. Using the audio control process 800, each load control device can be configured to determine which load control devices to control depending on whether the received command includes information about a zone to control (e.g., one or more load control devices in a zone). For example, if a user states “Alexa, turn on the kitchen lights,” the voice service can interpret the voice command as “turn on” and the zone as “kitchen.” The cloud server can then translate the voice command and zone into a format recognizable to the load control system. The load control devices can receive the response from the cloud server including the command “turn on” and the zone “kitchen,” and the load control devices can be configured to determine that the desired zone is the kitchen lights. However, if the user simply states “Alexa, turn on the lights,” the load control devices can determine that the desired zone is the load control device that best (e.g., loudest or closest) heard the voice command.

[0118] Reference is made to Figure 8, one or more of the load control devices can detect the wake word at step 810 and receive the voice command at step 812. The load control devices can communicate with each other at step 814 in order to determine which of the load control devices (e.g., the load control device that best heard the voice command) should transmit the voice command to the voice service, as previously described in Figure 7 Other examples are possible. At step 816, the selected load control device can transmit the voice command to the voice service in the cloud (e.g., directly to the Internet via the router 114 and / or via the system controller 110). The selected load control device can receive a response from the voice service at step 818 and interpret the response at step 820 to determine a load control command for controlling the electrical load(s) (e.g., as described in Figure 5A and 5B ).

[0119] At step 822, the selected load control device can determine whether the response command from the voice service includes zone information. For example, the received command can be "turn on" with the "kitchen" zone. If the received command at step 822 includes information indicating the zone to control, then the load control device can control the connected electrical load of that load control device (if needed) at step 824 in response to the interpreted command and can transmit the received command to other load control devices at step 826 before the audio control process 800 exits. For example, if the command from the voice service includes zone information, then each load control device can know via configuration information which zone it controls. The selected load control device can then determine whether the zone control information matches its own zone information and, if so, the selected load control device can control the connected load of that selected load control device at step 824. However, if the zone control information is for a zone controlled by another load control device, then the selected load control device does not control its electrical load. At step 826, the selected load control device can transmit the command including the zone control command to other load control devices in the load control system. For example, the other load control devices can then receive the zone control command and determine whether to control the connected electrical load of those other load control devices based on the zone control command and the specific configuration of those other load control devices.

[0120] Alternatively / in addition, the system controller can determine zone information. For example, the system controller can be configured to know what devices are in which zones based on configuration data established during setup of the load control system. For example, the selected load control device can transmit zone control information and load control commands from the voice service to the system controller. The system controller can determine which load control device(s) correspond to the respective zone control information and can then transmit the load control commands to the one or more load control devices associated with the respective zone control commands. These devices can then control their respective loads accordingly.

[0121] Alternatively, if the received command does not include information about a desired zone to control, the selected load control device (e.g., the load control device that best received the voice command) can interpret the response to determine the load control command. At step 828, the selected load control device can directly control the connected electrical load in response to the interpreted command. The audio control process 800 can exit.

[0122] It will be appreciated that other examples are possible. For example, when the load control command does not include zone information, the selected load control device can still transmit the load control command to one or more other load control devices. For example, the command can be a scene that includes multiple load control devices that adjust respective loads of these multiple load control devices. In this case, the selected load control device can transmit the command to the system controller and / or one or more load control devices that can interpret the load control or scene command. For example, the response from the voice service includes an action that is a scene. The scene can be received by the selected load control device. The selected control device can translate the scene action into load control commands to the load control system. Other load control devices in the load control system can receive the scene command and determine whether to control respective loads of these other load control devices based on whether the load control devices have been programmed to respond to the particular scene during configuration of the load control system. Alternatively, the selected load control device can transmit the scene command and / or the response from the voice service to the system controller. The system controller can then transmit corresponding scene or load control commands to the load control system and / or individual load control devices.

[0123] In another example, while described herein as a load control device, it will be recognized that a standalone microphone device that is not a load control device can be used to receive load control commands from the voice service. Thus, if the selected device is a standalone microphone device that receives load control commands that do not include zone information, the selected microphone device can then transmit the load control commands to the system controller and / or to load control devices in the same room as the selected microphone device. For example, the system controller can determine which load control device(s) are in the same zone or area as the selected microphone device (based on information from room settings obtained during system setup of the load control system and microphone devices), and can transmit the control commands to the respective load control device(s). For example, the selected microphone device can transmit the response received from the voice service to the system controller, and / or can transmit the interpreted response to the system controller. The system controller can receive the interpreted response, or can interpret the received response, and can determine which load control devices are in the same room as the microphone device. The system controller can then transmit one or more load control commands to one or more load control devices that are in the same room as the selected microphone device. Alternatively, the selected microphone device can know which load control devices are in the same room and can transmit the interpreted commands directly to the load control devices within the same room.

[0124] Figure 9is an example audio control process 900 that can be performed by a load control device (e.g., one of the lighting control devices 120) and / or a system controller (e.g., the system controller 150) of a load control system (e.g., the load control system 100). The load control device can include a microphone for receiving audio data (e.g., voice commands and / or sounds). The load control device can be configured to transmit the audio data to the system controller, which can be configured to communicate with a voice service in the cloud to allow for control of an electrical load connected to the load control device. After detecting a wake word at step 910, the load control device can receive a voice command at step 912. The load control device can transmit the voice command to the system controller at step 914, and the system controller can transmit the voice command to the voice service in the cloud at step 916. The system controller can receive a response from the voice service at step 919, and interpret the response at step 920 to determine a load control command for controlling the electrical load. At step 922, the system controller can transmit the interpreted command to the load control device. For example, the system controller can use zone information in the received interpreted command to determine which load control device(s) to send the interpreted command to. For example, the system controller can determine which devices are associated with the received zone, and can control the devices in that zone based on the interpreted command. In another example, the system controller can transmit the interpreted command to the load control device from which the system controller received the voice command and / or the load control device that best heard the voice command, as previously described. Alternatively, if the interpreted command is a scene command, then the system controller can identify the scene command and transmit the load control command to the corresponding load control devices to create the desired scene. At step 924, the load control device(s) can then directly control the connected electrical load in response to the interpreted command received from the voice service. The audio control process 900 can then exit.

[0125] Figure 10Ais a flowchart of an example audio control process 1000 that can be performed by a load control device (e.g., one of the lighting control devices 120) and / or a system controller (e.g., the system controller 150) of a load control system (e.g., the load control system 100). The load control device can include a microphone for receiving audio data (e.g., voice commands and / or sounds). The load control device can be configured to transmit the audio data to the system controller, which can be configured to communicate with voice services in the cloud to allow for control of electrical loads connected to the load control device. After detecting a wake word at step 1010, the load control device can receive a voice command at step 1012. The load control device can transmit the voice command to the system controller at step 1014. Then, at step 1016, the system controller can transmit the voice command received from the load control device to one or more voice services in the cloud. It will be understood that any or multiple voice services can be used, or the voice services can be selectable, as previously described. After the voice services have processed the audio data of the voice command, the system controller can receive a response from the voice services at step 1018. At step 1020, the system controller can interpret the response to determine a command for controlling an electrical load.

[0126] At step 1022, if the received command includes information indicating a zone to be controlled, then the system controller can transmit the interpreted command to the corresponding load control device(s) for that zone. For example, the system controller can maintain a configuration database associating load control devices with zones, which can be used to determine which load control devices to transmit the interpreted command to for the corresponding zone. The configuration database can be created during a configuration state of the load control system, for example. Additionally and / or alternatively, the system controller can also have a configuration database for one or more scenes, which can also be configured during a configuration state or a setup state of the load control system, where each scene is associated with a corresponding lighting control. Then, before the audio control process 1000 exits, the load control device can directly control the connected electrical load in response to the interpreted command at step 1024.

[0127] If the command received at step 1022 does not include information indicating a zone to control and the area in which the load control device that best heard the audio command is located is occupied at step 1026, the system controller can transmit the interpreted command to the load control device that best heard the audio command at step 1028. The load control device that best heard the audio command can be associated with a zone. The system controller can additionally or alternatively determine which load control devices are included in the same zone as the load control device that best heard the audio command. The system controller can then transmit load control commands to one or more load control devices in the same zone as the load control device that best heard the audio command based on the interpreted command. For example, additional load control devices that can not be microphone devices can be located in the same zone as the load control device that best heard the audio command. In this way, the system controller can transmit the command to all of the load control devices in the zone. The audio control process 1000 can then exit. If the area in which the load control device that best heard the audio command is located is not occupied at step 1026, the system controller can determine which load control device in the occupied area best heard the audio command at step 1030. The system controller can then transmit the interpreted command to that load control device (or, as previously described, load control devices in the same zone) at step 1032 before the audio control process 1000 exits.

[0128] Figure 10B is a message flow diagram similar to the flow diagram shown in Figure 10A The message flow diagram can be a flow diagram similar to the flow diagram shown in

[0129] For example, one or more occupancy sensors 1052 can transmit occupancy information 1056 to the system controller 1046. The system controller 1046 can use the occupancy information 1056 to maintain an occupancy status for one or more areas associated with the one or more occupancy sensors 1052. The system controller 1046 can also control one or more load control devices based on the occupancy information.

[0130] When the user speaks the wake-up word, one or more load control devices 1040, 1042 can hear the wake-up word and record the voice input 1058 following the wake-up word. The one or more load control devices can transmit the voice input 1058 to the system controller 1046 for processing. The system controller can process the voice input 1058 locally. For example, the system controller can process the voice input 1058 to determine which voice input has better audio quality. As previously described, audio quality can be based on signal-to-noise ratio, volume, multipath reflections, etc. The system controller 1046 can then transmit the voice input 1060 having better audio quality to the cloud service 1050 for voice processing.

[0131] The cloud service 1050 can process the voice input 1060 and can return the processed output 1062 to the system controller. The processed output can be a load control command. For example, the processed output can be an instruction to turn on or off a specific lighting load or zone. For example, if the voice input is "turn on the kitchen lights," then the cloud service can digitally instruct the system controller 1046 to turn on the lighting loads in the kitchen. The system controller can then translate the instruction into a command for the one or more load control devices. Alternatively, if the voice input 1060 is a request that is not associated with controlling a load, then the cloud service 1050 can respond to the request with an acoustic data in the form of an answer. The system controller 1046 can receive the acoustic data and transmit it directly to the one or more load control devices, or to the one or more speakers.

[0132] The system controller 1046 can determine to transmit the command or acoustic data to the one or more load control devices based on one or more parameters. That is, the system controller can attempt to send the command or acoustic data to a load control device located in the area where the user made the request. For example, the system controller 1046 can transmit the command to the load control device that best heard the voice input. For example, if the voice input 1060 transmitted to the cloud service 1050 was received from the load control device 1040 (and the redundant voice input 1058 from the load control device 1042 was ignored), then the system controller can return the command 1064 to the load control device 1040 and / or any devices in the same zone as the load control device 1040. In this way, the load control device 1040 can respond to the command.

[0133] Additionally or alternatively, the system controller may use occupancy information 1056 to transmit commands to the load control devices. For example, if both areas where load control devices 1040 and 1042 are located are occupied, the system controller may transmit commands to both load control devices. However, for example, if only one area is occupied, the system controller may transmit commands only to the occupied area. For example, the system controller may transmit command 1064 only to load control device 1042. Although this document describes the system controller transmitting commands to load control devices in occupied areas, multiple load control devices may exist in an occupied area. For example, the example of a single load control device in an area is for illustrative purposes only, and it will be appreciated that the system controller may transmit commands to any or all load control devices in an occupied area.

[0134] Although this method has been described herein as using a system controller, it will be understood that systems can be designed without a system controller. For example, load control devices 1040, 1042 can communicate wirelessly or via a wired connection (e.g., power line communication) to receive occupancy information 1056 and / or determine which load control device best hears the voice input, thus acting as a system controller among them. Additionally or alternatively, the load control devices themselves may include occupancy sensors. Other variations are possible.

[0135] Figure 11 This is a simplified block diagram of an example control device 1100, which can be configured as a microphone device and can be deployed as, for example... Figure 1 The load control system 110 shown includes a wall-mounted keypad 176. The control device 1100 can be powered by a power supply 1112, which can receive power from a power connection 1116, which can receive power from an external AC or DC power source. The power supply 1112 can provide an output DC power voltage V. CC, for powering the control circuit 1102. The control device 1102 can include one or more general purpose processors, special purpose processors, conventional processors, digital signal processors (DSPs), microprocessors, microcontrollers, integrated circuits, programmable logic devices (PLDs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or any suitable controller or processing device or the like (hereinafter collectively referred to as processor(s) or control circuit(s) 1002). The control circuit 1102 can be configured to execute one or more software-based applications, including instructions that, when executed by the control circuit, can configure the control circuit to perform signal coding, data processing, power control, input / output processing, or any other functionality enabling the control device 1100 to perform as described herein. It will be appreciated that the features and processes described herein can also and / or alternatively be provided by firmware and / or hardware, in addition to / instead of software-based instructions.

[0136] The control circuit 1102 can store and / or retrieve information in and from a memory 1118. The memory 1118 can also store software-based instructions for execution by the control circuit 1102, and can further provide an execution space as the control circuit executes the instructions. The memory 1118 can be implemented as an external integrated circuit (IC), or as an internal circuit of the control circuit 1102. The memory 1118 can include volatile and non-volatile memory modules / devices, and can be non-removable memory modules / devices and / or removable memory modules / devices. Non-removable memory can include random access memory (RAM), read-only memory (ROM), hard disks, or any other type of non-removable memory storage. Removable memory can include subscriber identity module (SIM) cards, memory sticks, memory cards, or any other type of removable memory. The control device 1100 can further include one or more communication circuits 1106 for transmitting and / or receiving wireless and / or wired communications. The control device 1100 can include a user interface 1120 coupled to the control circuit 1102 for receiving user input and displaying feedback information.

[0137] The control circuit 1102 can be adapted to receive audio signals from the input microphone 1104 and determine how to process the audio signals, as described herein. The control circuit 1102 can be configured to detect a wake-up word spoken by a user of the control device 1100, e.g., by comparing audio data to one or more predetermined wake-up words stored in memory 1118. For example, the control circuit can send or receive commands related to controlling an electrical load, such as a lighting load, or for sending acoustic data to a system controller or a remote network server for further processing of the audio data. The remote network server can be a cloud server. The control circuit can communicate with the remote network via one or more intermediary devices, such as a system controller and / or a router device. The communication protocol can include one or more of the following: Wi-Fi, HaLow, ZigBee, Bluetooth, Z-Wave, ClearConnect, or other similar protocols.

[0138] The input microphone 1104 can be a digital or analog MEM device, which can be available at low cost and small package size. However, for example, an electret condenser, magnetic, or other wideband acoustic input device available at suitably small package size can be used instead. The microphone device can include multiple input microphones, which can be physically spaced apart from one another. The multiple input microphones can allow for improved ambient noise rejection, and can also allow for acoustic beamforming or beamsteering, whereby the microphone device is sensitive to input sound in a direction.

[0139] The control device 1100 can also include a speaker 1111 coupled to the control circuit 1102 for allowing communication with a user. The speaker can allow the microphone device to audibly communicate with a user, or can be used to play music. The control circuit 1102 can be configured to cause the speaker 1111 to generate an audio signal, e.g., in response to data received from a voice service in the cloud. For example, the data received from the voice service in the cloud can indicate an answer to a question asked by a user of the control device 1100, and the control circuit 1102 can be configured to cause the speaker 1111 to broadcast the answer for use.

[0140] The control device 1100 can include additional circuitry not shown here, including but not limited to: load control, passive infrared occupancy sensing, microwave occupancy sensing, ambient light sensing, clock or time-of-day tracking, etc.

[0141] Figure 12 is a simplified block diagram of an example load control device 1200, e.g., which can be deployed as Figure 1The load control device 1200 can include a hot terminal H, which can be adapted to be coupled to an AC power source 1202. The load control device 1200 can include a dimmed hot terminal DH, which can be adapted to be coupled to an electrical load, such as a lighting load 1204. The load control device 1200 can include a controllably conductive device 1210 coupled in a series electrical connection between the AC power source 1202 and the lighting load 1204. The controllably conductive device 1210 can control power delivered to the lighting load. The controllably conductive device 1210 can include a suitable type of bidirectional semiconductor switch, such as, for example, a triac, a field effect transistor (FET) in a rectifier bridge, two FETs in anti- series connection, or one or more insulated gate bipolar transistors (IGBT). An air gap switch 1229 can be coupled in series with the controllably conductive device 1210. The air gap switch 1229 can open and close in response to actuation of an air gap actuator. When the air gap switch 1229 is closed, the controllably conductive device 1210 is operable to conduct current to the load. When the air gap switch 1229 is open, the lighting load 1204 is disconnected from the AC power source 1202. The load control device 1200 can include a control circuit 1214. The control device 1102 can include one or more general purpose processors, special purpose processors, conventional processors, digital signal processors (DSPs), microprocessors, microcontrollers, integrated circuits, programmable logic devices (PLDs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or any suitable controller or processing device, etc. (hereinafter collectively referred to as processor(s) or control circuit(s) 1214). The control circuit 1214 can be configured to execute one or more software-based applications that include instructions that, when executed by the control circuit, can configure the control circuit to perform signal coding, data processing, power control, input / output processing, or any other functionality enabling the load control device 1200 to perform any other functions as described herein. It will be recognized that the features and processes described herein can also and / or alternatively be provided by firmware and / or hardware, in addition to / instead of software-based instructions. The control circuit 1214 can store and / or retrieve information from a memory 1220. The memory 1220 can also store software-based instructions for execution by the control circuit 1214 and can further provide an execution space as the control circuit executes the instructions. The memory 1220 can be implemented as an external integrated circuit (IC), or as an internal circuit of the control circuit 1214. The memory 1220 can include volatile and non-volatile memory modules / devices, and can be non-removable memory modules / devices and / or removable memory modules / devices.The non-removable memory can include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of non-removable memory storage. The removable memory can include a subscriber identity module (SIM) card, a memory stick, a memory card, or any other type of removable memory.

[0142] The control circuit 1214 can be operably coupled to a control input of the controllably conductive device 1210, e.g., via a gate drive circuit 1212. The control circuit 1214 can be used to render the controllably conductive device 1210 conductive or non-conductive, e.g., to control the amount of power delivered to the lighting load 1204.

[0143] The load control device 1200 can include a user interface 1216 coupled to the control circuit 1214 for receiving user input and displaying feedback information. The control circuit 1214 can receive input in response to actuation of buttons of the user interface 1216 and can control the controllably conductive device 1210 to adjust the intensity of the lighting load 1204 in response to the actuation. The control circuit 1214 can be configured to illuminate a visual indicator (e.g., an LED) of the user interface 1216 to provide feedback of the current intensity of the lighting load 1204.

[0144] The control circuit 1214 can receive a control signal from a zero-crossing detector 1218 that is representative of zero-crossings of the AC mains voltage of the AC power source 1202. The control circuit 1214 can be operable to use a phase control dimming technique to render the controllably conductive device 1210 conductive and / or non-conductive at a predetermined time relative to the zero-crossings of the AC waveform. Examples of dimmers are described in greater detail in commonly-assigned U.S. Patent No. 7,242,150, entitled “Dimmer Having a Power Supply Monitoring Circuit,” issued July 10, 2007, U.S. Patent No. 7,546,473, entitled “Dimmer having a microprocessor-controlled power supply,” issued June 9, 2009, and U.S. Patent No. 8,664,881, entitled “Two-wire dimmer switch for low-power loads,” issued March 4, 2014, the entire disclosures of which are incorporated by reference herein.

[0145] The load control device 1200 can include a power supply 1222. The power supply 1222 can generate a direct current (DC) supply voltage V CCfor powering control circuit 1214 and other low voltage circuitry of load control device 1200. Power supply 1222 can be coupled in parallel with controllably conductive device 1210. Power supply 1222 can be operable to conduct a charging current through lighting load 1204 to generate a DC supply voltage V CC .

[0146] Load control device 1200 can include first and second wireless communication circuits 1224, 1226, although one or more of the communication circuits 1224, 1226 can be configured to communicate over a wired network. Each of the wireless communication circuits 1224, 1226 can include, for example, a radio frequency (RF) transceiver coupled to an antenna for transmitting and / or receiving RF signals. The wireless communication circuits 1224, 1226 can each also include an RF transmitter for transmitting RF signals, an RF receiver for receiving RF signals, or an infrared (IR) transmitter and / or receiver for transmitting and / or receiving IR signals.

[0147] Control circuit 1214 can be coupled to first and second wireless communication circuits 1224, 1226 for transmitting and / or receiving digital messages via RF signals. Control circuit 1214 can be configured to communicate RF signals (e.g., wireless communication signals 108) via first wireless communication circuit 1224 using a first wireless protocol (e.g., a proprietary communication protocol, such as the Clear Connect® protocol). Control circuit 1214 can be configured to receive load control commands in digital messages communicated using first wireless communication circuit 1224 for controlling lighting load 1204, and can be configured to control controllably conductive device 1210 to adjust the intensity of lighting load 1204 in response to digital messages received via first wireless communication circuit 1224. Control circuit 1214 can be configured to transmit feedback information (e.g., regarding the amount of power being delivered to lighting load 1204) via digital messages communicated using first wireless communication circuit 1224. Control circuit 1214 can be configured to communicate RF signals (e.g., wireless communication signals 109) via second wireless communication circuit 1226 using a second wireless protocol (e.g., a standard communication protocol, such as a Wi-Fi or Bluetooth protocol). For example, control circuit 1214 can be configured to communicate audio data (e.g., voice commands) in digital messages communicated using second wireless communication circuit 1226 (as will be described in greater detail below).

[0148] The load control device 1200 can also include a microphone 1230 coupled to the control circuit 1214 for receiving audio data (e.g., voice commands). The control circuit 1214 can be configured to detect a wake word spoken by a user of the load control device 1200, e.g., by comparing the audio data to one or more predetermined wake words stored in the memory 1220. The control circuit 1214 can be configured to transmit the audio data in a digital message communicated using the second wireless communication circuit 1226 (e.g., to the system controller 120). The control circuit 1214 can also be configured to transmit the audio data directly to a voice service in the cloud using the second wireless communication circuit 1226 (e.g., via the router 124). The control circuit 1214 can be configured to receive load control commands via digital messages received over the first or second wireless communication circuits 1226 in response to the transmitted audio data.

[0149] In addition, the control circuit 1214 can be configured to compare the received audio data to one or more voice commands stored in the memory 1220 and control the controllably conductive device 1210 to adjust the intensity of the lighting load 1204 without transmitting the audio data to a voice service in the cloud.

[0150] A user can open an air gap switch 1229 such that the control circuit 1214 cannot receive audio data from the microphone 1230.

[0151] The load control device 1200 can also include a speaker 1232 coupled to the control circuit 1214. The control circuit 1214 can be configured to cause the speaker 1232 to generate an audio signal, e.g., in response to data received from a voice service in the cloud. For example, the data received from the voice service in the cloud can indicate an answer to a question asked by a user of the load control device 1200, and the control circuit 1214 can be configured to cause the speaker 1232 to broadcast the answer to the user.

[0152] In addition to the embodiments described herein, a microphone device can also provide an indication to a user when the microphone device is transmitting data to a cloud server. For example, voice integrated devices typically use one or more light emitting diodes (LEDs) that light up on the device when a user speaks a wake word. However, because a microphone device can be placed anywhere in a room (i.e., on a wall, ceiling, etc.), the LED indicator on the device can not be sufficient to alert a user that the device is listening.

[0153] To alert a user that one or more microphone devices are listening in response to a wake word, a lighting control device can cause a connected lighting load to flicker in response to detecting a keyword (e.g., a wake word), and continue causing the connected lighting load to flicker while the microphone device is recording audio data. For example, the microphone can be integrated with the load control device, or the microphone device can be separate from the load control device, and can transmit a command to the load control device or system controller to cause the lighting load to flicker.

[0154] Figure 13 is a block diagram illustrating an example system controller 1300, such as the system controller 110 described herein. The system controller 1300 can include one or more general processors, special purpose processors, conventional processors, digital signal processors (DSPs), microprocessors, microcontrollers, integrated circuits, programmable logic devices (PLDs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or any suitable controller or processing device or the like (hereinafter collectively referred to as processor(s) or control circuit(s) 1314). The control circuit 1314 can be configured to execute one or more software-based applications, including instructions that, when executed by the control circuit, can configure the control circuit to perform signal coding, data processing, power control, input / output processing, or any other functionality enabling the system controller 1300 to perform as described herein. It will be appreciated that the features and processes described herein can also be provided, and / or alternatively implemented by, firmware and / or hardware, in addition to, or alternatively to, software-based instructions. The control circuit 1314 can store and / or retrieve information from the memory 1320. The memory 1320 can also store software-based instructions for execution by the control circuit 1314 and can further provide an execution space when the control circuit executes the instructions. The memory 1320 can be implemented as an external integrated circuit (IC), or as internal circuitry of the control circuit 1314. The memory 1320 can include volatile and nonvolatile memory modules / devices, and can be non-removable memory modules / devices and / or removable memory modules / devices. Non-removable memory can include random access memory (RAM), read only memory (ROM), hard disks, or any other type of non-removable memory storage. Removable memory can include a subscriber identity module (SIM) card, a memory stick, a memory card, or any other type of removable memory.

[0155] The system controller 1300 can include a communication circuit 1306 for transmitting and / or receiving information. The communication circuit 1306 can perform wireless and / or wired communication. The system controller 1300 can also or instead include a communication circuit 1308 for transmitting and / or receiving information. The communication circuit 1308 can perform wireless and / or wired communication. The communication circuit 1306 and 1308 can be in communication with the control circuit 1302. The communication circuit 1306 and 1308 can include an RF transceiver or other communication module capable of performing wireless communication via an antenna. The communication circuit 1306 and 1308 can be capable of performing communication via the same communication channel or different communication channels. For example, the communication circuit 1306 can be capable of communicating via a wireless communication channel (e.g., BLUETOOTH®, ZIGBEE®, near field communication (NFC), cellular, etc.), and the communication circuit 1308 can be capable of communicating via another wireless communication channel (e.g., BLUETOOTH®, ZIGBEE®, or a proprietary communication channel such as CLEAR CONNECT TM ).

[0156] The control circuit 1302 can be in communication with an LED indicator 1313 for providing indications to a user. The control circuit 1302 can be in communication with an actuator 1314 (e.g., one or more buttons) that can be actuated by a user to communicate user selections to the control circuit 1302. For example, the actuator 1314 can be actuated to place the control circuit 1302 in an association mode and / or to communicate an association message from the system controller 1300.

[0157] Each module within the system controller 1300 can be powered by a power supply 1310. For example, the power supply 1310 can include an AC power supply or a DC power supply. The power supply 1310 can generate a power supply voltage V CC for powering the modules within the system controller 1300.

[0158] Figure 14A is an example method 1400A that can be performed by a control circuit of a load control device in response to hearing a wake-up word. At step 1410, the load control device can detect a wake-up word. For example, a user can speak a wake-up word in proximity to the load control device, which can be received by one or more microphones of the load control device. The control circuit of the load control device can recognize the received audio data as the wake-up word. Upon recognizing the wake-up word, the load control device can cause a connected lighting load to flash at step 1414. For example, the load control device can turn the lighting load on and off at a duty cycle that is readily visible to the human eye. For example, the load control device can turn the lighting load on and off every two seconds.

[0159] At step 1416, the audio device can receive the voice input after the wake word and can transmit the voice input to a voice service in the cloud at step 1420. If the user does not want the device to transmit the audio data, the flashing light can alert the user to put the load control device in a mute mode. The mute mode can prevent the load control (or audio device) from recording acoustic data (i.e., recording the voice input). The load control device can stop causing the load to flash at 1422.

[0160] Figure 14B is an example method 1400B similar to 1400A that can be performed by a microphone device that is not integrated with a load control device. The microphone device can detect the wake word at step 1430. At 1432, the microphone device can transmit a command indicating that the wake word has been detected. For example, the microphone device can transmit the command to a control device and / or a system controller.

[0161] At step 1440, the load control device can receive the command. According to a first example, the load control device can receive the command from the microphone device. That is, the microphone device can transmit a command indicating that the wake word has been detected. Based on system configuration information determined during setup of the load control system, the load control devices can know whether they are in the same room as the microphone device. For example, the microphone device can transmit the command along with a location or device identifier, thereby identifying the specific microphone device that transmitted the command or the location. The load control device can only respond to the command if it is in the same room as the microphone device. Alternatively, according to a second example, the microphone device can transmit the wake word to a system controller, or can transmit a command to the system controller indicating that the wake word has been heard. The system controller can then transmit a command to load control devices that are in the same room and / or zone as the microphone device that heard the wake word. The command can indicate to the control devices that the wake word has been detected, or the command can be a load control command instructing the control devices to cause their connected loads to flash. The control devices can receive the command from the system controller.

[0162] In response to receiving the command, the control device can cause the connected lighting load to flash at step 1436. For example, the control device and / or the system controller can know which room the microphone device is located in, such that only the load control devices of the room in which the microphone device is located can respond. At step 1442, the microphone device can receive the voice input after the wake word. At step 1444, the microphone device can transmit the voice input to a voice service in the cloud. At 1448, the microphone device can transmit a command to the load control devices (either directly or through the system controller) to instruct the load control devices to stop causing the electrical load to flash. At step 1450, the load control devices can receive the command and stop causing the connected lighting load to flash.

[0163] While described herein as causing a lighting load to flash, it will be appreciated that other types of warnings are possible. In another example, the load control device can increase and decrease the intensity of a connected lighting load. Alternatively, the lighting control device can change the color of the light. The load control device can cause the lighting load to flash once or repeatedly to indicate to the user that the wake word was detected and the load control device will begin transmitting audio data. According to another embodiment, the load control device can control an electrical load that is not a lighting load, such as a fan, a speaker, etc., and can actuate the load to turn on and off to alert the user.

[0164] In another example, when the microphone device is in a mute mode, the microphone device can still respond to the wake word, but can not process audio input after the wake word. For example, the microphone device can receive the wake word while in the mute mode and can cause one or more lighting loads to flash (according to the method of Figure 14A 、 Figure 14B as previously described) to indicate to the user that the device is in the mute mode.

[0165] It will be understood that while embodiments described herein can be with respect to a load control device having a microphone, the microphone can be separate from the load control device. Further, the system controller can also be a load control device and / or can have one or more microphones.

[0166] In addition to what has been described herein, the described method and system can also be implemented in computer program(s), software, or firmware-based instructions embodied in one or more computer readable media for execution by control circuit(s) to perform the operations thereof. Examples of computer readable media include electronic signals (optical, electrical or the like) and tangible, non-transitory computer readable storage medium. Examples of tangible, non-transitory computer readable storage medium include, but are not limited to, read-only memory (ROM), random access memory (RAM), removable disk, and optical media, such as CD-ROM disks and digital versatile disks (DVDs).

[0167] While the present disclosure has been described in some embodiments and generally associated with methods, alterations and permutations of embodiments and methods will be apparent to those skilled in the art. Thus, the above-described above description of example embodiments does not limit at times the present disclosure. Other changes, modifications, and variations can be apparent to those skilled in the art.

Claims

1. A system comprising: a microphone device adapted to receive acoustic sound from a space and transmit acoustic data; a load control device for controlling an electrical load, the load control device configured to transmit a notification based on controlling the electrical load; a server configured to: receive acoustic data; receive a notification from a load control device; record the acoustic data, the notification, a time of day that the acoustic data and the notification were received, and a current day of the week; determine an amount of time between receiving the acoustic data and the notification from the load control device; based on the determination, compare the recorded acoustic data and the notification to one or more previous records; based on the comparison of the recorded acoustic data and the notification to the one or more previous records associated with the time of day and the current day of the week, determine a pattern that associates the time of day and the current day of the week with a corresponding action; based on the pattern, associate the corresponding action with the acoustic data, wherein the corresponding action is associated with the acoustic data based on the time of day and the current day of the week that conform to the pattern, wherein the corresponding action generates the notification from the load control device; and in response to receiving subsequent acoustic data after the association based on the pattern, transmit a control command to perform the corresponding action. The microphone device communicates with the server via Wi-Fi.

2. The system of claim 1, wherein, The microphone device communicates with the server through a system controller.

3. The system of claim 2, wherein, The microphone device and the load control device are integral.

4. The system of claim 3, wherein, The load control device is a dimmer.

5. The system of claim 3, wherein, The server is a cloud service.

6. The system of claim 1, wherein, The pattern includes a daytime pattern.

7. The system of claim 1, wherein, The pattern includes a nighttime pattern.

8. The system of claim 1, wherein, 9. A method comprising: receiving acoustic data indicative of acoustic sound in a space; receiving a notification indicative of control of an electrical load; recording the acoustic data, notification, a time of day that the acoustic data and the notification were received, and a current day of the week; determining an amount of time between receiving the acoustic data and receiving the notification; based on the determination, comparing the recorded acoustic data and the notification to one or more previous records; based on the comparison of the recorded acoustic data and the notification to the one or more previous records associated with the time of day and the current day of the week, determining a pattern that associates the time of day and the current day of the week with a corresponding action; based on the pattern, associating the corresponding action with the acoustic data, wherein the corresponding action is associated with the acoustic data based on the time of day and the current day of the week that conform to the pattern, wherein the corresponding action generates the notification; and in response to receiving subsequent acoustic data after the association based on the pattern, transmitting a command to perform the corresponding action. The pattern includes a daytime pattern.

10. The method of claim 9, wherein, The pattern includes a nighttime pattern.

11. The method of claim 9, wherein, ​ 12. At least one computer-readable storage medium comprising executable instructions to configure at least one processor to: receive acoustic data indicative of an acoustic sound in a space; receive a notification indicative of a control of an electrical load; record the acoustic data, the notification, a time of day at which the acoustic data and the notification were received, and a current day of the week; determine an amount of time between the receipt of the acoustic data and the receipt of the notification; based on the determination, compare the recorded acoustic data and the notification to one or more previous recordings; based on the comparison of the recorded acoustic data and the notification to the one or more previous recordings associated with the time of day and the current day of the week, determine a pattern that associates the time of day and the current day of the week with a corresponding action; based on the pattern, associate the corresponding action with the acoustic data, wherein the corresponding action is associated with the acoustic data based on the time of day and the current day of the week that conform to the pattern, wherein the corresponding action generates the notification; and in response to receiving subsequent acoustic data after the association based on the pattern, transmit a command to perform the corresponding action.

13. The at least one computer readable storage medium of claim 12, wherein, The pattern includes a daytime pattern.

14. The at least one computer readable storage medium of claim 12, wherein, The pattern includes a nighttime pattern.

Images