Acoustic listening area mapping and frequency correction

Abstract

A smart speaker device for acoustic listening area mapping and frequency correction includes: a non-transitory storage device configured to maintain a listening area response map indicating filter settings corresponding to each of a plurality of positions within a listening area; a microphone array; a loudspeaker; and a controller. The controller is programmed to execute a frequency correction application to: identify a current position of a mobile device in the listening area based on ultrasonic audio received from the microphone array from the mobile device, access the listening area response map to retrieve the filter settings corresponding to the current position, and apply the filter settings to an audio stream to be output to the loudspeaker to correct the frequency response of the loudspeaker at the current position of the mobile device.

Description

Technical Field

[0001] Aspects of the present disclosure generally relate to acoustic listening area mapping and frequency correction. Background Art

[0002] Due to room modes and other acoustic effects, the frequency response of a speaker in a room or other listening area can vary greatly as the listener moves around. These deviations in frequency response can result in large differences in the perceived balance of the speakers and loud resonances at various frequencies. Summary of the invention

[0003] In one or more illustrative examples, a smart speaker device for acoustic listening area mapping and frequency correction includes: a non-transitory storage device configured to maintain a listening area response map indicating filter settings corresponding to each of a plurality of locations within a listening area; a microphone array; a loudspeaker; and a controller. The controller is programmed to execute a frequency correction application to: identify a current location of a mobile device in the listening area based on ultrasonic audio received from the microphone array from a mobile device, access the listening area response map to retrieve the filter settings corresponding to the current location, and apply the filter settings to an audio stream to be output to the loudspeaker to correct the frequency response of the loudspeaker at the current location of the mobile device.

[0004] In one or more illustrative embodiments, a smart speaker device for acoustic listening area mapping and frequency correction includes: a non-transitory storage device configured to maintain a listening area response map indicating filter settings corresponding to each of a plurality of locations within a listening area; a microphone array; a loudspeaker; and a controller. The controller is programmed to execute a frequency correction application to: identify a current location of a mobile device in the listening area based on ultrasonic audio received from the mobile device to the microphone array, output frequency test audio from the loudspeaker for receipt by the mobile device, receive information indicating a room response at the current location from the mobile device, generate a room correction for the current location based on the information indicating the room response, the room correction indicating filter settings for the current location, and update the listening area response map to indicate the filter settings as corresponding to the current location.

[0005] In one or more illustrative embodiments, a method for acoustic listening area mapping and frequency correction includes: identifying a current position of a mobile device in a listening area based on ultrasonic audio received from the mobile device to a microphone array of a smart speaker device; accessing a listening area response map stored to a memory of the smart speaker device to retrieve a filter setting corresponding to the current position, the listening area response map indicating a filter setting corresponding to each of a plurality of positions within the listening area; and applying the filter setting to an audio stream to be output to a loudspeaker of the smart speaker device to correct a frequency response of the loudspeaker at the current position of the mobile device. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A system including a smart speaker and a mobile device is shown, the system being configured for acoustic listening area mapping and frequency correction;

[0007] Figure 2 An exemplary process for operating a mobile device in generating a listening area response map is shown;

[0008] Figure 3 An exemplary process for operating a smart speaker when generating a listening area response map is shown;

[0009] Figure 4 An exemplary process for operating a mobile device to send location updates to a smart speaker is shown;

[0010] Figure 5 An exemplary process for operating a smart speaker to filter audio output based on the location of a mobile device is shown;

[0011] Figure 6 An exemplary diagram showing a smart speaker filtering audio output for a mobile device at a first location in a listening area;

[0012] Figure 7 An exemplary diagram showing a smart speaker filtering audio output for a mobile device at a second location in a listening area; and

[0013] Figure 8 An exemplary diagram of a smart speaker filtering audio output for multiple mobile devices is shown. DETAILED DESCRIPTION

[0014] As required, detailed embodiments of the present invention are disclosed herein; however, it should be understood that the disclosed embodiments are merely examples of the present invention that may be embodied in different and alternative forms. The drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show details of particular components. Therefore, the specific structural details and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to employ the present invention in different ways.

[0015] Cell phones are capable of producing audio frequencies in the ultrasonic region. It turns out that small children are known to use specialized ultrasonic ringtones that adults cannot hear. Smart speakers can utilize microphone arrays to better locate users and adaptively perform beamforming to increase signal-to-noise ratio for voice recognition. These arrays can be used to locate people continuously (e.g., not just when they are speaking) via ultrasonic waves.

[0016] An application installed on a user's phone or other mobile device can be programmed to cause the device to emit short ultrasonic pulses at short intervals. The application installed on the smart speaker can then monitor these signals and determine the user's position via triangulation using a microphone array. Ultrasonic signals may be very suitable for accurately determining arrival times due to their short wavelength. In cases where the ultrasonic waves are blocked by objects and / or the user's own body, the smart speaker may default to a general (unspecified) position equalization. In order to avoid the audibility of the ultrasonic signal, the pulses emitted may be short and only emitted when music or other audio is playing (to mask the sound). In addition, pulses can be emitted in response to detected movement of the mobile device, so if the position has not changed, positioning sound may not be required.

[0017] During the setup of the smart speaker in the listening area, the application installed to the smart speaker can be programmed to cause the smart speaker to emit a low-frequency test signal using one or more loudspeakers of the smart speaker device. When the user moves the mobile device to various positions that the user may occupy in the listening area, the connected measurement application on the mobile device measures the low-frequency response of the speaker in the listening area. At the same time, the above-mentioned triangulation method can be used to locate the user and create a "map" of the listening area, that is, the low-frequency response of each position in the listening area (or at least each position where the user may be). As a possible optimization, during the learning phase, the user can spend more time in the position where he / she is more likely to be, thereby weighting the general solution to make it a better compromise. Once the learning is complete, the smart speaker can calculate the corresponding correction map to produce an optimized low-frequency response in all positions in the listening area. The smart speaker can also calculate a weighted average of the most likely positions and use this weighted average to make the best possible correction when the smart speaker is uncertain about the user's position.

[0018] At run time, the ultrasonic triangulation component operates, allowing the smart speaker to know the user's current location in the listening area. Using the previously generated listening area correction map from the learning phase, the smart speaker can determine the best correction to apply. This filter can be applied in real time to whatever the user is listening to on the smart speaker. If the person moves, the filter can be updated, and the updates can be performed stepwise to avoid detection. In cases where triangulation may not work or produce uncertain results due to occlusion of the source or other reasons, the listening area correction defaults to a universal solution based on measurements at all locations. Therefore, optimization of the smart speaker frequency response can be performed for the user to allow optimized and constant sound as the user moves around the listening area, without adding additional hardware to the smart speaker or mobile device.

[0019] Figure 1 A system 100 including a smart speaker 102 and a mobile device 126 is shown, the system being configured for acoustic listening area mapping and frequency correction. The smart speaker 102 receives audio through a microphone array 104 or other audio input, and passes the audio through an analog / digital (A / D) converter 106 to be recognized or otherwise processed by an audio processor 108. The audio processor 108 also generates an audio output, which can be passed through a digital / analog (D / A) converter 112 and an amplifier 114 for reproduction by one or more loudspeakers 116 of the smart speaker 102. The smart speaker 102 also includes a controller 118 connected to the audio processor 108, the controller being configured to maintain a listening area response map 152 and execute a frequency correction application 158. Based on the input audio of the audio processor 108, the controller 118 in the learning mode uses the loudspeaker 116 to play the frequency test audio 154 to be received by the mobile device 126, and creates an equalized listening area response map 152 for the corresponding position in response to the results of the frequency test included in the wireless signal received from the mobile device 126 to the wireless transceiver 124 of the smart speaker 102. In the playback mode, the controller 118 determines the current position of the user in response to receiving the high frequency audio output 156 received from the mobile device 126, and directs the audio processor 108 to filter the audio signal being played back according to the predetermined equalization setting for the current position identified from the listening area response map 152.

[0020] The mobile device 126 receives audio through a microphone 128 of the mobile device 126 and passes the audio through an A / D converter 130 for recognition or other processing by an audio processor 134. The audio processor 134 also generates an audio output that can be passed through a D / A converter 136 and an amplifier 138 for reproduction by one or more loudspeakers 140 of the mobile device 126. The mobile device 126 also includes a controller 142 connected to the audio processor 134, the controller being configured to execute a frequency correction application 158 to determine a listening area response based on the frequency test audio 154, providing the results of the frequency test in a wireless signal provided by the wireless transceiver 148. The controller 142 can also indicate the location of the mobile device 126 based on the high frequency audio output 156 sent using the loudspeaker 140 of the mobile device 126. It should be noted that the illustrated system 100 is merely an example, and more elements, fewer elements, and / or elements positioned in a different manner may be used.

[0021] More specifically, the microphone array 104 may include a plurality of microphone elements arranged so that sounds in the listening area may arrive at the microphone elements at different times. These timing differences may be used to determine the direction from which the sounds are received. The A / D converter 106 receives an audio input signal from the microphone array 104. The A / D converter 106 converts the received signal from an analog format to a digital signal in a digital format for further processing by the audio processor 108.

[0022] Although only one audio processor is shown, one or more audio processors 108 may be included in the smart speaker 102. The audio processor 108 may be one or more computing devices capable of processing audio and / or video signals, such as a computer processor, a microprocessor, a digital signal processor, or any other device, a series of devices, or other mechanism capable of performing logical operations. The audio processor 108 may operate in association with the memory 110 to execute instructions stored in the memory 110. The instructions may be in the form of software, firmware, computer code, or some combination thereof. The memory 110 may be any form of one or more data storage devices, such as volatile memory, non-volatile memory, electronic memory, magnetic memory, optical memory, or any other form of data storage device. In addition to instructions, operating parameters and data may also be stored in the memory 110.

[0023] The audio processor 108 may also be configured to provide an audio output signal including media content or other audio to be provided from the smart speaker 102. The audio processor 108 may also filter the audio output according to filter settings received by the audio processor 108. The D / A converter 112 receives the digital output signal from the audio processor 108 and converts the digital output signal from a digital format to an output signal in an analog format. The output signal may then be used by an amplifier 114 or other analog components for further processing.

[0024] The amplifier 114 may be any circuit or stand-alone device that receives an audio input signal of relatively small amplitude and outputs a similar audio signal of relatively large amplitude. The audio input signal may be received by the amplifier 114 and output to the loudspeaker 116 on one or more connections. In addition to amplifying the amplitude of the audio signal, the amplifier 114 may also include signal processing capabilities such as phase shifting, adjusting frequency equalization, adjusting delay, or performing any other form of manipulation or adjustment on the audio signal to prepare for provision to the loudspeaker 116. As described above, the signal processing functionality may additionally or alternatively occur within the domain of the audio processor 108. Moreover, the amplifier 114 may include the ability to adjust the volume, balance, and / or attenuation of the audio signal provided to the loudspeaker 116. In an optional example, the loudspeaker 116 may include the amplifier 114 so that the loudspeaker 116 is self-powered.

[0025] The loudspeakers 116 can have various sizes and can operate in various frequency ranges. Each of the loudspeakers 116 can include a single transducer, or in other cases can include multiple transducers. The loudspeakers 116 can also operate in different frequency ranges, such as a subwoofer, a woofer, a midrange speaker, and a tweeter. Multiple loudspeakers 116 can be included in the smart speaker 102.

[0026] The controller 118 may include various types of computing devices that support performing the functions of the smart speaker 102 described herein. In one example, the controller 118 may include one or more processors 120 configured to execute computer instructions and a storage medium 122 on which computer-executable instructions and / or data may be maintained. Computer-readable storage media (also referred to as processor-readable media or storage devices 122) include any non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by one or more processors 120). Typically, the processor 120 receives instructions and / or data, for example, from a storage device 122, etc., to a memory, and uses the data to execute the instructions, thereby performing one or more processes, including one or more of the processes described herein. Computer executable instructions may be compiled or interpreted from a computer program created using a variety of programming languages ​​and / or techniques including, but not limited to (alone or in combination): JAVA, C, C++, C#, ASSEMBLY, FORTRAN, PASCAL, VISUAL BASIC, PYTHON, JAVA SCRIPT, PERL, PL / SQL, etc.

[0027] As shown, the controller 118 may include a wireless transceiver 124 or other network hardware configured to facilitate communication between the controller 118 and other networked devices. As one possibility, the wireless transceiver 124 may be a Wi-Fi transceiver configured to connect to a local wireless network to access a communication network. As another possibility, the wireless transceiver 124 may be a cellular network transceiver configured to transmit data over a cellular telephone network.

[0028] On the mobile device 126, the microphone 128 can provide a signal based on the received audio to the A / D converter 130 to convert from an analog format to a digital signal for further processing by the audio processor 134. Although only one audio processor is shown, one or more audio processors 134 may be included in the mobile device 126. Like the audio processor 108, the audio processor 134 may be one or more computing devices capable of processing audio and / or video signals, such as a computer processor, a microprocessor, a digital signal processor, or any other device, a series of devices, or other mechanisms capable of performing logical operations. The audio processor 108 may operate in association with the memory 132 to execute instructions stored in the memory 132. The instructions may be in the form of software, firmware, computer code, or some combination thereof. The memory 132 may be any form of one or more data storage devices, such as volatile memory, non-volatile memory, electronic memory, magnetic memory, optical memory, or any other form of data storage device. In addition to instructions, operating parameters and data may also be stored in the memory 132.

[0029] The audio processor 134 may also be configured to provide an audio output signal including media content or other audio to be provided from the mobile device 126. The D / A converter 136 receives the digital output signal from the audio processor 134 and converts the digital output signal from a digital format to an output signal in an analog format. Similar to as discussed with respect to elements 114 and 116 of the smart speaker 102, the output signal is then available for use by an amplifier 138 or other analog component for further processing and output by a loudspeaker 140.

[0030] The controller 142 may include various types of computing devices that support performing the functions of the mobile device 126 described herein. In one example, the controller 142 may include one or more processors 144 configured to execute computer instructions and a storage medium 146 on which computer executable instructions and / or data may be maintained. As shown, the controller 142 also includes a wireless transceiver 148 or other network hardware configured to facilitate communication between the controller 142 and other networked devices (such as the smart speaker 102).

[0031] The mobile device 126 may also include a human machine interface (HMI) 150. In some examples, the HMI 150 may include a touch screen display that may be used to display information and also receive user input. The HMI 150 may also include other controls and / or displays that may be used to receive user input and provide input to the user.

[0032] The listening area response map 152 is a data structure configured to store equalization information corresponding to a location within the listening area. For example, the listening area response map 152 may indicate a low frequency response for each of a plurality of locations in the listening area. Additionally or alternatively, the listening area response map 152 may include equalization or other filtering settings that may be used to correct the low frequency response associated with each of a plurality of locations in the listening area. The listening area response map 152 may be stored to the storage device 122 of the smart speaker 102.

[0033] The frequency test audio 154 is an audio output provided by the microphone 116 of the smart speaker 102 based on a frequency test signal. The frequency test signal may be a frequency sweep, a test tone, or other test signal that may be used to determine the room frequency response of the microphone 116 at the measurement location.

[0034] The high frequency audio output 156 is a high frequency audio output provided by the microphone 140 of the mobile device 126. The high frequency audio output 156 may be provided in the form of one or more pulses, pulse trains, chirps, sweeps, or other forms of audio output that may be used to determine the starting position of the high frequency audio output 156. In many examples, the high frequency audio output 156 may be at one or more ultrasonic frequencies above the typical human hearing range so that it can be played without being perceived by the listener. In some cases, the high frequency audio output 156 may be added to the existing audio output of the microphone 140 in order to mask the sound of the high frequency audio output 156.

[0035] The frequency correction application 158 is an example of an application installed to the storage device 122 of the smart speaker 102. When executed by the smart speaker 102, the frequency correction application 158 can be programmed to cause the smart speaker 102 to perform a learning mode and a playback mode of operation in which a listening zone response map 152 is created for a listening zone and in which the listening zone response map 152 is used to filter the output of the smart speaker 102. Figure 3 and Figure 5 Other aspects of the operation of the frequency correction application 158 are described.

[0036] The listener application 160 is an example of an application installed to the storage device 146 of the mobile device 126. When executed by the mobile device 126, the listener application 160 can be programmed to cause the mobile device 126 to perform a learning mode of operation in which frequency measurements are made based on receiving the frequency test audio 154 at the microphone 128 of the mobile device 126, and a signal including the frequency measurement is transmitted from the mobile device 126 to the smart speaker 102. The listener application 160 can also be programmed to cause the mobile device 126 to play the high frequency audio output 156 via the loudspeaker 140 for receipt by the microphone array 104 of the smart speaker 102 to allow the smart speaker 102 to locate the mobile device 126 in a listening area. Figure 2 and Figure 4 Other aspects of the operation of listener application 160 are described.

[0037] Figure 2 An exemplary process 200 for operating a mobile device 126 when generating a listening zone response map 152 is shown. In one example, the process 200 can be performed by executing a listener application 160 in a learning mode by the mobile device 126. For example, the listener application 160 can transition to the learning mode in response to a user input to the HMI 150 of the mobile device 126 (e.g., via a menu selection) or in response to a command received by the mobile device 126 from the smart speaker 102 to begin learning (e.g., via a wireless signal wirelessly received from the wireless transceiver 148 of the mobile device 126 to the wireless transceiver 124 of the smart speaker 102 over WiFi or another protocol, encoded into an audio format, and provided by the loudspeaker 116 to be received by the microphone 128 and interpreted by the mobile device 126).

[0038] At operation 202, the mobile device 126 sends a request to play the frequency test audio 154 to the smart speaker 102 via the loudspeaker 116 of the smart speaker 102. In one example, the request may be sent as a wireless signal from the wireless transceiver 148 of the mobile device 126 to the wireless transceiver 124 of the smart speaker 102 via WiFi or another protocol. In another example, the request may be encoded into an audio format and may be sent from the loudspeaker 140 to be received by the microphone array 104 of the smart speaker 102. In yet another example, if the mobile device 126 is in a learning mode, the mobile device 126 may listen to the frequency test audio 154 and may analyze the signal after receiving it without sending an additional request to the smart speaker 102.

[0039] At 204, the mobile device 126 measures the listening area response at the location of the mobile device 126. In one example, the frequency test audio 154 is received by the microphone 128 of the mobile device 126, which is used to record amplitude measurements of the frequencies of the audio included in the frequency test audio 154 provided by the smart speaker 102. For example, these measurements can be used to identify low frequency response characteristics of the current location of the mobile device 126 in the listening area.

[0040] At 206, the mobile device 126 sends the listening area response to the smart speaker 102. In one example, the listening area response may be sent as a wireless signal over WiFi or another protocol from the wireless transceiver 148 of the mobile device 126 to the wireless transceiver 124 of the smart speaker 102. In another example, the listening area response may be encoded into an audio format and may be sent from the loudspeaker 140 to be received by the microphone array 104 of the smart speaker 102.

[0041] At operation 208, the mobile device 126 transmits the high frequency audio output 156 for receipt by the smart speaker 102. In one example, the mobile device 126 may utilize the loudspeaker 140 to transmit the high frequency audio output 156 for pickup by the microphone array 104 of the smart speaker 102 to allow the smart speaker 102 to attempt to locate the mobile device 126 within the listening area. In some cases, the mobile device 126 explicitly provides the high frequency audio output 156 in a predefined manner before, concurrently with, and / or after sending the listening area response data to the smart speaker 102. In another example, the mobile device 126 periodically provides the high frequency audio output 156 independent of transmitting the listening area response to the smart speaker 102.

[0042] At 210, the mobile device 126 determines whether to learn listening zone response data for additional locations. In one example, the listener application 160 can provide a prompt to the HMI 150 of the mobile device 126 to ask the user of the mobile device 126 whether the user has other locations in the listening area to measure. If the HMI 150 receives input indicating that additional locations are to be measured, control returns to operation 202. If not, the mobile device 126 can notify the smart speaker 102 that learning is complete (e.g., via a wireless signal or audio communication), and the process 200 ends.

[0043] Figure 3 An exemplary process 300 is shown for operating a smart speaker 102 when generating a listening area response map 152. In one example, the process 300 can be performed by the smart speaker 102 executing the frequency correction application 158 in a learning mode.

[0044] At operation 302, the smart speaker 102 provides a frequency test signal as an audio output. In one example, the smart speaker 102 plays the frequency test audio 154 via the loudspeaker 116 of the smart speaker 102 in response to receiving the request at operation 202. In another example, the smart speaker 102 plays the frequency test audio 154 in response to an instruction to check for responses at different locations (e.g., such as discussed at operation 210), or automatically plays the frequency test audio 154 in response to entering a learning mode.

[0045] At operation 304, the smart speaker 102 receives the room response information from the mobile device 126. In one example, the smart speaker 102 receives the information sent at operation 206 of process 200.

[0046] At 306, the smart speaker 102 identifies the location of the mobile device 126. In one example, the smart speaker 102 receives the high frequency audio output 156 sent at operation 208 of the process 200 and uses the high frequency audio output 156 to determine the location of the mobile device 126. For example, the smart speaker 102 can calculate the angle of incidence of the received audio relative to the microphone array 104 using the time and phase differences of the signals received from each of the microphones of the microphone array 104. It should be noted that it is not critical to identify the actual location of the mobile device 126 within the listening area. Instead, it is more important that the location determination is repeatable so that mapping of the location can be used at a later time to identify situations where the mobile device 126 is located in the same location.

[0047] At operation 308, the smart speaker 102 generates a room correction for the identified location of the mobile device 126. In one example, the room correction may be determined as a filter to be applied to the audio output to reduce nonlinearity of the response in the room response information received at operation 304. As one possibility, the room correction may be determined as an equalization in the form of an inverse of the difference between the room response information and a target response (e.g., a flat response, a target equalization, etc.). As another possibility, the room correction may be determined as a set of one or more parametric filters, each of which includes a frequency center point, a gain (positive or negative), and a Q that determines the width of the filter. For example, the frequency correction application 158 may define one or more parametric EQ settings using an algorithm designed to minimize the difference between the measured response and the target response.

[0048] At 310, the smart speaker 102 updates the listening zone response map 152. In one example, the smart speaker 102 saves the room correction determined at operation 308 according to the position index determined at operation 306. After operation 310, the process 300 ends.

[0049] Figure 4 An exemplary process 400 is shown for operating a mobile device 126 to send location updates to a smart speaker 102. In one example, the process 400 may be performed by the mobile device 126 executing the listener application 160 in playback mode.

[0050] At 402, the mobile device 126 determines whether the mobile device 126 is in playback mode. In one example, the playback mode may be entered in response to a user request of the mobile device 126 (e.g., via the HMI 150) to play back audio content. The playback mode may be exited in response to the completion of the playback. In another example, if the smart speaker 102 is not identified as being in the learning mode discussed in detail above, the mobile device 126 may determine that the smart speaker 102 is in playback mode. If the smart speaker 102 is in playback mode, control passes to operation 404. Otherwise, control remains at operation 402.

[0051] At operation 404, the mobile device 126 determines whether an event has occurred that causes the mobile device 126 to send a location update. For example, the listener application 160 may periodically send location updates and may accordingly determine to send updates in response to the expiration of a timer. In another example, the listener application 160 may additionally or alternatively send a location update in response to identifying movement of the mobile device 126. For example, the mobile device 126 may include one or more accelerometers that provide signals indicating the acceleration of the mobile device 126 in one or more directions. If one or more such events have occurred, control passes to operation 406.

[0052] At 406, similar to as discussed above with respect to operation 208 of process 200, the mobile device 126 transmits the high frequency audio output 156 for receipt by the smart speaker 102. Such updates may be used to allow the smart speaker 102 to track the location of the mobile device 126, and therefore the location of the user of the mobile device 126. After operation 406, process 400 proceeds to operation 402.

[0053] Figure 5 An exemplary process 500 is shown for operating a smart speaker 102 to filter audio output based on the location of a mobile device 126. In one example, the process 300 may be performed by the smart speaker 102 executing a frequency correction application 158 in playback mode.

[0054] At 502, the smart speaker 102 determines whether the smart speaker 102 is in playback mode. In one example, the playback mode may be entered in response to a user request of the mobile device 126 (e.g., via the HMI 150) for the mobile device 126 to play back audio content. The playback mode may be exited in response to the completion of the playback. In another example, if the smart speaker 102 is not identified as being in the learning mode discussed in detail above, the smart speaker 102 is in playback mode. If the smart speaker 102 is in playback mode, control passes to operation 504. Otherwise, control remains at operation 502.

[0055] At 504, the smart speaker 102 identifies the location of the mobile device 126. In one example, the high frequency audio output 156 received by the microphone array 104 of the smart speaker 102 can be compared to the mapped location of the listening area response map 152 saved using a process such as processes 200 and 300.

[0056] At operation 506, the smart speaker 102 retrieves the filter parameters for the listening position of the mobile device 126. For example, if a matching position is identified at operation 502, the filter settings for that position are retrieved from the listening area response map 152. However, if no match is identified, other settings for the position may be used. For example, the smart speaker 102 may utilize an average of the filter parameters for all positions across the listening area response map 152.

[0057] At 508, the smart speaker 102 applies the filtering parameters for the listening position to the audio stream. At 510, the smart speaker 102 provides the audio stream to the microphone 116 of the smart speaker 102 to generate an audio output. Thus, the audio output of the smart speaker 102 can be filtered according to the current position of the mobile device 126. After operation 508, control returns to operation 502.

[0058] Figure 6 An exemplary diagram 600 of a smart speaker 102 filtering audio output for a mobile device 126 at a first location in a listening area is shown. As shown, the mobile device 126 is located at the first location in the listening area and transmits a high-frequency audio output 156 that allows the smart speaker 102 to identify the mobile device 126 as being located at the first location. In response to the identification, the smart speaker 102 uses the listening area response map 152 to filter the audio output 602 provided by the smart speaker 102 according to the filtering associated with the first location.

[0059] Figure 7An exemplary diagram 700 of the smart speaker 102 filtering the audio output for the mobile device 126 at a second location in the listening area is shown. As shown, the mobile device 126 is now located at the second location in the listening area. Using the high frequency audio output 156 from the mobile device 126, the smart speaker 102 now recognizes that the mobile device 126 is located at the second location and uses the listening area response map 152 to filter the audio output 602 according to the filtering associated with the second location.

[0060] Figure 8 An exemplary diagram 800 of the smart speaker 102 filtering audio outputs for multiple mobile devices 126A and 126B is shown. Notably, mobile device 126A is located at a first position and mobile device 126B is located at a second position. Using the high-frequency audio output 156A from mobile device 126A and the high-frequency audio output 156B from mobile device 126B, the smart speaker 102 identifies that mobile device 126A is located at a first position and mobile device 126B is located at a second position. Therefore, since the smart speaker 102 cannot simultaneously apply a filtering program for the first position and a filtering program for the second position, the smart speaker 102 can alternatively provide a combined filtering program, such as an average of the equalizations for the first position and the second position. Alternatively, the smart speaker 102 can provide a default equalization, which can, for example, be an average of all equalizations for listening recorded in the response diagram 152.

[0061] Other variations on the system 100 are possible. For example, in determining the average equalization, the system 100 may weight the equalization for the amount of time the user spends at various locations within the listening area when determining the average equalization. For example, if the user spends 60% of his time at one location and 40% of his time at a second location, then if the user's location cannot be determined, the smart speaker 102 may utilize an average equalization that is a weighted average of 3 / 5 equalization at the first location and 2 / 5 equalization at the second area.

[0062] The computing devices described herein (such as the audio processors 108, 134 and the controllers 118, 142) generally include computer-executable instructions, which can be executed by one or more computing devices such as those listed above. The computer-executable instructions can be compiled or interpreted by a computer program created using a variety of programming languages ​​and / or technologies, including, alone or in combination, but not limited to JAVA. TM, JAVASCRIPT, C, C++, C#, VISUAL BASIC, JAVA SCRIPT, PYTHON, PERL, etc. In general, a processor (e.g., a microprocessor) receives instructions, such as from a memory, a computer-readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. A variety of computer-readable media can be used to store and transmit such instructions and other data.

[0063] With respect to the processes, systems, methods, heuristics, etc. described herein, it should be understood that although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes may be practiced with the described steps performed in an order other than the order described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the description of the processes herein is provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claims.

[0064] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the present invention. On the contrary, the words used in this specification are descriptive rather than restrictive, and it should be understood that various changes can be made without departing from the spirit and scope of the present invention. In addition, the features of the various implemented embodiments can be combined to form other embodiments of the present invention.

Claims

1. A smart speaker device for acoustic listening area mapping and frequency correction, comprising: a non-transitory storage device configured to maintain a listening area response map indicating filter settings corresponding to each of a plurality of locations within the listening area; Microphone array; loudspeaker; as well as A controller programmed to execute a frequency correction application to: identifying a current location of the mobile device within the listening area based on ultrasonic audio received from the mobile device to the microphone array, accessing the listening area response map to retrieve filter settings corresponding to the current position, In response to the current location lacking a corresponding filter setting in the listening area response map or the current location being unidentifiable, a weighted average of the filter settings for the most likely locations in the listening area response map is used as the filter setting to be applied to the audio stream, wherein filter settings corresponding to locations where the mobile device spent more time during the learning phase are given a greater weight, and The filter settings are applied to the audio stream to be output to the loudspeaker to correct a frequency response of the loudspeaker at the current position of the mobile device.

2. The smart speaker device of claim 1 , wherein each of the filter settings includes equalization to be adjusted for a difference in the response of the listening area compared to a target frequency response, and wherein a weighted average equalization is calculated based on the equalization of the filter settings.

3. The smart speaker device of claim 1 , wherein the filter setting comprises a set of one or more parametric filters, each parametric filter comprising a frequency center point of the parametric filter, a gain level of the parametric filter, and a Q indicating a width of the parametric filter.

4. The smart speaker device of claim 1, wherein the current location of the mobile device is determined by triangulating the ultrasonic audio using signals received from a plurality of microphones of the microphone array.

5. The smart speaker device of claim 1, wherein the controller is further programmed to: during a learning phase, outputting frequency test audio from the loudspeaker for receipt by the mobile device at the current location, receiving information indicating a room response at the current location, generating a room correction for the current position based on the information indicative of the room response, the room correction indicating filter settings for the current position, updating the listening area response map to indicate the filter settings as corresponding to the current position, and The amount of time the mobile device was at the current location during the learning phase is obtained for use in calculating the weights of the filter settings corresponding to the current location.

6. The smart speaker device of claim 1, wherein the controller is further programmed to: identifying a second current location of a second mobile device in the listening area based on second ultrasonic audio received from the microphone array from the second mobile device, accessing the listening area response map to retrieve a second filter setting corresponding to the second current position, and An average of the filter setting and the second filter setting is applied to the audio stream to be output to the loudspeaker.

7. A smart speaker device for acoustic listening area mapping and frequency correction, comprising: a non-transitory storage device configured to maintain a listening area response map indicating filter settings corresponding to each of a plurality of locations within the listening area; Microphone array; loudspeaker; as well as A controller programmed to execute a frequency correction application to: identifying a current location of the mobile device within the listening area based on ultrasonic audio received from the mobile device to the microphone array, outputting a frequency test audio from the loudspeaker for reception by the mobile device, receiving information indicating a room response at the current location from the mobile device, obtaining an amount of time that the mobile device is at the current location during the learning phase for use in calculating a weighting of a filter setting corresponding to the current location, such that in response to a lack of a corresponding filter setting for the identified location in the listening area response map or the identified location being unidentifiable, a weighted average of the filter settings for the most likely locations in the listening area response map is used as the filter setting for application, wherein filter settings corresponding to locations where the mobile device spends more time during the learning phase are given a greater weighting, generating a room correction for the current position based on the information indicative of the room response, the room correction indicating filter settings for the current position, and The listening area response map is updated to indicate the filter settings as corresponding to the current position.

8. The smart speaker device of claim 7, wherein the controller is further programmed to apply the filter settings to the audio stream to be output to the loudspeaker to correct the frequency response of the loudspeaker at the current position of the mobile device.

9. The smart speaker device of claim 7, wherein the controller is further programmed to determine the room correction as an equalization in the form of an inverse of a difference between the information indicative of the room response and a target response.

10. The smart speaker device of claim 9, wherein the target response is a flat response.

11. The smart speaker device of claim 9, wherein the target response is a predefined equalization.

12. The smart speaker device of claim 7, wherein the controller is further programmed to: identifying a second current location of the mobile device in the listening area based on second ultrasonic audio received from the mobile device to the microphone array, outputting a second frequency test audio from the loudspeaker for reception by the mobile device, receiving second information indicating a room response at the second current location from the mobile device, generating a second room correction for the second current position based on the second information indicative of the room response, the room correction indicating a second filter setting for the current position, and The listening area response map is updated to indicate the second filter setting as corresponding to the second current position.

13. A method for acoustic listening area mapping and frequency correction, comprising: identifying a current location of the mobile device in a listening area based on ultrasonic audio received from the mobile device to a microphone array of a smart speaker device; accessing a listening area response map stored to a memory of the smart speaker device to retrieve a filter setting corresponding to the current position, the listening area response map indicating a filter setting corresponding to each of a plurality of positions within a listening area; In response to the current location lacking a corresponding filter setting in the listening area response map, utilizing a weighted average of the filter settings for the most likely locations in the listening area response map as the filter setting to apply, wherein filter settings corresponding to locations where the mobile device spent more time during a learning phase are given greater weighting; as well as The filter settings are applied to an audio stream to be output to a microphone of the smart speaker device to correct a frequency response of the microphone at the current position of the mobile device.

14. The method of claim 13, wherein the filtering settings include equalization that adjusts for differences in the listening area's response compared to a target frequency response.

15. The method of claim 13, wherein the filter settings comprise a set of one or more parametric filters, each parametric filter comprising a frequency center point of the parametric filter, a gain level of the parametric filter, and a Q indicating a width of the parametric filter.

16. The method of claim 13, wherein the current location of the mobile device is determined by triangulating the ultrasonic audio using signals received from a plurality of microphones of the microphone array.

17. The method of claim 13, further comprising: outputting frequency test audio from the loudspeaker for receipt by the mobile device at the current location, receiving information indicating a room response at the current location, generating a room correction for the current position based on the information indicative of the room response, the room correction indicating filter settings for the current position, updating the listening area response map to indicate the filter settings as corresponding to the current position, and The amount of time the mobile device was at the current location during the learning phase is obtained for use in calculating the weights of the filter settings corresponding to the current location.

18. The method of claim 13, further comprising: identifying a second current location of a second mobile device in the listening area based on second ultrasonic audio received from the microphone array from the second mobile device; accessing the listening area response map to retrieve a second filter setting corresponding to the second current position; as well as An average of the filter setting and the second filter setting is applied to the audio stream to be output to the loudspeaker.

Images